{"title":"方法:经济学因果分析中的p-Hacking和发表偏倚","authors":"A. Brodeur, Nikolai Cook, A. Heyes","doi":"10.1257/aer.20220277","DOIUrl":null,"url":null,"abstract":"In Brodeur, Cook, and Heyes (2020) we present evidence that instrumental variable (and to a lesser extent difference-in-difference) articles are more p-hacked than randomized controlled trial and regression discontinuity design articles. We also find no evidence that (i) articles published in the top five journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz and Pütz (2022) apply a novel adjustment to address rounding errors. They successfully replicate our results with the exception of our shakiest finding: after adjusting for rounding errors, bunching of test statistics for difference-in-difference articles is now smaller around the 5 percent level (and coincidentally larger at the 10 percent level). (JEL A14, C12, C52)","PeriodicalId":48472,"journal":{"name":"American Economic Review","volume":null,"pages":null},"PeriodicalIF":10.5000,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Reply\",\"authors\":\"A. Brodeur, Nikolai Cook, A. Heyes\",\"doi\":\"10.1257/aer.20220277\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In Brodeur, Cook, and Heyes (2020) we present evidence that instrumental variable (and to a lesser extent difference-in-difference) articles are more p-hacked than randomized controlled trial and regression discontinuity design articles. We also find no evidence that (i) articles published in the top five journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz and Pütz (2022) apply a novel adjustment to address rounding errors. They successfully replicate our results with the exception of our shakiest finding: after adjusting for rounding errors, bunching of test statistics for difference-in-difference articles is now smaller around the 5 percent level (and coincidentally larger at the 10 percent level). (JEL A14, C12, C52)\",\"PeriodicalId\":48472,\"journal\":{\"name\":\"American Economic Review\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":10.5000,\"publicationDate\":\"2022-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"American Economic Review\",\"FirstCategoryId\":\"96\",\"ListUrlMain\":\"https://doi.org/10.1257/aer.20220277\",\"RegionNum\":1,\"RegionCategory\":\"经济学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ECONOMICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Economic Review","FirstCategoryId":"96","ListUrlMain":"https://doi.org/10.1257/aer.20220277","RegionNum":1,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ECONOMICS","Score":null,"Total":0}
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Reply
In Brodeur, Cook, and Heyes (2020) we present evidence that instrumental variable (and to a lesser extent difference-in-difference) articles are more p-hacked than randomized controlled trial and regression discontinuity design articles. We also find no evidence that (i) articles published in the top five journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz and Pütz (2022) apply a novel adjustment to address rounding errors. They successfully replicate our results with the exception of our shakiest finding: after adjusting for rounding errors, bunching of test statistics for difference-in-difference articles is now smaller around the 5 percent level (and coincidentally larger at the 10 percent level). (JEL A14, C12, C52)
期刊介绍:
The American Economic Review (AER) stands as a prestigious general-interest economics journal. Founded in 1911, it holds the distinction of being one of the nation's oldest and most esteemed scholarly journals in economics. With a commitment to academic excellence, the AER releases 12 issues annually, featuring articles that span a wide spectrum of economic topics.