方法:经济学因果分析中的p-Hacking和发表偏倚

IF 10.5 1区 经济学 Q1 ECONOMICS American Economic Review Pub Date : 2022-09-01 DOI:10.1257/aer.20220277
A. Brodeur, Nikolai Cook, A. Heyes
{"title":"方法:经济学因果分析中的p-Hacking和发表偏倚","authors":"A. Brodeur, Nikolai Cook, A. Heyes","doi":"10.1257/aer.20220277","DOIUrl":null,"url":null,"abstract":"In Brodeur, Cook, and Heyes (2020) we present evidence that instrumental variable (and to a lesser extent difference-in-difference) articles are more p-hacked than randomized controlled trial and regression discontinuity design articles. We also find no evidence that (i) articles published in the top five journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz and Pütz (2022) apply a novel adjustment to address rounding errors. They successfully replicate our results with the exception of our shakiest finding: after adjusting for rounding errors, bunching of test statistics for difference-in-difference articles is now smaller around the 5 percent level (and coincidentally larger at the 10 percent level). (JEL A14, C12, C52)","PeriodicalId":48472,"journal":{"name":"American Economic Review","volume":null,"pages":null},"PeriodicalIF":10.5000,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Reply\",\"authors\":\"A. Brodeur, Nikolai Cook, A. Heyes\",\"doi\":\"10.1257/aer.20220277\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In Brodeur, Cook, and Heyes (2020) we present evidence that instrumental variable (and to a lesser extent difference-in-difference) articles are more p-hacked than randomized controlled trial and regression discontinuity design articles. We also find no evidence that (i) articles published in the top five journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz and Pütz (2022) apply a novel adjustment to address rounding errors. They successfully replicate our results with the exception of our shakiest finding: after adjusting for rounding errors, bunching of test statistics for difference-in-difference articles is now smaller around the 5 percent level (and coincidentally larger at the 10 percent level). (JEL A14, C12, C52)\",\"PeriodicalId\":48472,\"journal\":{\"name\":\"American Economic Review\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":10.5000,\"publicationDate\":\"2022-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"American Economic Review\",\"FirstCategoryId\":\"96\",\"ListUrlMain\":\"https://doi.org/10.1257/aer.20220277\",\"RegionNum\":1,\"RegionCategory\":\"经济学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ECONOMICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Economic Review","FirstCategoryId":"96","ListUrlMain":"https://doi.org/10.1257/aer.20220277","RegionNum":1,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ECONOMICS","Score":null,"Total":0}
引用次数: 1

摘要

在Brodeur、Cook和Heyes(2020)中,我们提出证据表明,工具变量(以及较小程度上的差异中差异)文章比随机对照试验和回归不连续设计文章更具p-hack性。我们也没有发现证据表明:(1)在排名前五的期刊上发表的文章不同;(ii)“修订及重新提交”程序可减轻问题;(iii)随着时间的推移,情况正在改善。Kranz和p兹(2022)应用了一种新的调整来解决舍入误差。他们成功地复制了我们的结果,除了我们最不稳定的发现:在调整了舍入误差之后,差中差文章的测试统计数据现在在5%的水平上更小(巧合的是,在10%的水平上更大)。(jel a14, c12, c52)
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Reply
In Brodeur, Cook, and Heyes (2020) we present evidence that instrumental variable (and to a lesser extent difference-in-difference) articles are more p-hacked than randomized controlled trial and regression discontinuity design articles. We also find no evidence that (i) articles published in the top five journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz and Pütz (2022) apply a novel adjustment to address rounding errors. They successfully replicate our results with the exception of our shakiest finding: after adjusting for rounding errors, bunching of test statistics for difference-in-difference articles is now smaller around the 5 percent level (and coincidentally larger at the 10 percent level). (JEL A14, C12, C52)
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
18.60
自引率
2.80%
发文量
122
期刊介绍: The American Economic Review (AER) stands as a prestigious general-interest economics journal. Founded in 1911, it holds the distinction of being one of the nation's oldest and most esteemed scholarly journals in economics. With a commitment to academic excellence, the AER releases 12 issues annually, featuring articles that span a wide spectrum of economic topics.
期刊最新文献
Urban Public Works in Spatial Equilibrium: Experimental Evidence from Ethiopia Household Labor Supply and the Value of Social Security Survivors Benefits Resisting Social Pressure in the Household Using Mobile Money: Experimental Evidence on Microenterprise Investment in Uganda Mental Models and Learning: The Case of Base-Rate Neglect Disentangling Moral Hazard and Adverse Selection
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1