Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?

Abel Brodeur, Nikolai Cook, Jonathan S. Hartley and Anthony Heyes ()
Additional contact information
Jonathan S. Hartley: Stanford University
Anthony Heyes: University of Ottawa

No 15476, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: Randomized controlled trials (RCTs) are increasingly prominent in economics, with pre-registration and pre-analysis plans (PAPs) promoted as important in ensuring the credibility of findings. We investigate whether these tools reduce the extent of p-hacking and publication bias by collecting and studying the universe of test statistics, 15,992 in total, from RCTs published in 15 leading economics journals from 2018 through 2021. In our primary analysis, we find no meaningful difference in the distribution of test statistics from pre-registered studies, compared to their non-pre-registered counterparts. However, pre-registerd studies that have a complete PAP are significantly less p-hacked. This results point to the importance of PAPs, rather than pre-registration in itself, in ensuring credibility.

Keywords: research credibility; publication bias; p-hacking; pre-registration; pre-analysis plan (search for similar items in EconPapers)
JEL-codes: B41 C13 C40 C93 (search for similar items in EconPapers)
Pages: 44 pages
Date: 2022-08
New Economics Papers: this item is included in nep-exp and nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (8)

Published - published as 'Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement' in: Journal of Political Economy Microeconomics, 2024. 2 (3), 527–561

Downloads: (external link)
https://docs.iza.org/dp15476.pdf (application/pdf)

Related works:
Working Paper: Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias? (2022)
Working Paper: Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias? (2022)
Working Paper: Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias? (2022)
Working Paper: Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias? (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp15476

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().