Unpacking P-Hacking and Publication Bias

Abel Brodeur, Scott Carrell, David Figlio and Lester Lusher

No 16369, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: We use unique data from journal submissions to identify and unpack publication bias and p-hacking. We find that initial submissions display significant bunching, suggesting the distribution among published statistics cannot be fully attributed to a publication bias in peer review. Desk-rejected manuscripts display greater heaping than those sent for review i.e. marginally significant results are more likely to be desk rejected. Reviewer recommendations, in contrast, are positively associated with statistical significance. Overall, the peer review process has little effect on the distribution of test statistics. Lastly, we track rejected papers and present evidence that the prevalence of publication biases is perhaps not as prominent as feared.

Keywords: publication bias; p-hacking; selective reporting (search for similar items in EconPapers)
JEL-codes: A11 C13 C40 (search for similar items in EconPapers)
Pages: 89 pages
Date: 2023-08
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (17)

Published - published in: American Economic Review, 2023, 113 (11), 2974-3002

Downloads: (external link)
https://docs.iza.org/dp16369.pdf (application/pdf)

Related works:
Journal Article: Unpacking P-hacking and Publication Bias (2023)
Working Paper: Unpacking P-Hacking and Publication Bias (2023)
Working Paper: Unpacking P-Hacking and Publication Bias (2023)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp16369

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().