Unpacking P-hacking and Publication Bias

Abel Brodeur, Scott Carrell, David Figlio and Lester Lusher

American Economic Review, 2023, vol. 113, issue 11, 2974-3002

Abstract: We use unique data from journal submissions to identify and unpack publication bias and p-hacking. We find initial submissions display significant bunching, suggesting the distribution among published statistics cannot be fully attributed to a publication bias in peer review. Desk-rejected manuscripts display greater heaping than those sent for review; i.e., marginally significant results are more likely to be desk rejected. Reviewer recommendations, in contrast, are positively associated with statistical significance. Overall, the peer review process has little effect on the distribution of test statistics. Lastly, we track rejected papers and present evidence that the prevalence of publication biases is perhaps not as prominent as feared.

JEL-codes: A11 A14 C13 L82 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (14)

Downloads: (external link)
https://www.aeaweb.org/doi/10.1257/aer.20210795 (application/pdf)
https://doi.org/10.3886/E192271V1 (text/html)
https://www.aeaweb.org/doi/10.1257/aer.20210795.appx (application/pdf)
https://www.aeaweb.org/doi/10.1257/aer.20210795.ds (application/zip)
Access to full text is restricted to AEA members and institutional subscribers.

Related works:
Working Paper: Unpacking P-Hacking and Publication Bias (2023)
Working Paper: Unpacking P-Hacking and Publication Bias (2023)
Working Paper: Unpacking P-Hacking and Publication Bias (2023)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aea:aecrev:v:113:y:2023:i:11:p:2974-3002

Ordering information: This journal article can be ordered from
https://www.aeaweb.org/journals/subscriptions

DOI: 10.1257/aer.20210795

Access Statistics for this article

American Economic Review is currently edited by Esther Duflo

More articles in American Economic Review from American Economic Association Contact information at EDIRC.
Bibliographic data for series maintained by Michael P. Albert ().