Reproduction Report on "Unpacking p-Hacking and Publication Bias"

Alexandra de Gendre and Nicolás Salamanca

No 305, I4R Discussion Paper Series from The Institute for Replication (I4R)

Abstract: Brodeur, Carrell, Figlio and Lusher (2023) present the first descriptive evidence on p-hacking and publication bias throughout the publication process in economics. Using data from the Journal of Human Resources, they show that 1) p-values in initial submissions present humping around the 10 and 5% significance thresholds, 2) the distribution of desk rejections shows more humping than that of papers sent for review, 3) favorable reviewer recommendations are positively related to statistical significance and 4) overall, the p-value distribution of initial and final versions of accepted drafts appear similar. We first conduct a mostly successful computational reproduction of those results. Next, we conduct a series of robustness replication exercises and show that, while the authors' main results are robust to random removals of small amounts of data and to bandwidth choices, they are sensitive to removing papers that contribute many estimates, to some key modelling choices in density discontinuity tests, and to some conceptually equivalent variations of their tests. Last, we complement the authors' results by providing statistical tests of their main hypotheses. Our results support the claims that 1) the distribution of p-values of deskrejected papers displays greater bunching than that of papers sent for review, and 2) the distributions of p-values of initial submissions and final drafts are similar. Overall, our findings lend support to the authors' main claims, suggesting that after excluding desk-rejected papers, the peer review process does not seem to affect p-hacking or publication bias.

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/342310/1/I4R-DP305.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:305

Access Statistics for this paper

More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().