Mass Reproducibility in Psychological Science

Abel Brodeur, Ghina Abdul Baki and Luna Fazio

No 326, I4R Discussion Paper Series from The Institute for Replication (I4R)

Abstract: Concerns about the credibility and reproducibility of published findings have motivated numerous initiatives to improve transparency in psychology. In this article, we present evidence from a large-scale reproducibility initiative conducted in partnership between Psychological Science and the Institute for Replication (I4R). Our project systematically reproduces empirical results published in the journal between 2024 and 2025 using independent research teams. Through a combination of Replication Games and researcher-led initiatives, 67 independent teams examined 44% of the articles (i.e., 64 articles) published during our time frame, documenting data and code availability, checking for coding errors, and deviations from preregistration, and computationally reproducing the main numerical results. We find that the majority of articles provide cleaned data and analysis scripts, though raw data and data-cleaning scripts are shared far less often. Computational reproducibility depends strongly on the starting point: 58.2% of articles are fully reproducible from final analysis data, but only 28.4% are fully reproducible from raw data. Teams identify coding errors or statistical reporting inconsistencies in 63% of articles, although most issues are minor; 40.5% of the articles flagged with coding or statistical reporting issues contain discrepancies with potentially major implications for estimates or interpretation. Preregistration is common (74.6% of articles), yet 84% of preregistered articles deviate from their pre-analysis plan, and nearly half of these deviations go unacknowledged. Our findings provide new insights into the reliability of empirical results in a leading psychology journal and highlight the value of structured mass reproducibility initiatives.

Keywords: reproducibility; replication; open science; psychology; research transparency (search for similar items in EconPapers)
Date: 2026
Note: See pp. 4-7 for full list of authors.
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/344024/1/I4R-DP326.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:326

Access Statistics for this paper

More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().