A Protocol for Structured Robustness Reproductions and Replicability Assessments

Jörg Peters, Abel Brodeur, Anna Dreber Almenberg (), Magnus Johannesson, Florian Neubauer () and Julian Rose ()
Additional contact information
Anna Dreber Almenberg: Stockholm School of Economics
Florian Neubauer: RWI – Leibniz Institute for Economic Research and University Duisburg-Essen
Julian Rose: RWI

No 17691, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: Robustness reproductions and replicability discussions are on the rise in response to concerns about a potential credibility crisis in economics. This paper proposes a protocol to structure reproducibility and replicability assessments, with a focus on robustness. Starting with a computational reproduction upon data availability, the protocol encourages replicators to prespecify robustness tests, prior to implementing them. The protocol contains three different reporting tools to streamline the presentation of results. Beyond reproductions, our protocol assesses adherence to the pre-analysis plans in the replicated papers as well as external and construct validity. Our ambition is to put often controversial debates between replicators and replicated authors on a solid basis and contribute to an improved replication culture in economics.

Keywords: research transparency; robustness; reproducibility; replication; meta-science (search for similar items in EconPapers)
JEL-codes: A11 C18 (search for similar items in EconPapers)
Pages: 32 pages
Date: 2025-02
New Economics Papers: this item is included in nep-sog
References: View complete reference list from CitEc
Citations:

Published - published online in: Open , 21 March 2025

Downloads: (external link)
https://docs.iza.org/dp17691.pdf (application/pdf)

Related works:
Working Paper: A Protocol for Structured Robustness Reproductions and Replicability Assessments (2024)
Working Paper: A protocol for structured robustness reproductions and replicability assessments (2024)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp17691

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().