Is economics self-correcting? Replications in the American Economic Review

Jörg Peters, Nathan Fiala and Florian Neubauer

No 1005, Ruhr Economic Papers from RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen

Abstract: Replication and constructive controversy are essential for scientific progress. This paper reviews the impact of all replications published as comments in the American Economic Review between 2010 and 2020. We investigate the citation rates of comments and whether a comment affects its original paper's citation rates. We find that most comments are barely cited, and they have no impact on the original papers' subsequent citations. This finding holds for original papers for which the comment diagnoses a substantive problem. We conclude from these citation patterns that replications do not update the economics literature. In an online opinion survey, we elicited viewpoints of both comment authors and original authors and find that in most cases, there is no consensus regarding the replication's success and to what extent the original paper's contribution sustains. This resonates with the conventional wisdom that robustness and replicability are hard to define in economics.

Keywords: Replication; citations; meta-science (search for similar items in EconPapers)
JEL-codes: A11 A14 (search for similar items in EconPapers)
Date: 2023
New Economics Papers: this item is included in nep-dem, nep-hme, nep-hpe and nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/269235/1/1838894071.pdf (application/pdf)

Related works:
Working Paper: Is Economics Self-Correcting? Replications in the American Economic Review (2023)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:rwirep:1005

DOI: 10.4419/96973171

Access Statistics for this paper

More papers in Ruhr Economic Papers from RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().