The Meaning of Failed Replications: A Review and Proposal

Michael Clemens

No 9000, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: The welcome rise of replication tests in economics has not been accompanied by a single, clear definition of replication. A discrepant replication, in current usage of the term, can signal anything from an unremarkable disagreement over methods to scientific incompetence or misconduct. This paper proposes an unambiguous definition of replication, one that reflects currently common but unstandardized use. It contrasts this definition with decades of unsuccessful attempts to standardize terminology, and argues that many prominent results described as replication tests – in labor, development, and other fields of economics – should not be described as such. Adopting this definition can improve incentives for researchers, encouraging more and better replication tests.

Keywords: replication; robustness; transparency; open data; ethics; reproducible; replicate; misconduct; fraud; error; code; registry (search for similar items in EconPapers)
JEL-codes: B40 C18 C80 (search for similar items in EconPapers)
Pages: 27 pages
Date: 2015-04
New Economics Papers: this item is included in nep-ecm, nep-hme and nep-hpe
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (24)

Published - published in: Journal of Economic Surveys, 2017, 31 (1), 326-342

Downloads: (external link)
https://docs.iza.org/dp9000.pdf (application/pdf)

Related works:
Journal Article: THE MEANING OF FAILED REPLICATIONS: A REVIEW AND PROPOSAL (2017)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp9000

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().