The Meaning of Failed Replications: A Review and Proposal - Working Paper 399

Michael Clemens

No 399, Working Papers from Center for Global Development

Abstract: Economists are increasingly using publicly shared data and code to check each other’s work, an exercise often called ‘replication’ testing. But this much-needed trend has not been accompanied by a consensus about what ‘replication’ means. If a follow-up study does not ‘replicate’ an original result, according to current usage of the term, this can mean anything from an unremarkable disagreement over methods to scientific incompetence or misconduct. This paper proposes an unambiguous definition of replication. Many social scientists already use the term in the way suggested here, but many more do not. The paper contrasts this definition with decades of unsuccessful attempts to standardize terminology, and argues that many prominent results described as replication tests should not be described as such. It argues that professional associations should formally adopt this definition, thereby improving incentives for researchers to conduct more and better replication tests.

Keywords: replication; studies (search for similar items in EconPapers)
JEL-codes: B40 C18 C80 (search for similar items in EconPapers)
Pages: 24 pages
Date: 2015-04
New Economics Papers: this item is included in nep-hme and nep-hpe
References: Add references at CitEc
Citations: View citations in EconPapers (25)

Downloads: (external link)
http://www.cgdev.org/publication/meaning-failed-re ... al-working-paper-399

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cgd:wpaper:399

Access Statistics for this paper

More papers in Working Papers from Center for Global Development Contact information at EDIRC.
Bibliographic data for series maintained by Publications Manager ().