Do Negative Replications Affect Citations?

Tom Coupé and W. Reed ()

Working Papers in Economics from University of Canterbury, Department of Economics and Finance

Abstract: This study examines the effect of negative replications on the citation rates of replicated studies. We study a set of 204 replicated studies and compare their citation performance with an initial sample of 112,000 potential controls taken from Scopus. Approximately half of the replication studies refuted key findings from the original studies, with the remaining half either providing a confirmation or a mixed conclusion. Using matching criteria that accommodate differences in the lengths of time between publication of the original study and its replication, as well as differences in the number of citations, we match each replicated study with multiple controls based on having comparable citation histories. Our main samples consist of 74, 103, and 142 replicated studies and 7,044, 7,552, and 11,202 matched control studies, respectively. We have two main findings. First, studies that are replicated receive somewhat more citations than their matched control studies. Second, there is no evidence that studies that receive negative replications suffer a penalty in the form of fewer citations.

Keywords: Replications; Citations; Matching; Hierarchical Linear Modelling; Quantile Regression (search for similar items in EconPapers)
JEL-codes: A11 B41 Z00 (search for similar items in EconPapers)
Pages: 44 pages
Date: 2021-11-01
New Economics Papers: this item is included in nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://repec.canterbury.ac.nz/cbt/econwp/2114.pdf (application/pdf)

Related works:
Working Paper: Do Negative Replications Affect Citations? (2022)
Working Paper: Do Negative Replications Affect Citations? (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cbt:econwp:21/14

Access Statistics for this paper

More papers in Working Papers in Economics from University of Canterbury, Department of Economics and Finance Private Bag 4800, Christchurch, New Zealand. Contact information at EDIRC.
Bibliographic data for series maintained by Albert Yee ().