Do Replications Receive Fewer Citations? A Counterfactual Approach

Tom Coupé (), Thomas Logchies and W. Reed ()
Additional contact information
Tom Coupé: University of Canterbury, https://www.canterbury.ac.nz

Working Papers in Economics from University of Canterbury, Department of Economics and Finance

Abstract: There is a widespread belief that replication studies are less cited than original research. This study introduces three counterfactual approaches for measuring the citations that an author or a journal would have received had they produced a non-replication study. Two of the measures are designed to measure citation incentives from the perspective of authors. One measure focuses on the perspective of journals. We collect data on 428 replications in economics published between 1958 and 2021 and assess whether these are cited less frequently than their matched counterfactuals. We obtain a wide range of estimates. Our preferred estimates use the ratio of citations of a replication to the citations of its matched counterfactuals. Using this measure, we estimate citation penalties as large as 51% and citation benefits as great as 227%. Most replications receive fewer citations than their matched counterfactuals, but a sizable portion, and sometimes even a majority, receive more. Finally, there is some evidence that replications that do not fully support the original study have more favorable citation rates than those that confirm the original study. While our analysis does not produce an unambiguous answer to whether replications receive less citations than their counterfactuals, it does revise the widely held, one-sided view that replications receive fewer citations.

Keywords: Replications; Citations; Incentives; Academic Publishing (search for similar items in EconPapers)
JEL-codes: A10 A14 B41 C80 (search for similar items in EconPapers)
Pages: 42 pages
Date: 2024-12-01
New Economics Papers: this item is included in nep-hpe and nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://repec.canterbury.ac.nz/cbt/econwp/2418.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cbt:econwp:24/18

Access Statistics for this paper

More papers in Working Papers in Economics from University of Canterbury, Department of Economics and Finance Private Bag 4800, Christchurch, New Zealand. Contact information at EDIRC.
Bibliographic data for series maintained by Albert Yee ().