Getting seen: Results from an online experiment to draw more attention to replications

Tom Coupé, W. Reed () and Christian Zimmermann

Research Policy, 2023, vol. 52, issue 8

Abstract: Are users of bibliographic databases interested in learning about replications? Can they be induced to learn? To answer these questions, we performed an experiment using an online research bibliography, Research Papers in Economics (RePEc). RePEc is the main research bibliography for preprints and published papers in economics. Using stratified randomization, we allocated 324 replications and their corresponding original papers to clusters. We then drew from these clusters to construct the treatment and control groups. We added brightly coloured tabs to the relevant webpages to alert visitors to the existence of a replication paper. We monitored traffic over three phases lasting several months: a) no treatment, b) treatment in one group, and c) treatment in both groups. Our estimates indicate that this intervention generated an average click-through rate (CTR) of 1.6 %, resulting in a 13 % increase in the number of visits to the webpages of the replication papers, although only the former estimate was statistically significant.

Keywords: Replications; RePEc; Experiment; Online research bibliography; Webpages; Click-throughs (search for similar items in EconPapers)
JEL-codes: A11 B41 Z00 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0048733323001257
Full text for ScienceDirect subscribers only

Related works:
Working Paper: Paving the Road for Replications: Experimental Results from an Online Research Repository (2022)
Working Paper: Paving the Road for Replications: Experimental Results from an Online Research Bibliography (2022)
Working Paper: Paving the Road for Replications: Experimental Results from an Online Research Repository (2021)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:respol:v:52:y:2023:i:8:s0048733323001257

DOI: 10.1016/j.respol.2023.104841

Access Statistics for this article

Research Policy is currently edited by M. Bell, B. Martin, W.E. Steinmueller, A. Arora, M. Callon, M. Kenney, S. Kuhlmann, Keun Lee and F. Murray

More articles in Research Policy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().