Expanding the scope of reproducibility research through data analysis replications
Jake M. Hofman,
Daniel G. Goldstein,
Ling Liang Dong,
Etta Rapp and
Organizational Behavior and Human Decision Processes, 2021, vol. 164, issue C, 192-202
In recent years, researchers in several scientific disciplines have become concerned with published studies replicating less often than expected. A positive side effect of this concern is an appreciation that replicating other researchers’ work is an essential part of the scientific process. To date, many such efforts have come from the experimental sciences, where replication entails running new experiments, generating new data, and analyzing it. In this article, we emphasize not experimental replications but data analysis replications. We do so for three reasons. First, experimental replication excludes entire classes of publications that do not run experiments or even collect original data (e.g., archival data analysis). Second, experimental replication may in some cases be a needlessly high bar: there is great value in replicating just the data analyses of published experimental work. As data analysis replications require a lower investment of resources than experimental replications, their adoption should expand the number and variety of scientific reproducibility studies undertaken. Third, we propose that teaching undergraduate students to perform data analysis replications will greatly increase the number of replications done while providing them with research experience that should inform their decisions to pursue research or to attend graduate school. Towards this end, we provide details of a pilot program we created to teach undergraduates the skills necessary to conduct data analysis replications, and include a case study of the first set of students who completed this program and attempted to replicate the data analyses in a widely-cited social science paper on policing. In addition, we present a summary of ten additional data analysis replications carried out entirely by students in a university course.
Keywords: Reproducibility; Replication; Robustness; Education; Data analysis (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed
Downloads: (external link)
Full text for ScienceDirect subscribers only
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:eee:jobhdp:v:164:y:2021:i:c:p:192-202
Access Statistics for this article
Organizational Behavior and Human Decision Processes is currently edited by John M. Schaubroeck
More articles in Organizational Behavior and Human Decision Processes from Elsevier
Bibliographic data for series maintained by Catherine Liu ().