Deciding what to replicate: A formal definition of “replication value” and a decision model for replication study selection

Peder Mortvedt Isager, Robbie Cornelis Maria van Aert, Štěpán Bahník, Mark John Brandt, Kurt Andrew DeSoto, Roger Giner-Sorolla, Joachim Krueger, Marco Perugini, Ivan Ropovik and Anna Elisabeth van 't Veer
Additional contact information
Peder Mortvedt Isager: Eindhoven University of Technology
Štěpán Bahník: University of Economics, Prague
Mark John Brandt: Tilburg University
Kurt Andrew DeSoto: Association for Psychological Science
Ivan Ropovik: University of Presov
Anna Elisabeth van 't Veer: Leiden University

No 2gurz, MetaArXiv from Center for Open Science

Abstract: Robust scientific knowledge is contingent upon replication of original findings. However, researchers who conduct replication studies face a difficult problem; there are many more studies in need of replication than there are funds available for replicating. To select studies for replication efficiently, we need to understand which studies are the most in need of replication. In other words, we need to understand which replication efforts have the highest expected utility. In this article we propose a general rule for study selection in replication research based on the replication value of the claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by replicating the claim, and is a function of (1) the value of being certain about the claim, and (2) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value.

Date: 2020-09-02
New Economics Papers: this item is included in nep-upt
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/5f4f4314a392b9002f1d9576/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:2gurz

DOI: 10.31219/osf.io/2gurz

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().