Assessing replication success via skeptical mixture priors

Guido Consonni () and Leonardo Egidi ()
Additional contact information
Guido Consonni: Università Cattolica del Sacro Cuore, Department of Statistical Sciences
Leonardo Egidi: University of Trieste, Department of Economics, Business, Mathematics, and Statistics “Bruno de Finetti”

TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, 2025, vol. 34, issue 4, No 6, 1012-1035

Abstract: Abstract There is growing interest in the analysis of replication studies aimed at reassessing original findings across a wide range of scientific disciplines. In the context of hypothesis testing for effect sizes, two Bayesian approaches stand out for their principled use of the Bayes factor (BF): the replication BF and the skeptical BF. The latter, built around the skeptical prior, represents the perspective of an investigator who remains unconvinced by the original results and seeks to critically reassess them. In this paper, we adopt the skeptical viewpoint and introduce a novel mixture prior that incorporates skepticism while offering control over prior-data conflict. We study the consistency properties of the resulting skeptical mixture Bayes factor and examine its relationship to the standard skeptical BF. Through a focused simulation study, we conduct a sensitivity analysis of the skeptical mixture BF with respect to prior-data conflict, covering a range of plausible experimental scenarios. Our results show broad agreement with the standard skeptical BF under typical conditions. However, in situations where the standard skeptical BF suffers from severe prior-data conflict, our approach can yield a meaningful adjustment in the reported strength of replication success. Finally,we illustrate the practical application of our method using case studies from the Social Sciences Replication Project.

Keywords: Bayes factor; Bayesian hypothesis testing; Consistency; Prior-data conflict; Replication studies; Reverse-Bayes; 62A01; 62F15 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11749-025-00985-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:testjl:v:34:y:2025:i:4:d:10.1007_s11749-025-00985-7

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/11749/PS2

DOI: 10.1007/s11749-025-00985-7

Access Statistics for this article

TEST: An Official Journal of the Spanish Society of Statistics and Operations Research is currently edited by Alfonso Gordaliza and Ana F. Militino

More articles in TEST: An Official Journal of the Spanish Society of Statistics and Operations Research from Springer, Sociedad de Estadística e Investigación Operativa
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().