 A new standard for the analysis and design of replication studiesLeonhard Held Journal of the Royal Statistical Society Series A, 2020, vol. 183, issue 2, 431-448 Abstract: A new standard is proposed for the evidential assessment of replication studies. The approach combines a specific reverse Bayes technique with prior‐predictive tail probabilities to define replication success. The method gives rise to a quantitative measure for replication success, called the sceptical p‐value. The sceptical p‐value integrates traditional significance of both the original and the replication study with a comparison of the respective effect sizes. It incorporates the uncertainty of both the original and the replication effect estimates and reduces to the ordinary p‐value of the replication study if the uncertainty of the original effect estimate is ignored. The framework proposed can also be used to determine the power or the required replication sample size to achieve replication success. Numerical calculations highlight the difficulty of achieving replication success if the evidence from the original study is only suggestive. An application to data from the Open Science Collaboration project on the replicability of psychological science illustrates the methodology proposed. Date: 2020 Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:bla:jorssa:v:183:y:2020:i:2:p:431-448 Ordering information: This journal article can be ordered from Access Statistics for this article Journal of the Royal Statistical Society Series A is currently edited by A. Chevalier and L. Sharples More articles in Journal of the Royal Statistical Society Series A from Royal Statistical Society Contact information at EDIRC. |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.