Predicting Replication Rates with Z-Curve: A Brief Exploratory Validation Study Using the Replication Database

Lukas Röseler
Additional contact information
Lukas Röseler: University of Bamberg

No ewb2t, MetaArXiv from Center for Open Science

Abstract: Concerns of replicability are widespread in the social sciences. As it is not feasible to replicate every published study, researchers have been developing methods to estimate replicability. I used the Replication Database (Röseler et al., 2023) to compare actual replication rates with replicability estimates provided by z-curve. After drawing stratified samples with actual replication rates that were uniformly distributed, z-curve’s replicability estimates had lower variance but correlated strongly with actual replicability rates, r = .933. Using a linear model, predicted replication rates deviated from actual replication rates by <±16% when samples from 322 studies (2.5 and 97.5% quantiles) were drawn. I propose that z-curve is a valid and economic method to compare replicability estimates for large sets of studies. Future studies of moderators in the context of z-curve or replicability should be tested using replication databases. The study’s code and data are available online (https://osf.io/k4d6w/).

Date: 2023-10-12
New Economics Papers: this item is included in nep-sog
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6529302513d27b039394c9e8/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:ewb2t

DOI: 10.31219/osf.io/ewb2t

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().