Estimating the extent of inflated significance in economics

Stephan Bruns, Teshome Kebede Deressa, T. Stanley, Chris Doucouliagos and John P.A. Ioannidis

No h29xn_v1, MetaArXiv from Center for Open Science

Abstract: Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as being non-significant. We estimate the extent of inflated significance to range between 56.2% and 71.3% of the significant p-values. Subsample analysis suggests that the extent of inflated significance is reduced in research fields that use experimental designs, analyze microeconomics research questions and have at least some adequately powered studies.

Date: 2022-03-11
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/622b744153a4e80277516456/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:h29xn_v1

DOI: 10.31219/osf.io/h29xn_v1

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().