How Credible Is the Credibility Revolution?

Kevin Lang

Journal of Labor Economics, 2025, vol. 43, issue 2, 635 - 663

Abstract: Economists analyzing a well-conducted randomized controlled trial or natural experiment and finding a statistically significant effect conclude that the null of no effect is unlikely to be true. But how frequently is this conclusion warranted? The answer depends on the proportion of tested nulls that are true and the test's power. I model the distribution of t-statistics in leading economics journals. Using my preferred model, 65% of narrowly rejected null hypotheses and 41% of all rejected null hypotheses with |t|<10 are likely to be false rejections. For the null to have only a .05 probability of being true requires a t of 5.48.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1086/732772 (application/pdf)
http://dx.doi.org/10.1086/732772 (text/html)
Access to the online full text or PDF requires a subscription.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ucp:jlabec:doi:10.1086/732772

Access Statistics for this article

More articles in Journal of Labor Economics from University of Chicago Press
Bibliographic data for series maintained by Journals Division ().