Statistical Significance, p-Values, and the Reporting of Uncertainty

Guido Imbens

Journal of Economic Perspectives, 2021, vol. 35, issue 3, 157-74

Abstract: The use of statistical significance and p-values has become a matter of substantial controversy in various fields using statistical methods. This has gone as far as some journals banning the use of indicators for statistical significance, or even any reports of p-values, and, in one case, any mention of confidence intervals. I discuss three of the issues that have led to these often-heated debates. First, I argue that in many cases, p-values and indicators of statistical significance do not answer the questions of primary interest. Such questions typically involve making (recommendations on) decisions under uncertainty. In that case, point estimates and measures of uncertainty in the form of confidence intervals or even better, Bayesian intervals, are often more informative summary statistics. In fact, in that case, the presence or absence of statistical significance is essentially irrelevant, and including them in the discussion may confuse the matter at hand. Second, I argue that there are also cases where testing null hypotheses is a natural goal and where p-values are reasonable and appropriate summary statistics. I conclude that banning them in general is counterproductive. Third, I discuss that the overemphasis in empirical work on statistical significance has led to abuse of p-values in the form of p-hacking and publication bias. The use of pre-analysis plans and replication studies, in combination with lowering the emphasis on statistical significance may help address these problems.

JEL-codes: C01 C12 C13 D81 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (35)

Downloads: (external link)
https://www.aeaweb.org/doi/10.1257/jep.35.3.157 (application/pdf)
https://www.aeaweb.org/doi/10.1257/jep.35.3.157.ds (application/zip)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aea:jecper:v:35:y:2021:i:3:p:157-74

Ordering information: This journal article can be ordered from
https://www.aeaweb.org/journals/subscriptions

DOI: 10.1257/jep.35.3.157

Access Statistics for this article

Journal of Economic Perspectives is currently edited by Enrico Moretti

More articles in Journal of Economic Perspectives from American Economic Association Contact information at EDIRC.
Bibliographic data for series maintained by Michael P. Albert ().