Beyond Bonferroni: Multiple Testing in Empirical Research

Sebastian Calonico and Sebastian Galiani

No 34050, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: Empirical work in economics routinely tests many hypotheses at once, but applied researchers often lack clear guidance on how to handle the resulting multiplicity. This paper offers a practical guide. We argue that the choice of error criterion — the chance of any false rejection, or the share of false rejections among discoveries—should follow from the structure of the decision the tests inform. Beyond standard corrections like Bonferroni and Holm, we make the case for resampling-based procedures, particularly Romano–Wolf, which exploit dependence among test statistics — the central feature of multiple testing in economic applications. We also recommend hierarchical methods that use causal or logical structure to deliver more powerful results. By linking these tools to real applications, we provide a clear roadmap, anchored in pre-specification, for credible empirical work.

JEL-codes: C1 C11 C13 C15 C18 (search for similar items in EconPapers)
Date: 2025-07
New Economics Papers: this item is included in nep-ecm
Note: DEV
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.nber.org/papers/w34050.pdf (application/pdf)
Access to the full text is generally limited to series subscribers, however if the top level domain of the client browser is in a developing country or transition economy free access is provided. More information about subscriptions and free access is available at http://www.nber.org/wwphelp.html. Free access is also available to older working papers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:34050

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w34050
The price is Paper copy available by mail.

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().