Beyond Bonferroni: Hierarchical Multiple Testing in Empirical Research

Sebastian Calonico and Sebastian Galiani

No 34050, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: This paper offers a general, practical review of methods for dealing with multiple hypothesis testing, aimed at applied researchers who must draw inference on multiple outcomes, subgroups, or model specifications but often lack clear guidance on how to proceed. We distill key lessons from the methodological literature, explain why several common practices are flawed, and clarify the guarantees provided by alternative approaches. Beyond standard corrections such as Bonferroni, Holm, and false discovery rate controls, we emphasize the value of hierarchical procedures that exploit causal or logical structure to deliver more powerful and interpretable results. By linking these tools to real applications, we translate methodological advances into accessible guidance and provide researchers with a clear roadmap for producing credible, transparent, and policy-relevant empirical work.

JEL-codes: C1 C11 C13 C15 C18 (search for similar items in EconPapers)
Date: 2025-07
New Economics Papers: this item is included in nep-ecm
Note: DEV
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.nber.org/papers/w34050.pdf (application/pdf)
Access to the full text is generally limited to series subscribers, however if the top level domain of the client browser is in a developing country or transition economy free access is provided. More information about subscriptions and free access is available at http://www.nber.org/wwphelp.html. Free access is also available to older working papers.

Related works:
Working Paper: Beyond Bonferroni: Hierarchical Multiple Testing in Empirical Research (2025)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:34050

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w34050
The price is Paper copy available by mail.

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().