The Romano-Wolf Multiple Hypothesis Correction in Stata

Damian Clarke, Joseph P. Romano () and Michael Wolf ()
Additional contact information
Joseph P. Romano: Stanford University
Michael Wolf: University of Zurich

No 12845, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: When considering multiple hypothesis tests simultaneously, standard statistical techniques will lead to over-rejection of null hypotheses unless the multiplicity of the testing framework is explicitly considered. In this paper we discuss the Romano-Wolf multiple hypothesis correction, and document its implementation in Stata. The Romano-Wolf correction (asymptotically) controls the familywise error rate (FWER), that is, the probability of rejecting at least one true null hypothesis in a family of hypotheses under test. This correction is considerably more powerful than earlier multiple testing procedures such as the Bonferroni and Holm corrections, given that it takes into account the dependence structure of the test statistics by resampling from the original data. We describe a Stata command rwolf that implements this correction, and provide a number of examples based on a wide range of models. We document and discuss the performance gains from using rwolf over other multiple correction procedures that control the FWER.

Keywords: step-down procedure; bootstrap; familywise error rate; multiple hypothesis testing; permutation methods; rwolf (search for similar items in EconPapers)
JEL-codes: C12 C15 C63 C87 (search for similar items in EconPapers)
Pages: 32 pages
Date: 2019-12
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (42)

Published - published in: Stata Journal, 2020, 13 (4), 812-843

Downloads: (external link)
https://docs.iza.org/dp12845.pdf (application/pdf)

Related works:
Journal Article: The Romano–Wolf multiple-hypothesis correction in Stata (2020)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp12845

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().