Stepwise multiple testing as formalized data snooping

Joseph P. Romano and Michael Wolf

Economics Working Papers from Department of Economics and Business, Universitat Pompeu Fabra

Abstract: It is common in econometric applications that several hypothesis tests are carried out at the same time. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. In this paper, we suggest a stepwise multiple testing procedure which asymptotically controls the familywise error rate at a desired level. Compared to related single-step methods, our procedure is more powerful in the sense that it often will reject more false hypotheses. In addition, we advocate the use of studentization when it is feasible. Unlike some stepwise methods, our method implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect alternative hypotheses. We prove our method asymptotically controls the familywise error rate under minimal assumptions. We present our methodology in the context of comparing several strategies to a common benchmark and deciding which strategies actually beat the benchmark. However, our ideas can easily be extended and/or modi ed to other contexts, such as making inference for the individual regression coecients in a multiple regression framework. Some simulation studies show the improvements of our methods over previous proposals. We also provide an application to a set of real data.

Keywords: Bootstrap; data snooping; familywise error; multiple testing; step-down method (search for similar items in EconPapers)
JEL-codes: C12 C14 C52 (search for similar items in EconPapers)
Date: 2003-10
New Economics Papers: this item is included in nep-ecm and nep-fin
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://econ-papers.upf.edu/papers/712.pdf Whole Paper (application/pdf)

Related works:
Working Paper: Stepwise Multiple Testing as Formalized Data Snooping (2015)
Journal Article: Stepwise Multiple Testing as Formalized Data Snooping (2005)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:upf:upfgen:712

Access Statistics for this paper

More papers in Economics Working Papers from Department of Economics and Business, Universitat Pompeu Fabra
Bibliographic data for series maintained by ( this e-mail address is bad, please contact ).