Formalized Data Snooping Based on Generalized Error Rates

Joseph P, Romano, Azeem Shaikh and Michael Wolf

No 259, IEW - Working Papers from Institute for Empirical Research in Economics - University of Zurich

Abstract: It is common in econometric applications that several hypothesis tests are carried out at the same time. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. The classical approach is to control the familywise error rate (FWE), that is, the probability of one or more false rejections. But when the number of hypotheses under consideration is large, control of the FWE can become too demanding. As a result, the number of false hypotheses rejected may be small or even zero. This suggests replacing control of the FWE by a more liberal measure. To this end, we review a number of proposals from the statistical literature. We briefly discuss how these procedures apply to the general problem of model selection. A simulation study and two empirical applications illustrate the methods.

Keywords: Data snooping; false discovery proportion; false discovery rate; generalized familywise error rate; model selection; multiple testing; stepwise methods (search for similar items in EconPapers)
JEL-codes: C12 C14 C52 (search for similar items in EconPapers)
Date: 2005-12
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.zora.uzh.ch/id/eprint/52149/1/iewwp259.pdf (application/pdf)

Related works:
Journal Article: FORMALIZED DATA SNOOPING BASED ON GENERALIZED ERROR RATES (2008)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zur:iewwpx:259

Access Statistics for this paper

More papers in IEW - Working Papers from Institute for Empirical Research in Economics - University of Zurich
Bibliographic data for series maintained by Severin Oswald ().