Multiple Testing. Part III. Procedures for Control of the Generalized Family-Wise Error Rate and Proportion of False Positives

Mark van der Laan, Sandrine Dudoit and Katherine Pollard
Additional contact information
Mark van der Laan: Division of Biostatistics, School of Public Health, University of California, Berkeley
Sandrine Dudoit: Division of Biostatistics, School of Public Health, University of California, Berkeley
Katherine Pollard: Center for Biomolecular Science & Engineering, University of California, Santa Cruz

No 1140, U.C. Berkeley Division of Biostatistics Working Paper Series from Berkeley Electronic Press

Abstract: The accompanying articles by Dudoit et al. (2003b) and van der Laan et al. (2003) provide single-step and step-down resampling-based multiple testing procedures that asymptotically control the family-wise error rate (FWER) for general null hypotheses and test statistics. The proposed procedures fundamentally differ from existing approaches in the choice of null distribution for deriving cut-offs for the test statistics and are shown to provide asymptotic control of the FWER under general data generating distributions, without the need for conditions such as subset pivotality. In this article, we show that any multiple testing procedure (asymptotically) controlling the FWER at level alpha can be augmented into: (i) a multiple testing procedure (asymptotically) controlling the generalized family-wise error rate (i.e., the probability, gFWER(k), of having more than k false positives) at level alpha and (ii) a multiple testing procedure (asymptotically) controlling the probability, PFP(q), that the proportion of false positives among the rejected hypotheses exceeds a user-supplied value q in (0,1) at level alpha. Existing procedures for control of the proportion of false positives typically rely on the assumption that the test statistics are independent, while our proposed augmentation procedures control the PFP and gFWER for general data generating distributions, with arbitrary dependence structures among variables. Applying our augmentation methods to step-down multiple testing procedures that asymptotically control the FWER at exact level alpha (van der Laan et al., 2003), yields multiple testing procedures that also asymptotically control the gFWER and PFP at exact level alpha. Finally, the adjusted p-values for the gFWER and PFP-controlling augmentation procedures are shown to be simple functions of the adjusted p-values for the original FWER-controlling procedure.

Keywords: Adjusted p-value; asymptotic control; false discovery rate; generalized family-wise error rate; multiple testing; proportion of false positives; single-step; step-down; Type I error rate (search for similar items in EconPapers)
Date: 2004-07-11
Note: oai:bepress.com:ucbbiostat-1140
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (28)

Downloads: (external link)
http://www.bepress.com/cgi/viewcontent.cgi?article=1140&context=ucbbiostat (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bep:ucbbio:1140

Access Statistics for this paper

More papers in U.C. Berkeley Division of Biostatistics Working Paper Series from Berkeley Electronic Press
Bibliographic data for series maintained by Christopher F. Baum ().