Evaluations of FWER-controlling methods in multiple hypothesis testing

Yi-Ting Hwang, Jia-Jung Lai and Shyh-Tyan Ou

Journal of Applied Statistics, 2010, vol. 37, issue 10, 1681-1694

Abstract: Simultaneously testing a family of n null hypotheses can arise in many applications. A common problem in multiple hypothesis testing is to control Type-I error. The probability of at least one false rejection referred to as the familywise error rate (FWER) is one of the earliest error rate measures. Many FWER-controlling procedures have been proposed. The ability to control the FWER and achieve higher power is often used to evaluate the performance of a controlling procedure. However, when testing multiple hypotheses, FWER and power are not sufficient for evaluating controlling procedure's performance. Furthermore, the performance of a controlling procedure is also governed by experimental parameters such as the number of hypotheses, sample size, the number of true null hypotheses and data structure. This paper evaluates, under various experimental settings, the performance of some FWER-controlling procedures in terms of five indices, the FWER, the false discovery rate, the false non-discovery rate, the sensitivity and the specificity. The results can provide guidance on how to select an appropriate FWER-controlling procedure to meet a study's objective.

Keywords: Bonferroni's method; false discovery rate; false non-discovery rate; familywise error rate; multiple hypothesis testing; sensitivity; specificity (search for similar items in EconPapers)
Date: 2010
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.tandfonline.com/doi/abs/10.1080/02664760903136960 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:japsta:v:37:y:2010:i:10:p:1681-1694

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/CJAS20

DOI: 10.1080/02664760903136960

Access Statistics for this article

Journal of Applied Statistics is currently edited by Robert Aykroyd

More articles in Journal of Applied Statistics from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().