Large-Scale Simultaneous Testing Using Kernel Density Estimation

Santu Ghosh () and Alan M. Polansky
Additional contact information
Santu Ghosh: Augusta University
Alan M. Polansky: Northern Illinois University

Sankhya A: The Indian Journal of Statistics, 2022, vol. 84, issue 2, No 16, 808-843

Abstract: Abstract A century ago, when Student’s t-statistic was introduced, no one ever imagined its increasing applicability in the modern era. It finds applications in highly multiple hypothesis testing, feature selection and ranking, high dimensional signal detection, etc. Student’s t-statistic is constructed based on the empirical distribution function (EDF). An alternative choice to the EDF is the kernel density estimate (KDE), which is a smoothed version of the EDF. The novelty of the work consists of an alternative to Student’s t-test that uses the KDE technique and exploration of the usefulness of KDE based t-test in the context of its application to large-scale simultaneous hypothesis testing. An optimal bandwidth parameter for the KDE approach is derived by minimizing the asymptotic error between the true p-value and its asymptotic estimate based on normal approximation. If the KDE-based approach is used for large-scale simultaneous testing, then it is interesting to consider, when does the method fail to manage the error rate? We show that the suggested KDE-based method can control false discovery rate (FDR) if total number tests diverge at a smaller order of magnitude than N3/2, where N is the total sample size. We compare our method to several possible alternatives with respect to FDR. We show in simulations that our method produces a lower proportion of false discoveries than its competitors. That is, our method better controls the false discovery rate than its competitors. Through these empirical studies, it is shown that the proposed method can be successfully applied in practice. The usefulness of the proposed methods is further illustrated through a gene expression data example.

Keywords: Two-sample t-test; Kernel density estimator; Edgeworth expansion; False discovery rate; Primary 62F03; Secondary 62G10 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s13171-020-00220-5 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sankha:v:84:y:2022:i:2:d:10.1007_s13171-020-00220-5

Ordering information: This journal article can be ordered from
http://www.springer.com/statistics/journal/13171

DOI: 10.1007/s13171-020-00220-5

Access Statistics for this article

Sankhya A: The Indian Journal of Statistics is currently edited by Dipak Dey

More articles in Sankhya A: The Indian Journal of Statistics from Springer, Indian Statistical Institute
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().