Simultaneous estimation of cluster number and feature sparsity in high‐dimensional cluster analysis

Yujia Li, Xiangrui Zeng, Chien‐Wei Lin and George C. Tseng

Biometrics, 2022, vol. 78, issue 2, 574-585

Abstract: Estimating the number of clusters (K) is a critical and often difficult task in cluster analysis. Many methods have been proposed to estimate K, including some top performers using resampling approach. When performing cluster analysis in high‐dimensional data, simultaneous clustering and feature selection is needed for improved interpretation and performance. To our knowledge, little has been studied for simultaneous estimation of K and feature sparsity parameter in a high‐dimensional exploratory cluster analysis. In this paper, we propose a resampling method to bridge this gap and evaluate its performance under the sparse K‐means clustering framework. The proposed target function balances between sensitivity and specificity of clustering evaluation of pairwise subjects from clustering of full and subsampled data. Through extensive simulations, the method performs among the best over classical methods in estimating K in low‐dimensional data. For high‐dimensional simulation data, it also shows superior performance to simultaneously estimate K and feature sparsity parameter. Finally, we evaluated the methods in four microarray, two RNA‐seq, one SNP, and two nonomics datasets. The proposed method achieves better clustering accuracy with fewer selected predictive genes in almost all real applications.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/biom.13449

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:biomet:v:78:y:2022:i:2:p:574-585

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0006-341X

Access Statistics for this article

More articles in Biometrics from The International Biometric Society
Bibliographic data for series maintained by Wiley Content Delivery ().