A new paradigm for high‐dimensional data: Distance‐based semiparametric feature aggregation framework via between‐subject attributes

Jinyuan Liu, Xinlian Zhang, Tuo Lin, Ruohui Chen, Yuan Zhong, Tian Chen, Tsungchin Wu, Chenyu Liu, Anna Huang, Tanya T. Nguyen, Ellen E. Lee, Dilip V. Jeste and Xin M. Tu

Scandinavian Journal of Statistics, 2024, vol. 51, issue 2, 672-696

Abstract: This article proposes a distance‐based framework incentivized by the paradigm shift toward feature aggregation for high‐dimensional data, which does not rely on the sparse‐feature assumption or the permutation‐based inference. Focusing on distance‐based outcomes that preserve information without truncating any features, a class of semiparametric regression has been developed, which encapsulates multiple sources of high‐dimensional variables using pairwise outcomes of between‐subject attributes. Further, we propose a strategy to address the interlocking correlations among pairs via the U‐statistics‐based estimating equations (UGEE), which correspond to their unique efficient influence function (EIF). Hence, the resulting semiparametric estimators are robust to distributional misspecification while enjoying root‐n consistency and asymptotic optimality to facilitate inference. In essence, the proposed approach not only circumvents information loss due to feature selection but also improves the model's interpretability and computational feasibility. Simulation studies and applications to the human microbiome and wearables data are provided, where the feature dimensions are tens of thousands.

Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/sjos.12695

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:scjsta:v:51:y:2024:i:2:p:672-696

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0303-6898

Access Statistics for this article

Scandinavian Journal of Statistics is currently edited by ÿrnulf Borgan and Bo Lindqvist

More articles in Scandinavian Journal of Statistics from Danish Society for Theoretical Statistics, Finnish Statistical Society, Norwegian Statistical Association, Swedish Statistical Association
Bibliographic data for series maintained by Wiley Content Delivery ().