Block-diagonal precision matrix regularization for ultra-high dimensional data

Yihe Yang, Hongsheng Dai and Jianxin Pan

Computational Statistics & Data Analysis, 2023, vol. 179, issue C

Abstract: A method that estimates the precision matrix of multiple variables in the extreme scope of “ultrahigh dimension” and “small sample-size” is proposed. Initially, a covariance column-wise screening method is provided in order to identify a small sub-group, which are significantly correlated, from thousands and even millions of variables. Then, a regularization of block-diagonal covariance structure of the thousands or millions of variables is imposed, in which only the covariances of variables in that small sub-group are retained and all others vanish. It is further proven that under some mild conditions the vital sub-group identified by the covariance column-wise screening method is consistent. A major advantage of the proposed method is its efficiency - it produces a reliable precision matrix estimator for thousands of variables within a few of seconds while the existing methods take at least several hours and even so still yield inaccurate estimators. Empirical data studies and numerical simulations show that the proposed precision matrix estimation greatly outperforms existing methods in the sense of taking much less computing time and resulting in much more accurate estimation when dealing with ultrahigh dimensional data.

Keywords: Precision matrix estimation; Block-diagonal structure; Ultrahigh dimensionality (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167947322002109
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:csdana:v:179:y:2023:i:c:s0167947322002109

DOI: 10.1016/j.csda.2022.107630

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().