Forward Variable Selection in Ultra-High Dimensional Linear Regression Using Gram-Schmidt Orthogonalization

Jialuo Chen, Zhaoxing Gao and Ruey S. Tsay

Papers from arXiv.org

Abstract: We investigate forward variable selection for ultra-high dimensional linear regression using a Gram-Schmidt orthogonalization procedure. Unlike the commonly used Forward Regression (FR) method, which computes regression residuals using an increasing number of selected features, or the Orthogonal Greedy Algorithm (OGA), which selects variables based on their marginal correlations with the residuals, our proposed Gram-Schmidt Forward Regression (GSFR) simplifies the selection process by evaluating marginal correlations between the residuals and the orthogonalized new variables. Moreover, we introduce a new model size selection criterion that determines the number of selected variables by detecting the most significant change in their unique contributions, effectively filtering out redundant predictors along the selection path. While GSFR is theoretically equivalent to FR except for the stopping rule, our refinement and the newly proposed stopping rule significantly improve computational efficiency. In ultra-high dimensional settings, where the dimensionality far exceeds the sample size and predictors exhibit strong correlations, we establish that GSFR achieves a convergence rate comparable to OGA and ensures variable selection consistency under mild conditions. We demonstrate the proposed method {using} simulations and real data examples. Extensive numerical studies show that GSFR outperforms commonly used methods in ultra-high dimensional variable selection.

Date: 2025-07
References: Add references at CitEc
Citations:

Downloads: (external link)
http://arxiv.org/pdf/2507.04668 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2507.04668

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().