Ranked sparsity: a cogent regularization framework for selecting and estimating feature interactions and polynomials

Ryan A. Peterson () and Joseph E. Cavanaugh ()
Additional contact information
Ryan A. Peterson: University of Colorado School of Public Health
Joseph E. Cavanaugh: University of Iowa College of Public Health

AStA Advances in Statistical Analysis, 2022, vol. 106, issue 3, No 8, 427-454

Abstract: Abstract We explore and illustrate the concept of ranked sparsity, a phenomenon that often occurs naturally in modeling applications when an expected disparity exists in the quality of information between different feature sets. Its presence can cause traditional and modern model selection methods to fail because such procedures commonly presume that each potential parameter is equally worthy of entering into the final model—we call this presumption “covariate equipoise.” However, this presumption does not always hold, especially in the presence of derived variables. For instance, when all possible interactions are considered as candidate predictors, the premise of covariate equipoise will often produce over-specified and opaque models. The sheer number of additional candidate variables grossly inflates the number of false discoveries in the interactions, resulting in unnecessarily complex and difficult-to-interpret models with many (truly spurious) interactions. We suggest a modeling strategy that requires a stronger level of evidence in order to allow certain variables (e.g., interactions) to be selected in the final model. This ranked sparsity paradigm can be implemented with the sparsity-ranked lasso (SRL). We compare the performance of SRL relative to competing methods in a series of simulation studies, showing that the SRL is a very attractive method because it is fast and accurate and produces more transparent models (with fewer false interactions). We illustrate its utility in an application to predict the survival of lung cancer patients using a set of gene expression measurements and clinical covariates, searching in particular for gene–environment interactions.

Keywords: Derived variables; Feature selection; Information; Lasso; Model selection (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10182-021-00431-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:alstar:v:106:y:2022:i:3:d:10.1007_s10182-021-00431-7

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10182/PS2

DOI: 10.1007/s10182-021-00431-7

Access Statistics for this article

AStA Advances in Statistical Analysis is currently edited by Göran Kauermann and Yarema Okhrin

More articles in AStA Advances in Statistical Analysis from Springer, German Statistical Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().