Sparse-Group Boosting: Unbiased Group and Variable Selection

Fabian Obster and Christian Heumann

The American Statistician, 2025, vol. 79, issue 2, 184-197

Abstract: For grouped covariates, we propose a framework for boosting that allows for sparsity within and between groups. By using component-wise and group-wise gradient ridge boosting simultaneously with adjusted degrees of freedom or penalty parameters, a model with similar properties as the sparse-group lasso can be fitted through boosting. We show that within-group and between-group sparsity can be controlled by a mixing parameter, and discuss similarities and differences to the mixing parameter in the sparse-group lasso. Furthermore, we show under which conditions variable selection on a group or individual variable basis happens and provide selection bounds for the regularization parameters depending solely on the singular values of the design matrix in a boosting iteration of linear Ridge penalized boosting. In special cases, we characterize the selection chance of an individual variable versus a group of variables through a generalized beta prime distribution. With simulations as well as two real datasets from ecological and organizational research data, we show the effectiveness and predictive competitiveness of this novel estimator. The results suggest that in the presence of grouped variables, sparse-group boosting is associated with less biased variable selection and higher predictability compared to component-wise or group-component-wise boosting.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/00031305.2024.2408007 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:amstat:v:79:y:2025:i:2:p:184-197

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/UTAS20

DOI: 10.1080/00031305.2024.2408007

Access Statistics for this article

The American Statistician is currently edited by Eric Sampson

More articles in The American Statistician from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().