Combining LASSO-type Methods with a Smooth Transition Random Forest

Alexandre L. D. Gandini () and Flavio A. Ziegelmann ()
Additional contact information
Alexandre L. D. Gandini: Universidade Federal do Rio Grande do Sul
Flavio A. Ziegelmann: Universidade Federal do Rio Grande do Sul

Annals of Data Science, 2025, vol. 12, issue 3, No 4, 899-928

Abstract: Abstract In this work, we propose a novel hybrid method for the estimation of regression models, which is based on a combination of LASSO-type methods and smooth transition (STR) random forests. Tree-based regression models are known for their flexibility and skills to learn even very nonlinear patterns. The STR-Tree model introduces smoothness into traditional splitting nodes, leading to a non-binary labeling, which can be interpreted as a group membership degree for each observation. Our approach involves two steps. First, we fit a penalized linear regression using LASSO-type methods. Then, we estimate an STR random forest on the residuals from the first step, using the original covariates. This dual-step process allows us to capture any significant linear relationships in the data generating process through a parametric approach, and then addresses nonlinearities with a flexible model. We conducted numerical studies with both simulated and real data to demonstrate our method’s effectiveness. Our findings indicate that our proposal offers superior predictive power, particularly in datasets with both linear and nonlinear characteristics, when compared to traditional benchmarks.

Keywords: Regression; LASSO; adaLASSO; STR-tree; Random forest; Smoothness (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40745-024-00541-4 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:aodasc:v:12:y:2025:i:3:d:10.1007_s40745-024-00541-4

Ordering information: This journal article can be ordered from
https://www.springer ... gement/journal/40745

DOI: 10.1007/s40745-024-00541-4

Access Statistics for this article

Annals of Data Science is currently edited by Yong Shi

More articles in Annals of Data Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().