A Survey of Solution Path Algorithms for Regression and Classification Models

Guangrui Tang and Neng Fan ()
Additional contact information
Guangrui Tang: University of Arizona
Neng Fan: University of Arizona

Annals of Data Science, 2022, vol. 9, issue 4, No 5, 749-789

Abstract: Abstract The loss function expresses the differences between the predicted values from regression or classification models and the actual instances in machine learning. Regularization also plays an important role in machine learning, and it can mitigate overfitting problems, perform variable selection, and produce sparse models. The hyperparameter in these models controls the trade-off between the loss function and the regularization term, as well as the bias-variance trade-off. The choice of hyperparameter will influence the performance of the models. Thus, the hyperparameter needs to be tuned for effective learning from data. In some machine learning models, the optimal values for estimated coefficients are piecewise linear with respect to the hyperparameter. Efficient algorithms can be developed to compute all solutions, and these kinds of methods are called solution path algorithms. They can significantly reduce the efforts for cross-validation and highly speed up hyperparameter tuning. In this paper, we review the solution path algorithms widely used in regression and classification machine learning problems.

Keywords: Machine learning; Regularization; Solution path; Classification; Regression (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40745-022-00386-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:aodasc:v:9:y:2022:i:4:d:10.1007_s40745-022-00386-9

Ordering information: This journal article can be ordered from
https://www.springer ... gement/journal/40745

DOI: 10.1007/s40745-022-00386-9

Access Statistics for this article

Annals of Data Science is currently edited by Yong Shi

More articles in Annals of Data Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().