Optimization by Gradient Boosting

Gérard Biau () and Benoît Cadre ()
Additional contact information
Gérard Biau: Sorbonne Université, CNRS, LPSM
Benoît Cadre: Univ Rennes, CNRS, IRMAR - UMR 6625

A chapter in Advances in Contemporary Statistics and Econometrics, 2021, pp 23-44 from Springer

Abstract: Abstract Gradient boosting is a state-of-the-art prediction technique that sequentially produces a model in the form of linear combinations of elementary predictors—typically decision trees—by solving an infinite-dimensional convex optimization problem. We provide in the present paper a thorough analysis of two widespread versions of gradient boosting, and introduce a general framework for studying these algorithms from the point of view of functional optimization. We prove their convergence as the number of iterations tends to infinity and highlight the importance of having a strongly convex risk functional to minimize. We also present a reasonable statistical context ensuring consistency properties of the boosting predictors as the sample size grows. In our approach, the optimization procedures are run forever (that is, without resorting to an early stopping strategy), and statistical regularization is basically achieved via an appropriate $$L^2$$ L 2 penalization of the loss and strong convexity arguments.

Date: 2021
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-030-73249-3_2

Ordering information: This item can be ordered from
http://www.springer.com/9783030732493

DOI: 10.1007/978-3-030-73249-3_2

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().