A Double Penalty Model for Ensemble Learning

Wenjia Wang and Yi-Hui Zhou ()
Additional contact information
Wenjia Wang: Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 510000, China
Yi-Hui Zhou: Departments of Biological Sciences and Statistics, North Carolina State University, Raleigh, NC 27695, USA

Mathematics, 2022, vol. 10, issue 23, 1-23

Abstract: Modern statistical learning techniques often include learning ensembles, for which the combination of multiple separate prediction procedures (ensemble components) can improve prediction accuracy. Although ensemble approaches are widely used, work remains to improve our understanding of the theoretical underpinnings of aspects such as identifiability and relative convergence rates of the ensemble components. By considering ensemble learning for two learning ensemble components as a double penalty model, we provide a framework to better understand the relative convergence and identifiability of the two components. In addition, with appropriate conditions the framework provides convergence guarantees for a form of residual stacking when iterating between the two components as a cyclic coordinate ascent procedure. We conduct numerical experiments on three synthetic simulations and two real world datasets to illustrate the performance of our approach, and justify our theory.

Keywords: double penalty model; interpretability; partially linear model; separability (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/23/4532/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/23/4532/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:23:p:4532-:d:989340

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().