AUC-Maximizing Ensembles through Metalearning

LeDell Erin (), J. van der Laan Mark and Petersen Maya
Additional contact information
LeDell Erin: University of California Berkeley, Division of Biostatistics, Berkeley, CA 94720, USA
J. van der Laan Mark: University of California Berkeley, Division of Biostatistics, Berkeley, CA 94720, USA
Petersen Maya: University of California Berkeley, Division of Biostatistics, Berkeley, CA 94720, USA

The International Journal of Biostatistics, 2016, vol. 12, issue 1, 203-218

Abstract: Area Under the ROC Curve (AUC) is often used to measure the performance of an estimator in binary classification problems. An AUC-maximizing classifier can have significant advantages in cases where ranking correctness is valued or if the outcome is rare. In a Super Learner ensemble, maximization of the AUC can be achieved by the use of an AUC-maximining metalearning algorithm. We discuss an implementation of an AUC-maximization technique that is formulated as a nonlinear optimization problem. We also evaluate the effectiveness of a large number of different nonlinear optimization algorithms to maximize the cross-validated AUC of the ensemble fit. The results provide evidence that AUC-maximizing metalearners can, and often do, out-perform non-AUC-maximizing metalearning methods, with respect to ensemble AUC. The results also demonstrate that as the level of imbalance in the training data increases, the Super Learner ensemble outperforms the top base algorithm by a larger degree.

Keywords: AUC; binary classifiation; class imbalance; ensemble learning; machine learning (search for similar items in EconPapers)
Date: 2016
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1515/ijb-2015-0035 (text/html)
For access to full text, subscription to the journal or payment for the individual article is required.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:ijbist:v:12:y:2016:i:1:p:203-218:n:16

Ordering information: This journal article can be ordered from
https://www.degruyter.com/journal/key/ijb/html

DOI: 10.1515/ijb-2015-0035

Access Statistics for this article

The International Journal of Biostatistics is currently edited by Antoine Chambaz, Alan E. Hubbard and Mark J. van der Laan

More articles in The International Journal of Biostatistics from De Gruyter
Bibliographic data for series maintained by Peter Golla ().