ROC Curves, Loss Functions, and Distorted Probabilities in Binary Classification

Phuong Bich Le and Zung Tien Nguyen
Additional contact information
Phuong Bich Le: Department of Mathematics, Hanoi University of Mining and Geology, No. 18 Vien Street, Duc Thang Ward, Bac Tu Liem District, Hanoi City 100000, Vietnam
Zung Tien Nguyen: Institut de Mathématiques de Toulouse, Université Paul Sabatier, 31062 Toulouse, France

Mathematics, 2022, vol. 10, issue 9, 1-13

Abstract: The main purpose of this work is to study how loss functions in machine learning influence the “binary machines”, i.e., probabilistic AI models for predicting binary classification problems. In particular, we show the following results: (i) Different measures of accuracy such as area under the curve (AUC) of the ROC curve, the maximal balanced accuracy, and the maximally weighted accuracy are topologically equivalent, with natural inequalities relating them; (ii) the so-called real probability machines with respect to given information spaces are the optimal machines, i.e., they have the highest precision among all possible machines, and moreover, their ROC curves are automatically convex; (iii) the cross-entropy and the square loss are the most natural loss functions in the sense that the real probability machine is their minimizer; (iv) an arbitrary strictly convex loss function will also have as its minimizer an optimal machine, which is related to the real probability machine by just a reparametrization of sigmoid values; however, if the loss function is not convex, then its minimizer is not an optimal machine, and strange phenomena may happen.

Keywords: optimization; binary classification; machine learning; ROC curve; accuracy metrics; loss function; quadratic loss; quartic loss; cross-entropy; convexity; information space; optimal machine; real probability machine; distorted probabilities (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/9/1410/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/9/1410/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:9:p:1410-:d:799938

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().