Classifier calibration using splined empirical probabilities in clinical risk prediction

René Gaudoin (), Giovanni Montana (), Simon Jones, Paul Aylin and Alex Bottle

Health Care Management Science, 2015, vol. 18, issue 2, 156-165

Abstract: The aims of supervised machine learning (ML) applications fall into three broad categories: classification, ranking, and calibration/probability estimation. Many ML methods and evaluation techniques relate to the first two. Nevertheless, there are many applications where having an accurate probability estimate is of great importance. Deriving accurate probabilities from the output of a ML method is therefore an active area of research, resulting in several methods to turn a ranking into class probability estimates. In this manuscript we present a method, splined empirical probabilities, based on the receiver operating characteristic (ROC) to complement existing algorithms such as isotonic regression. Unlike most other methods it works with a cumulative quantity, the ROC curve, and as such can be tagged onto an ROC analysis with minor effort. On a diverse set of measures of the quality of probability estimates (Hosmer-Lemeshow, Kullback-Leibler divergence, differences in the cumulative distribution function) using simulated and real health care data, our approach compares favourably with the standard calibration method, the pool adjacent violators algorithm used to perform isotonic regression. Copyright Springer Science+Business Media New York 2015

Keywords: Calibration; Probability estimation; Logistic regression; Empirical probabilities; ROC; HES Data; 62H12; 62H30; 92C50; 92-08 (search for similar items in EconPapers)
Date: 2015
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1007/s10729-014-9267-1 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:kap:hcarem:v:18:y:2015:i:2:p:156-165

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10729

DOI: 10.1007/s10729-014-9267-1

Access Statistics for this article

Health Care Management Science is currently edited by Yasar Ozcan

More articles in Health Care Management Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().