Rethinking an ROC partial area index for evaluating the classification performance at a high specificity range

Juana-Maria Vivo, Manuel Franco () and Donatella Vicari ()
Additional contact information
Manuel Franco: University of Murcia
Donatella Vicari: Sapienza University of Rome

Advances in Data Analysis and Classification, 2018, vol. 12, issue 3, No 11, 683-704

Abstract: Abstract The area under a receiver operating characteristic (ROC) curve is valuable for evaluating the classification performance described by the entire ROC curve in many fields including decision making and medical diagnosis. However, this can be misleading when clinical tasks demand a restricted specificity range. The partial area under a portion of the ROC curve ( $${ pAUC}$$ p A U C ) has more practical relevance in such situations, but it is usually transformed to overcome some drawbacks and improve its interpretation. The standardized $${ pAUC}$$ p A U C ( $${ SpAUC}$$ S p A U C ) index is considered as a meaningful relative measure of predictive accuracy. Nevertheless, this $${ SpAUC}$$ S p A U C index might still show some limitations due to ROC curves crossing the diagonal line, and to the problem when comparing two tests with crossing ROC curves in the same restricted specificity range. This paper provides an alternative $${ pAUC}$$ p A U C index which overcomes these limitations. Tighter bounds for the $${ pAUC}$$ p A U C of an ROC curve are derived, and then a modified $${ pAUC}$$ p A U C index for any restricted specificity range is established. In addition, the proposed tighter partial area index ( $${ TpAUC}$$ T p A U C ) is also shown for classifier when high specificity must be clinically maintained. The variance of the $${ TpAUC}$$ T p A U C is also studied analytically and by simulation studies in a theoretical framework based on the most typical assumption of a binormal model, and estimated by using nonparametric bootstrap resampling in the empirical examples. Simulated and real datasets illustrate the practical utility of the $${ TpAUC}$$ T p A U C .

Keywords: ROC curve; Partial area under ROC curve; Classification performance; Binormal model; Bootstrap; Predictive accuracy; 62H30; 62P10 (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s11634-017-0295-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:advdac:v:12:y:2018:i:3:d:10.1007_s11634-017-0295-9

Ordering information: This journal article can be ordered from
http://www.springer. ... ds/journal/11634/PS2

DOI: 10.1007/s11634-017-0295-9

Access Statistics for this article

Advances in Data Analysis and Classification is currently edited by H.-H. Bock, W. Gaul, A. Okada, M. Vichi and C. Weihs

More articles in Advances in Data Analysis and Classification from Springer, German Classification Society - Gesellschaft für Klassifikation (GfKl), Japanese Classification Society (JCS), Classification and Data Analysis Group of the Italian Statistical Society (CLADAG), International Federation of Classification Societies (IFCS)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().