Simple Decision-Analytic Functions of the AUC for Ruling Out a Risk Prediction Model and an Added Predictor

Stuart G. Baker

Medical Decision Making, 2018, vol. 38, issue 2, 225-234

Abstract: Background. When using risk prediction models, an important consideration is weighing performance against the cost (monetary and harms) of ascertaining predictors. Methods. The minimum test tradeoff (MTT) for ruling out a model is the minimum number of all-predictor ascertainments per correct prediction to yield a positive overall expected utility. The MTT for ruling out an added predictor is the minimum number of added-predictor ascertainments per correct prediction to yield a positive overall expected utility. Results. An approximation to the MTT for ruling out a model is 1/[P (H(AUC model )], where H(AUC) = AUC – {½ (1–AUC)} ½ , AUC is the area under the receiver operating characteristic (ROC) curve, and P is the probability of the predicted event in the target population. An approximation to the MTT for ruling out an added predictor is 1 /[P {(H(AUC Model:2 ) – H(AUC Model:1 )], where Model 2 includes an added predictor relative to Model 1. Limitation . The latter approximation requires the Tangent Condition that the true positive rate at the point on the ROC curve with a slope of 1 is larger for Model 2 than Model 1. Conclusion. These approximations are suitable for back-of-the-envelope calculations. For example, in a study predicting the risk of invasive breast cancer, Model 2 adds to the predictors in Model 1 a set of 7 single nucleotide polymorphisms (SNPs). Based on the AUCs and the Tangent Condition, an MTT of 7200 was computed, which indicates that 7200 sets of SNPs are needed for every correct prediction of breast cancer to yield a positive overall expected utility. If ascertaining the SNPs costs $500, this MTT suggests that SNP ascertainment is not likely worthwhile for this risk prediction.

Keywords: AUC; decision analysis; net benefit; ROC curve; relative utility; risk prediction (search for similar items in EconPapers)
Date: 2018
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X17732994 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:38:y:2018:i:2:p:225-234

DOI: 10.1177/0272989X17732994

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().