Understanding the Value of Individualized Information: The Impact of Poor Calibration or Discrimination in Outcome Prediction Models
Natalia Olchanski,
Joshua T. Cohen,
Peter J. Neumann,
John B. Wong and
David M. Kent
Medical Decision Making, 2017, vol. 37, issue 7, 790-801
Abstract:
Background. Risk prediction models allow for the incorporation of individualized risk and clinical effectiveness information to identify patients for whom therapy is most appropriate and cost-effective. This approach has the potential to identify inefficient (or harmful) care in subgroups at different risks, even when the overall results appear favorable. Here, we explore the value of personalized risk information and the factors that influence it. Methods. Using an expected value of individualized care (EVIC) framework, which monetizes the value of customizing care, we developed a general approach to calculate individualized incremental cost effectiveness ratios (ICERs) as a function of individual outcome risk. For a case study (tPA v. streptokinase to treat possible myocardial infarction), we used a simulation to explore how an EVIC is influenced by population outcome prevalence, model discrimination (c-statistic) and calibration, and willingness-to-pay (WTP) thresholds. Results. In our simulations, for well-calibrated models, which do not over- or underestimate predicted v. observed event risk, the EVIC ranged from $0 to $700 per person, with better discrimination (higher c-statistic values) yielding progressively higher EVIC values. For miscalibrated models, the EVIC ranged from −$600 to $600 in different simulated scenarios. The EVIC values decreased as discrimination improved from a c-statistic of 0.5 to 0.6, before becoming positive as the c-statistic reached values of ~0.8. Conclusions. Individualizing treatment decisions using risk may produce substantial value but also has the potential for net harm. Good model calibration ensures a non-negative EVIC. Improvements in discrimination generally increase the EVIC; however, when models are miscalibrated, greater discriminating power can paradoxically reduce the EVIC under some circumstances.
Keywords: group decision making; quality of care; translating research into practice; outcomes research (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)
Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X17704855 (text/html)
Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
HTML/Text
Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:37:y:2017:i:7:p:790-801
DOI: 10.1177/0272989X17704855
Access Statistics for this article
More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().