Using QALYs as an Outcome for Assessing Global Prediction Accuracy in Diabetes Simulation Models

Helen A. Dakin, Ni Gao, José Leal, Rury R. Holman, An Tran-Duy and Philip Clarke
Additional contact information
Helen A. Dakin: Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
Ni Gao: Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
José Leal: Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
Rury R. Holman: Diabetes Trials Unit, Radcliffe Department of Medicine, University of Oxford, UK
An Tran-Duy: Centre for Health Policy, Melbourne School of Population and Global Health, University of Melbourne, Australia
Philip Clarke: Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK

Medical Decision Making, 2025, vol. 45, issue 1, 45-59

Abstract: Objectives (1) To demonstrate the use of quality-adjusted life-years (QALYs) as an outcome measure for comparing performance between simulation models and identifying the most accurate model for economic evaluation and health technology assessment. QALYs relate directly to decision making and combine mortality and diverse clinical events into a single measure using evidence-based weights that reflect population preferences. (2) To explore the usefulness of Q 2 , the proportional reduction in error, as a model performance metric and compare it with other metrics: mean squared error (MSE), mean absolute error, bias (mean residual), and R 2 . Methods We simulated all EXSCEL trial participants ( N  = 14,729) using the UK Prospective Diabetes Study Outcomes Model software versions 1 (UKPDS-OM1) and 2 (UKPDS-OM2). The EXSCEL trial compared once-weekly exenatide with placebo (median 3.2-y follow-up). Default UKPDS-OM2 utilities were used to estimate undiscounted QALYs over the trial period based on the observed events and survival. These were compared with the QALYs predicted by UKPDS-OM1/2 for the same period. Results UKPDS-OM2 predicted patients’ QALYs more accurately than UKPDS-OM1 did (MSE: 0.210 v. 0.253; Q 2 : 0.822 v. 0.786). UKPDS-OM2 underestimated QALYs by an average of 0.127 versus 0.150 for UKPDS-OM1. UKPDS-OM2 predictions were more accurate for mortality, myocardial infarction, and stroke, whereas UKPDS-OM1 better predicted blindness and heart disease. Q 2 facilitated comparisons between subgroups and (unlike R 2 ) was lower for biased predictors. Conclusions Q 2 for QALYs was useful for comparing global prediction accuracy (across all clinical events) of diabetes models. It could be used for model registries, choosing between simulation models for economic evaluation and evaluating the impact of recalibration. Similar methods could be used in other disease areas. Highlights Diabetes simulation models are currently validated by examining their ability to predict the incidence of individual events (e.g., myocardial infarction, stroke, amputation) or composite events (e.g., first major adverse cardiovascular event). We introduce Q 2 , the proportional reduction in error, as a measure that may be useful for evaluating and comparing the prediction accuracy of econometric or simulation models. We propose using the Q 2 or mean squared error for QALYs as global measures of model prediction accuracy when comparing diabetes models’ performance for health technology assessment; these can be used to select the most accurate simulation model for economic evaluation and to evaluate the impact of model recalibration in diabetes or other conditions.

Keywords: type 2 diabetes mellitus; quality-adjusted life-years; patient-level simulation; risk modeling; model performance; microsimulation (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X241285866 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:45:y:2025:i:1:p:45-59

DOI: 10.1177/0272989X241285866

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().