Validation of a Cardiovascular Disease Policy Microsimulation Model Using Both Survival and Receiver Operating Characteristic Curves

Ankur Pandya, Stephen Sy, Sylvia Cho, Sartaj Alam, Milton C. Weinstein and Thomas A. Gaziano

Medical Decision Making, 2017, vol. 37, issue 7, 802-814

Abstract: Background. Despite some advances, cardiovascular disease (CVD) remains the leading cause of death and healthcare costs in the United States. We therefore developed a comprehensive CVD policy simulation model that identifies cost-effective approaches for reducing CVD burden. This paper aims to: 1) describe our model in detail; and 2) perform model validation analyses. Methods. The model simulates 1,000,000 adults (ages 35 to 80 years) using a variety of CVD-related epidemiological data, including previously calibrated Framingham-based risk scores for coronary heart disease and stroke. We validated our microsimulation model using recent National Health and Nutrition Examination Survey (NHANES) data, with baseline values collected in 1999-2000 and cause-specific mortality follow-up through 2011. Model-based (simulated) results were compared to observed all-cause and CVD-specific mortality data (from NHANES) for the same starting population using survival curves and, in a method not typically used for disease model validation, receiver operating characteristic (ROC) curves. Results. Observed 10-year all-cause mortality in NHANES v. the simulation model was 11.2% (95% CI, 10.3% to 12.2%) v. 10.9%; corresponding results for CVD mortality were 2.2% (1.8% to 2.7%) v. 2.6%. Areas under the ROC curves for model-predicted 10-year all-cause and CVD mortality risks were 0.83 (0.81 to 0.85) and 0.84 (0.81 to 0.88), respectively; corresponding results for 5-year risks were 0.80 (0.77 to 0.83) and 0.81 (0.75 to 0.87), respectively. Limitations. The model is limited by the uncertainties in the data used to estimate its input parameters. Additionally, our validation analyses did not include non-fatal CVD outcomes due to NHANES data limitations. Conclusions. The simulation model performed well in matching to observed nationally representative longitudinal mortality data. ROC curve analysis, which has been traditionally used for risk prediction models, can also be used to assess discrimination for disease simulation models.

Keywords: cardiovascular disease; modeling; cost-effectiveness; validation; microsimulation (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X17706081 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:37:y:2017:i:7:p:802-814

DOI: 10.1177/0272989X17706081

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().