Choosing a Metamodel of a Simulation Model for Uncertainty Quantification

Tiago M. de Carvalho, Joost van Rosmalen, Harold B. Wolff, Hendrik Koffijberg and Veerle M. H. Coupé
Additional contact information
Tiago M. de Carvalho: Department of Epidemiology and Biostatistics, Amsterdam UMC, Location VUMC, Amsterdam, the Netherlands
Joost van Rosmalen: Department of Epidemiology, Erasmus MC
Harold B. Wolff: Department of Epidemiology and Biostatistics, Amsterdam UMC, Location VUMC, Amsterdam, the Netherlands
Hendrik Koffijberg: Health Technology and Services Research Department, Faculty of Behavioral Management and Social Sciences, Technical Medical Centre, University of Twente, Enschede, the Netherlands
Veerle M. H. Coupé: Department of Epidemiology and Biostatistics, Amsterdam UMC, Location VUMC, Amsterdam, the Netherlands

Medical Decision Making, 2022, vol. 42, issue 1, 28-42

Abstract: Background Metamodeling may substantially reduce the computational expense of individual-level state transition simulation models (IL-STM) for calibration, uncertainty quantification, and health policy evaluation. However, because of the lack of guidance and readily available computer code, metamodels are still not widely used in health economics and public health. In this study, we provide guidance on how to choose a metamodel for uncertainty quantification. Methods We built a simulation study to evaluate the prediction accuracy and computational expense of metamodels for uncertainty quantification using life-years gained (LYG) by treatment as the IL-STM outcome. We analyzed how metamodel accuracy changes with the characteristics of the simulation model using a linear model (LM), Gaussian process regression (GP), generalized additive models (GAMs), and artificial neural networks (ANNs). Finally, we tested these metamodels in a case study consisting of a probabilistic analysis of a lung cancer IL-STM. Results In a scenario with low uncertainty in model parameters (i.e., small confidence interval), sufficient numbers of simulated life histories, and simulation model runs, commonly used metamodels (LM, ANNs, GAMs, and GP) have similar, good accuracy, with errors smaller than 1% for predicting LYG. With a higher level of uncertainty in model parameters, the prediction accuracy of GP and ANN is superior to LM. In the case study, we found that in the worst case, the best metamodel had an error of about 2.1%. Conclusion To obtain good prediction accuracy, in an efficient way, we recommend starting with LM, and if the resulting accuracy is insufficient, we recommend trying ANNs and eventually also GP regression.

Keywords: cost-effectiveness analysis; metamodels/emulators; probabilistic sensitivity analyses; simulation models; uncertainty quantification (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X211016307 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:42:y:2022:i:1:p:28-42

DOI: 10.1177/0272989X211016307

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().