Metamodeling for Policy Simulations with Multivariate Outcomes

Huaiyang Zhong, Margaret L. Brandeau, Golnaz Eftekhari Yazdi, Jianing Wang, Shayla Nolen, Liesl Hagan, William W. Thompson, Sabrina A. Assoumou, Benjamin P. Linas and Joshua A. Salomon
Additional contact information
Huaiyang Zhong: Department of Management Science and Engineering, Stanford University, Stanford, CA, USA
Margaret L. Brandeau: Department of Management Science and Engineering, Stanford University, Stanford, CA, USA
Golnaz Eftekhari Yazdi: Section of Infectious Diseases, Department of Medicine, Boston Medical Center, Boston, MA, USA
Jianing Wang: Section of Infectious Diseases, Department of Medicine, Boston Medical Center, Boston, MA, USA
Shayla Nolen: Section of Infectious Diseases, Department of Medicine, Boston Medical Center, Boston, MA, USA
William W. Thompson: Division of Viral Hepatitis, Center for Disease Control and Prevention, Atlanta, GA, USA
Sabrina A. Assoumou: Section of Infectious Diseases, Department of Medicine, Boston Medical Center, Boston, MA, USA
Benjamin P. Linas: Section of Infectious Diseases, Department of Medicine, Boston Medical Center, Boston, MA, USA
Joshua A. Salomon: Center for Health Policy and Center for Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA

Medical Decision Making, 2022, vol. 42, issue 7, 872-884

Abstract: Purpose Metamodels are simplified approximations of more complex models that can be used as surrogates for the original models. Challenges in using metamodels for policy analysis arise when there are multiple correlated outputs of interest. We develop a framework for metamodeling with policy simulations to accommodate multivariate outcomes. Methods: We combine 2 algorithm adaptation methods—multitarget stacking and regression chain with maximum correlation—with different base learners including linear regression (LR), elastic net (EE) with second-order terms, Gaussian process regression (GPR), random forests (RFs), and neural networks. We optimize integrated models using variable selection and hyperparameter tuning. We compare the accuracy, efficiency, and interpretability of different approaches. As an example application, we develop metamodels to emulate a microsimulation model of testing and treatment strategies for hepatitis C in correctional settings. Results Output variables from the simulation model were correlated (average Ï = 0.58). Without multioutput algorithm adaptation methods, in-sample fit (measured by R 2 ) ranged from 0.881 for LR to 0.987 for GPR. The multioutput algorithm adaptation method increased R 2 by an average 0.002 across base learners. Variable selection and hyperparameter tuning increased R 2 by 0.009. Simpler models such as LR, EE, and RF required minimal training and prediction time. LR and EE had advantages in model interpretability, and we considered methods for improving the interpretability of other models. Conclusions In our example application, the choice of base learner had the largest impact on R 2 ; multioutput algorithm adaptation and variable selection and hyperparameter tuning had a modest impact. Although advantages and disadvantages of specific learning algorithms may vary across different modeling applications, our framework for metamodeling in policy analyses with multivariate outcomes has broad applicability to decision analysis in health and medicine.

Keywords: machine learning; metamodeling; model interpretability; simulation modeling (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X221105079 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:42:y:2022:i:7:p:872-884

DOI: 10.1177/0272989X221105079

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().