Methods for Model Calibration under High Uncertainty: Modeling Cholera in Bangladesh

Theresa Ryckman, Stephen Luby, Douglas K. Owens, Eran Bendavid and Jeremy D. Goldhaber-Fiebert
Additional contact information
Theresa Ryckman: Center for Health Policy and Center for Primary Care & Outcomes Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA
Stephen Luby: Infectious Diseases and Geographic Medicine, Stanford University School of Medicine, Stanford University, Stanford, CA, USA
Douglas K. Owens: VA Palo Alto Healthcare System, Palo Alto, CA, USA
Eran Bendavid: Division of Primary Care and Population Health, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA
Jeremy D. Goldhaber-Fiebert: Center for Health Policy and Center for Primary Care & Outcomes Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA

Medical Decision Making, 2020, vol. 40, issue 5, 693-709

Abstract: Background. Published data on a disease do not always correspond directly to the parameters needed to simulate natural history. Several calibration methods have been applied to computer-based disease models to extract needed parameters that make a model’s output consistent with available data. Objective. To assess 3 calibration methods and evaluate their performance in a real-world application. Methods. We calibrated a model of cholera natural history in Bangladesh, where a lack of active surveillance biases available data. We built a cohort state-transition cholera natural history model that includes case hospitalization to reflect the passive surveillance data-generating process. We applied 3 calibration techniques: incremental mixture importance sampling, sampling importance resampling, and random search with rejection sampling. We adapted these techniques to the context of wide prior uncertainty and many degrees of freedom. We evaluated the resulting posterior parameter distributions using a range of metrics and compared predicted cholera burden estimates. Results. All 3 calibration techniques produced posterior distributions with a higher likelihood and better fit to calibration targets as compared with prior distributions. Incremental mixture importance sampling resulted in the highest likelihood and largest number of unique parameter sets to better inform joint parameter uncertainty. Compared with naïve uncalibrated parameter sets, calibrated models of cholera in Bangladesh project substantially more cases, many of which are not detected by passive surveillance, and fewer deaths. Limitations. Calibration cannot completely overcome poor data quality, which can leave some parameters less well informed than others. Calibration techniques may perform differently under different circumstances. Conclusions . Incremental mixture importance sampling, when adapted to the context of high uncertainty, performs well. By accounting for biases in data, calibration can improve model projections of disease burden.

Keywords: Bayesian calibration; calibration; cholera; decision-analytic models; estimation (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X20938683 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:40:y:2020:i:5:p:693-709

DOI: 10.1177/0272989X20938683

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().