Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies

Anna Heath, Natalia Kunst, Christopher Jackson, Mark Strong, Fernando Alarid-Escudero, Jeremy D. Goldhaber-Fiebert, Gianluca Baio, Nicolas A. Menzies and Hawre Jalal
Additional contact information
Anna Heath: The Hospital for Sick Children, Toronto, ON, Canada
Natalia Kunst: Department of Health Management and Health Economics, Institute of Health and Society, Faculty of Medicine, University of Oslo, Oslo, Norway
Christopher Jackson: MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
Mark Strong: School of Health and Related Research, University of Sheffield, Sheffield, UK
Fernando Alarid-Escudero: Center for Research and Teaching in Economics (CIDE)
Jeremy D. Goldhaber-Fiebert: Stanford Health Policy, Centers for Health Policy and Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA
Gianluca Baio: University College London, London, UK
Nicolas A. Menzies: Harvard T. H. Chan School of Public Health, Boston, MA, USA
Hawre Jalal: University of Pittsburgh, Pittsburgh, PA, USA

Medical Decision Making, 2020, vol. 40, issue 3, 314-326

Abstract: Background. Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. Methods. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. Results. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. Conclusions. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.

Keywords: computation methods; expected value of sample information; health economic decision modelling; study design; value of information (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X20912402 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:40:y:2020:i:3:p:314-326

DOI: 10.1177/0272989X20912402

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().