Program Evaluation of Population- and System-Level Policies: Evidence for Decision Making

Simon Walker, Aimee Fox, James Altunkaya, Tim Colbourn, Mike Drummond, Susan Griffin, Nils Gutacker, Paul Revill and Mark Sculpher
Additional contact information
Simon Walker: Centre for Health Economics, University of York, York, UK
Aimee Fox: Adelphi Values, Bollington, Cheshire, UK
Tim Colbourn: Institute for Global Health, University College London, London, UK
Mike Drummond: Centre for Health Economics, University of York, York, UK
Susan Griffin: Centre for Health Economics, University of York, York, UK
Nils Gutacker: Centre for Health Economics, University of York, York, UK
Mark Sculpher: Centre for Health Economics, University of York, York, UK

Medical Decision Making, 2022, vol. 42, issue 1, 17-27

Abstract: Background Policy evaluations often focus on ex post estimation of causal effects on short-term surrogate outcomes. The value of such information is limited for decision making, as the failure to reflect policy-relevant outcomes and disregard for opportunity costs prohibits the assessment of value for money. Further, these evaluations do not always consider all relevant evidence, other courses of action, or decision uncertainty. Methods In this article, we explore how policy evaluation could better meet the needs of decision making. We begin by defining the evidence required to inform decision making. We then conduct a literature review of challenges in evaluating policies. Finally, we highlight potential methods available to help address these challenges. Results The evidence required to inform decision making includes the impacts on the policy-relevant outcomes, the costs and associated opportunity costs, and the consequences of uncertainty. Challenges in evaluating health policies are described using 8 categories: 1) valuation space; 2) comparators; 3) time of evaluation; 4) mechanisms of action; 5) effects; 6) resources, constraints, and opportunity costs; 7) fidelity, adaptation, and level of implementation; and 8) generalizability and external validity. Methods from a broad set of disciplines are available to improve policy evaluation, relating to causal inference, decision-analytic modeling, theory of change, realist evaluation, and structured expert elicitation. Limitations The targeted review may not identify all possible challenges, and the methods covered are not exhaustive. Conclusions Evaluations should provide appropriate evidence to inform decision making. There are challenges in evaluating policies, but methods from multiple disciplines are available to address these challenges. Implications Evaluators need to carefully consider the decision being informed, the necessary evidence to inform it, and the appropriate methods. Highlights Evaluating policies by estimating their causal effects on short-term surrogate outcomes is, in isolation, of limited value for decision making. Evidence for informing decision making needs to link policies to relevant outcomes, costs, and associated opportunity costs and reflect the magnitude and consequences of uncertainty in these estimations. Challenges in program evaluation range across defining the valuation space and comparators, understanding mechanisms of action and effects, estimating opportunity costs, and external validity. Methods from multiple disciplines are available to address these challenges.

Keywords: cost-effectiveness; economic evaluation; programm evaluation (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0272989X211016427 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:medema:v:42:y:2022:i:1:p:17-27

DOI: 10.1177/0272989X211016427

Access Statistics for this article

More articles in Medical Decision Making
Bibliographic data for series maintained by SAGE Publications ().