Using contribution analysis to evaluate the impacts of research on policy: Getting to ‘good enough’

Barbara L Riley, Alison Kernoghan, Lisa Stockton, Steve Montague, Jennifer Yessis and Cameron D Willis

Research Evaluation, 2018, vol. 27, issue 1, 16-27

Abstract: Assessing societal impacts of research is more difficult than assessing advances in knowledge. Methods to evaluate research impact on policy processes and outcomes are especially underdeveloped, and are needed to optimize the influence of research on policy for addressing complex issues such as chronic diseases. Contribution analysis (CA), a theory-based approach to evaluation, holds promise under these conditions of complexity. Yet applications of CA for this purpose are limited, and methods are needed to strengthen contribution claims and ensure CA is practical to implement. This article reports the experience of a public health research center in Canada that applied CA to evaluate the impacts of its research on policy changes. The main goal was to experiment with methods that were relevant to CA objectives, sufficiently rigorous for making credible claims, and feasible. Methods were ‘good enough’ if they achieved all three attributes. Three cases on government policy in tobacco control were examined: creation of smoke-free multiunit dwellings, creation of smoke-free outdoor spaces, and regulation of flavored tobacco products. Getting to ‘good enough’ required careful selection of nested theories of change; strategic use of social science theories, as well as quantitative and qualitative data from diverse sources; and complementary methods to assemble and analyze evidence for testing the nested theories of change. Some methods reinforced existing good practice standards for CA, and others were adaptations or extensions of them. Our experience may inform efforts to influence policy with research, evaluate research impacts on policy using CA, and apply CA more broadly.

Keywords: contribution analysis; research impact evaluation; strengthening quality; tobacco control policy (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://hdl.handle.net/10.1093/reseval/rvx037 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:rseval:v:27:y:2018:i:1:p:16-27.

Access Statistics for this article

Research Evaluation is currently edited by Julia Melkers, Emanuela Reale and Thed van Leeuwen

More articles in Research Evaluation from Oxford University Press
Bibliographic data for series maintained by Oxford University Press ().