The (Lack of) Impact of Impact: Why Impact Evaluations Seldom Lead to Evidence-based Policymaking

Jean-Louis Arcand

Revue d’économie du développement, 2014, vol. 22, issue HS01, 289-311

Abstract: A recurring puzzle to many academics and some policymakers is why impact evaluations, which have become something of a cottage industry in the development field, have so little impact on actual policymaking. In this paper, I study the impact of impact evaluations. I show, in a simple Bayesian framework embedded within a standard contest success function-based model of competition amongst anti-evaluation policymakers, Bayesian policymakers, and frequentist evaluators, that the likelihood of a program being cancelled is a decreasing function both of the impact estimated by the evaluation and of the prior on whose basis the program was approved to begin with. Moreover, the probability of cancellation is a decreasing function of the effectiveness of the influence exerted by frequentist evaluators. Since the latter?s effectiveness in terms of lobbying in favor of their findings in the real world is likely to be close to zero, the likelihood of cancelling a program that was approved in the first place, despite its suffering a highly negative evaluation, is extremely low. The model thus provides one possible explanation for why impact evaluations have so little impact in the realm of decisionmaking, and why they have contributed so little to evidence-based policymaking. JEL Classification: O12, D04, D72, C11, C21, C72

Keywords: impact evaluation; Bayesian analysis; contest success functions (search for similar items in EconPapers)
JEL-codes: C11 C21 C72 D04 D72 O12 (search for similar items in EconPapers)
Date: 2014
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.cairn.info/load_pdf.php?ID_ARTICLE=EDD_HS01_0289 (application/pdf)
http://www.cairn.info/revue-d-economie-du-developpement-2014-HS01-page-289.htm (text/html)
free

Related works:
Working Paper: The (lack of) impact of impact: Why impact evaluations seldom lead to evidence-based policymaking (2013)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cai:edddbu:edd_hs01_0289

Access Statistics for this article

More articles in Revue d’économie du développement from De Boeck Université Contact information at EDIRC.
Bibliographic data for series maintained by Jean-Baptiste de Vathaire ().