Impact evaluation in field settings: experience from a complex NGO programme in Ethiopia

Bridget Fenn

Journal of Development Effectiveness, 2012, vol. 4, issue 4, 566-577

Abstract: Evidence from impact evaluations of community-based nutrition interventions addressing the question of 'what works' to reduce linear growth retardation (stunting) is growing but consensus is mixed. Whilst the number of such impact evaluations is increasing, especially by non-governmental organisations, many studies fail to be publicised due to unclear and/or limited positive outcomes, denying experiences of lessons learnt. Those that are published need to be scrutinised to understand the methods used in deciding the validity of the reported outcomes, since impact evaluations in operation settings can suffer a number of limitations. The author uses as an example an integrated community-based non-randomised controlled impact evaluation to highlight some of the problems and issues encountered in carrying out impact evaluations in operational settings and how results need careful interpretation. In this Save the Children UK study, the only intervention to show a positive significant impact in reducing linear growth retardation (stunting) in children 6-36 months was water, sanitation and hygiene. A comprehensive analysis of the nature, reliability and robustness of the study was necessary to determine the validity of the results. This article looks at the important factors that may have been influential in these outcomes, including key issues generally faced in impact evaluations that threaten both the external and internal validity through bias, confounding, contamination and spillover effects as well as implementation problems encountered. The main issues in this evaluation were lack of randomisation and power, alongside implementation issues, resulting in potential confounding and bias possibly producing spurious results. Agencies involved in community-based impact evaluations should engage in robust methods and systematic reporting to create a stronger evidence base and as such be prepared to increase the transparency of their work.

Date: 2012
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/19439342.2012.725085 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:jdevef:v:4:y:2012:i:4:p:566-577

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/RJDE20

DOI: 10.1080/19439342.2012.725085

Access Statistics for this article

Journal of Development Effectiveness is currently edited by Howard White

More articles in Journal of Development Effectiveness from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().