How to conduct impact evaluations in humanitarian and conflict settings

Aysegül Kayaoglu, Ghassan Baliki (), Tilman Brück (), Melodie Al Daccache and Dorothee Weiffen ()
Additional contact information
Aysegül Kayaoglu: ISDC - International Security and Development Center, Germany; Department of Economics, Istanbul Technical University, Türkiye; IMIS, University of Osnabrück, Germany
Ghassan Baliki: ISDC - International Security and Development Center, Germany
Tilman Brück: Humboldt-University of Berlin, Germany; ISDC - International Security and Development Center, Berlin, Germany; Thaer-Institute, Humboldt-University of Berlin, Germany; Leibniz Institute of Vegetable and Ornamental Crops (IGZ), Germany
Melodie Al Daccache: American University of Beirut, Lebanon
Dorothee Weiffen: ISDC - International Security and Development Center, Germany

No 387, HiCN Working Papers from Households in Conflict Network

Abstract: Methodological, ethical and practical challenges make it difficult to use experimental and rigorous quasi-experimental approaches to conduct impact evaluations in humanitarian emergencies and conflict settings (HECS). This paper discusses recent developments in the design, measurement, data and analysis of impact evaluations that can overcome these challenges and provide concrete examples from our recent research where we analyse the impact of agricultural emergency interventions in post-war Syria. More specifically, the paper offers solutions: First, discuss the challenges in designing rapid and rigorous impact evaluations in HECS. By doing so, we mainly show alternative ways to construct counterfactuals in the absence of meaningful control groups; Second, we review how researchers can use additional data sources to create a counterfactual or even data on treated units when it is difficult to collect data and in some cases provide ethical and methodological benefits in addition to providing cost-effectiveness. Third, we argue that finding and fine-tuning proxy measures for the ‘unmeasurable’ concepts and outcomes such as resilience and fragility are crucial. Fourth, we highlight how adaptive machine learning algorithms are helpful in rigorous impact evaluations in HECS to overcome the drawbacks related to data availability and heterogeneity analysis. We provide an example from our recent work where we use honest causal forest estimation to test the heterogeneous impact of an agricultural intervention when sample sizes are small. Fifth, we discuss how standardisation across methods, data and measures ensures the external validity and transferability of the evidence to other complex settings where impact evaluation is challenging to conduct. Finally, the paper recommends how future research and policy can adapt these tools to ensure significant and effective learning in conflict-affected and humanitarian settings.

Keywords: impact evaluation; research design; machine learning; conflict setting; humanitarian emergencies (search for similar items in EconPapers)
JEL-codes: C18 C30 C80 D04 D74 Q34 (search for similar items in EconPapers)
Pages: 37 pages
Date: 2023-03
New Economics Papers: this item is included in nep-big, nep-des and nep-mac
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://hicn.org/wp-content/uploads/2023/03/HiCN-WP-387.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hic:wpaper:387

Access Statistics for this paper

More papers in HiCN Working Papers from Households in Conflict Network
Bibliographic data for series maintained by Tilman Brück () and () and () and ().