Causal inferences on the effectiveness of complex social programs: Navigating assumptions, sources of complexity and evaluation design challenges

Madhabi Chatterji

Evaluation and Program Planning, 2016, vol. 59, issue C, 128-140

Abstract: This paper explores avenues for navigating evaluation design challenges posed by complex social programs (CSPs) and their environments when conducting studies that call for generalizable, causal inferences on the intervention’s effectiveness. A definition is provided of a CSP drawing on examples from different fields, and an evaluation case is analyzed in depth to derive seven (7) major sources of complexity that typify CSPs, threatening assumptions of textbook-recommended experimental designs for performing impact evaluations. Theoretically-supported, alternative methodological strategies are discussed to navigate assumptions and counter the design challenges posed by the complex configurations and ecology of CSPs. Specific recommendations include: sequential refinement of the evaluation design through systems thinking, systems-informed logic modeling; and use of extended term, mixed methods (ETMM) approaches with exploratory and confirmatory phases of the evaluation. In the proposed approach, logic models are refined through direct induction and interactions with stakeholders. To better guide assumption evaluation, question-framing, and selection of appropriate methodological strategies, a multiphase evaluation design is recommended.

Keywords: Impact evaluations; Experimental designs; Mixed methods; Causal inferences; Complex social programs (search for similar items in EconPapers)
Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0149718916301094
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:epplan:v:59:y:2016:i:c:p:128-140

DOI: 10.1016/j.evalprogplan.2016.05.009

Access Statistics for this article

Evaluation and Program Planning is currently edited by Jonathan A. Morell

More articles in Evaluation and Program Planning from Elsevier
Bibliographic data for series maintained by Catherine Liu ().