Assumption Trade-Offs When Choosing Identification Strategies for Pre-Post Treatment Effect Estimation: An Illustration of a Community-Based Intervention in Madagascar

Weber Ann M. (), J. van der Laan Mark () and Petersen Maya L. ()
Additional contact information
Weber Ann M.: Stanford University, Stanford, CA, USA
J. van der Laan Mark: University of California – Berkeley, Berkeley, CA, USA
Petersen Maya L.: University of California – Berkeley, Berkeley, CA, USA

Journal of Causal Inference, 2015, vol. 3, issue 1, 109-130

Abstract: Failure (or success) in finding a statistically significant effect of a large-scale intervention may be due to choices made in the evaluation. To highlight the potential limitations and pitfalls of some common identification strategies used for estimating causal effects of community-level interventions, we apply a roadmap for causal inference to a pre-post evaluation of a national nutrition program in Madagascar. Selection into the program was non-random and strongly associated with the pre-treatment (lagged) outcome. Using structural causal models (SCM), directed acyclic graphs (DAGs) and simulated data, we illustrate that an estimand with the outcome defined as the post-treatment outcome controls for confounding by the lagged outcome but not by possible unmeasured confounders. Two separate differencing estimands (of the pre- and post-treatment outcome) have the potential to adjust for a certain type of unmeasured confounding, but introduce bias if the additional identification assumptions they rely on are not met. In order to illustrate the practical impact of choice between three common identification strategies and their corresponding estimands, we used observational data from the community nutrition program in Madagascar to estimate each of these three estimands. Specifically, we estimated the average treatment effect of the program on the community mean nutritional status of children 5 years and under and found that the estimate based on the post-treatment estimand was about a quarter of the magnitude of either of the differencing estimands (0.066 SD vs. 0.26–0.27 SD increase in mean weight-for-age z-score). Choice of estimand clearly has important implications for the interpretation of the success of the program to improve nutritional status of young children. A careful appraisal of the assumptions underlying the causal model is imperative before committing to a statistical model and progressing to estimation. However, knowledge about the data-generating process must be sufficient in order to choose the identification strategy that gets us closest to the truth.

Keywords: causal effect; average treatment effect; community-level intervention; difference-in-differences; change score (search for similar items in EconPapers)
Date: 2015
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1515/jci-2013-0019 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:causin:v:3:y:2015:i:1:p:109-130:n:1002

DOI: 10.1515/jci-2013-0019

Access Statistics for this article

Journal of Causal Inference is currently edited by Elias Bareinboim, Jin Tian and Iván Díaz

More articles in Journal of Causal Inference from De Gruyter
Bibliographic data for series maintained by Peter Golla ().