A cautionary tale on using panel data estimators to measure program impacts

Casey Wichman and Paul Ferraro

Economics Letters, 2017, vol. 151, issue C, 82-90

Abstract: We compare experimental and nonexperimental estimates from a social and informational messaging experiment. Our results show that applying a fixed effects estimator in conjunction with matching to pre-process nonexperimental comparison groups cannot replicate an experimental benchmark, despite parallel pre-intervention trends and good covariate balance. The results are a stark reminder about the role of untestable assumptions–in our case, conditional bias stability–in drawing causal inferences from observational data, and the dangers of relying on single studies to justify program scaling-up or canceling.

Keywords: Design replication; Program evaluation; Matching; Panel data; Water conservation (search for similar items in EconPapers)
JEL-codes: C52 C93 D12 H42 Q25 (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S016517651630489X
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ecolet:v:151:y:2017:i:c:p:82-90

DOI: 10.1016/j.econlet.2016.11.029

Access Statistics for this article

Economics Letters is currently edited by Economics Letters Editorial Office

More articles in Economics Letters from Elsevier
Bibliographic data for series maintained by Catherine Liu ().