Estimating Counterfactual Treatment Effects to Assess External Validity

Yu-Chin Hsu (), Tsung-Chih Lai () and Robert Lieli
Additional contact information
Yu-Chin Hsu: Institute of Economics, Academia Sinica, Taipei, Taiwan, https://www.econ.sinica.edu.tw/
Tsung-Chih Lai: Department of Economics Feng Chia University

No 17-A011, IEAS Working Paper : academic research from Institute of Economics, Academia Sinica, Taipei, Taiwan

Abstract: We propose statistical methods for assessing the external validity of treatment effect estimates obtained in a specific status-quo environment. In particular, we estimate counterfactual quantile treatment effects that would obtain if one were to change the composition of the population targeted by the status-quo treatment. Assuming unconfoundedness, and the invariance of the conditional distributions of the potential outcomes, the parameter of interest is identified and can be nonparametrically estimated by a kernel-based method. Viewed as a random function over the continuum of quantile indices, the estimator converges weakly to a zero mean Gaussian process at the parametric rate. Exploiting this result, we propose a multiplier bootstrap procedure to construct uniform confidence bands. We provide similar results for the counterfactually treated subpopulation and the average effect. As an application, we estimate the counterfactual quantile treatment effect of the Job Corps training program in the U.S. under various scenarios. The results suggest that strong economic conditions and the skill hypotheses both help explain the earlier finding in the literature that the program was ineffective at low quantiles of the earnings distribution.

Keywords: counterfactual analysis; external validity; program evaluation; multiplier bootstrap; Job Corps JEL Classification: C13; C31; J24; J30 (search for similar items in EconPapers)
Pages: 47 pages
Date: 2017-08
New Economics Papers: this item is included in nep-ecm and nep-lma
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.econ.sinica.edu.tw/~econ/pdfPaper/17-A011.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sin:wpaper:17-a011

Access Statistics for this paper

More papers in IEAS Working Paper : academic research from Institute of Economics, Academia Sinica, Taipei, Taiwan Contact information at EDIRC.
Bibliographic data for series maintained by HsiaoyunLiu ().