WHAT WORKS BEST AND WHEN: ACCOUNTING FOR MULTIPLE SOURCES OF PURESELECTION BIAS IN PROGRAM EVALUATIONS

Haeil Jung and Maureen A. Pirog

Journal of Policy Analysis and Management, 2014, vol. 33, issue 3, 752-777

Abstract: Most evaluations are still quasi‐experimental and most recent quasi‐experimental methodological research has focused on various types of propensity score matching to minimize conventional selection bias on observables. Although these methods create better‐matched treatment and comparison groups on observables, the issue of selection on unobservables still looms large. Thus, in the absence of being able to run randomized controlled trials (RCTs) or natural experiments, it is important to understand how well different regression‐based estimators perform in terms of minimizing pure selection bias, that is, selection on unobservables. We examine the relative magnitudes of three sources of pure selection bias: heterogeneous response bias, time‐invariant individual heterogeneity (fixed effects [FEs]), and intertemporal dependence (autoregressive process of order one [AR(1)]). Because the relative magnitude of each source of pure selection bias may vary in different policy contexts, it is important to understand how well different regression‐based estimators handle each source of selection bias. Expanding simulations that have their origins in the work of Heckman, LaLonde, and Smith ( ), we find that difference‐in‐differences (DID) using equidistant pre‐ and postperiods and FEs estimators are less biased and have smaller standard errors in estimating the Treatment on the Treated (TT) than other regression‐based estimators. Our data analysis using the Job Training Partnership Act (JTPA) program replicates our simulation findings in estimating the TT.

Date: 2014
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1002/pam.21764

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:jpamgt:v:33:y:2014:i:3:p:752-777

Access Statistics for this article

More articles in Journal of Policy Analysis and Management from John Wiley & Sons, Ltd.
Bibliographic data for series maintained by Wiley Content Delivery ().