The Sensitivity of Experimental Impact Estimates: Evidence from the National JTPA Study

James Heckman and Jeffrey Smith

No 6105, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: The recent experimental evaluation of the U.S. Job Training Partnership Act (JTPA) program found negative effects of training on the earnings of disadvantaged male youth and no effect on the earnings of disadvantaged female youth. These findings provided justification for Congress to cut the budget of JTPA's youth component by over 80 percent. In this paper, we examine the sensitivity of the experimental impact estimates along several dimensions of construction and interpretation. We find that the statistical significance of the male youth estimates is extremely fragile and that the magnitudes of the estimates for both youth groups are sensitive to nearly all the factors we consider. In particular, accounting for experimental control group members who substitute training from other providers leads to a much more positive picture regarding the effectiveness of JTPA classroom training. Our study indicates the value of sensitivity analyses in experimental evaluations and illustrates that experimental impact estimates, like those from nonexperimental analyses, require careful interpretation if they are to provide a reliable guide to policymakers.

JEL-codes: C93 H43 (search for similar items in EconPapers)
Date: 1997-07
Note: LS
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Published as Youth Employment and Joblessness in Advanced Countries, Blanchflower, David and Richard Freeman, eds., Chicago: University of Chicago Press, 2000, pp. 331-356.
Published as The Sensitivity of Experimental Impact Estimates (Evidence from the National JTPA Study) , James J. Heckman, Jeffrey Smith. in Youth Employment and Joblessness in Advanced Countries , Blanchflower and Freeman. 2000

Downloads: (external link)
http://www.nber.org/papers/w6105.pdf (application/pdf)

Related works:
Chapter: The Sensitivity of Experimental Impact Estimates (Evidence from the National JTPA Study) (2000)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:6105

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w6105

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().