How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment

Elizabeth Wilde and Robinson Hollister
Additional contact information
Robinson Hollister: Swarthmore College, Postal: Swarthmore College

Journal of Policy Analysis and Management, 2007, vol. 26, issue 3, 455-477

Abstract: In recent years, propensity score matching (PSM) has gained attention as a potential method for estimating the impact of public policy programs in the absence of experimental evaluations. In this study, we evaluate the usefulness of PSM for estimating the impact of a program change in an educational context (Tennessee's Student Teacher Achievement Ratio Project [Project STAR]). Because Tennessee's Project STAR experiment involved an effective random assignment procedure, the experimental results from this policy intervention can be used as a benchmark, to which we compare the impact estimates produced using propensity score matching methods. We use several different methods to assess these nonexperimental estimates of the impact of the program. We try to determine “how close is close enough,” putting greatest emphasis on the question: Would the nonexperimental estimate have led to the wrong decision when compared to the experimental estimate of the program? We find that propensity score methods perform poorly with respect to measuring the impact of a reduction in class size on achievement test scores. We conclude that further research is needed before policymakers rely on PSM as an evaluation tool. © 2007 by the Association for Public Policy Analysis and Management

Date: 2007
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (29)

Downloads: (external link)
http://hdl.handle.net/10.1002/pam.20262 Link to full text; subscription required (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:jpamgt:v:26:y:2007:i:3:p:455-477

DOI: 10.1002/pam.20262

Access Statistics for this article

More articles in Journal of Policy Analysis and Management from John Wiley & Sons, Ltd.
Bibliographic data for series maintained by Wiley Content Delivery ().