Sample Attrition Bias in Randomized Experiments: A Table of Two Surveys

Luc Behaghel, Bruno Crépon, Marc Gurgand and Thomas Le Barbanchon

No 2009-05, Working Papers from Center for Research in Economics and Statistics

Abstract: The randomized trial literature has helped to renew the field of microeconometric policy evaluation byemphasizing identification issues raised by endogenous program participation. Measurement andattrition issues have perhaps received less attention. This paper analyzes the dramatic impact of sampleattrition in a large job search experiment. We take advantage of two independent surveys on the sameinitial sample of 8,000 persons. The first one is a long telephone survey that had a strikingly low andunbalanced response rate of about 50%. The second one is a combination of administrative data and ashort telephone survey targeted at those leaving the unemployment registers; this enriched data sourcehas a balanced and much higher response rate (about 80%). With naive estimates that neglect nonresponses, these two sources yield puzzlingly different results.Using the enriched administrative data as benchmark, we find evidence that estimates from the longtelephone survey lack external and internal validity. We turn to existing methods to bound the effectsin the presence of sample selection; we extend them to the context of randomization with imperfectcompliance. The bounds obtained from the two surveys are compatible but those from the longtelephone survey are somewhat uninformative. We conclude on the consequences for data collectionstrategies.

Pages: 30
Date: 2009
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://crest.science/RePEc/wpstorage/2009-05.pdf Crest working paper version (application/pdf)

Related works:
Working Paper: Sample attrition bias in randomized experiments: A tale of two surveys (2009)
Working Paper: Sample attrition bias in randomized experiments: A tale of two surveys (2009)
Working Paper: Sample Attrition Bias in Randomized Experiments: A Tale of Two Surveys (2009)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:crs:wpaper:2009-05

Access Statistics for this paper

More papers in Working Papers from Center for Research in Economics and Statistics Contact information at EDIRC.
Bibliographic data for series maintained by Secretariat General () and Murielle Jules Maintainer-Email : murielle.jules@ensae.Fr.