Dealing with randomisation bias in a social experiment exploiting the randomisation itself: the case of ERA

Barbara Sianesi ()
Additional contact information
Barbara Sianesi: Institute for Fiscal Studies and Institute for Fiscal Studies

No W13/15, IFS Working Papers from Institute for Fiscal Studies

Abstract: We highlight the importance of randomisation bias, a situation where the process of participation in a social experiment has been affected by randomisation per se. We illustrate how this has happened in the case of the UK Employment Retention and Advancement (ERA) experiment, in which over one quarter of the eligible population was not represented. Our objective is to quantify the impact that the ERA eligible population would have experienced under ERA, and to assess how this impact relates to the experimental impact estimated on the potentially selected subgroup of study participants. We show that the typical matching assumption required to identify the average treatment effect of interest is made up of two parts. One part remains testable under the experiment even in the presence of randomisation bias, and offers a way to correct the non-experimental estimates should they fail to pass the test. The other part rests on what we argue is a very weak assumption, at least in the case of ERA. We implement these ideas to the ERA program and show the power of this strategy. Further exploiting the experiment we assess the validity in our application of the claim often made in the literature that knowledge of long and detailed labour market histories can control for most selection bias in the evaluation of labour market interventions. Finally, for the case of survey-based outcomes, we develop a reweighting estimator which takes account of both non-participation and non-response.

Keywords: social experiments; sample selection; treatment effects; matching methods; reweighting estimators (search for similar items in EconPapers)
JEL-codes: C21 J18 J38 (search for similar items in EconPapers)
Date: 2013-07-22
New Economics Papers: this item is included in nep-ecm and nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.ifs.org.uk/wps/wp201315.pdf (application/pdf)
Our link check indicates that this URL is bad, the error code is: 404 Not Found (http://www.ifs.org.uk/wps/wp201315.pdf [301 Moved Permanently]--> https://www.ifs.org.uk/wps/wp201315.pdf [302 Found]--> https://ifs.org.uk/wps/wp201315.pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ifs:ifsewp:13/15

Ordering information: This working paper can be ordered from
The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE

Access Statistics for this paper

More papers in IFS Working Papers from Institute for Fiscal Studies The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE. Contact information at EDIRC.
Bibliographic data for series maintained by Emma Hyman ().