Missing, presumed different: Quantifying the risk of attrition bias in education evaluations

Ben Weidmann and Luke Miratrix

Journal of the Royal Statistical Society Series A, 2021, vol. 184, issue 2, 732-760

Abstract: We estimate the magnitude of attrition bias for 10 randomized controlled trials (RCTs) in education. We make use of a unique feature of administrative school data in England that allows us to analyse post‐test academic outcomes for nearly all students, including those who originally dropped out of the RCTs we analyse. We find that the typical magnitude of attrition bias is 0.015 effect size units (ES), with no estimate greater than 0.034 ES. This suggests that, in practice, the risk of attrition bias is limited. However, this risk should not be ignored as we find some evidence against the common ‘Missing At Random’ assumption. Attrition appears to be more problematic for treated units. We recommend that researchers incorporate uncertainty due to attrition bias, as well as performing sensitivity analyses based on the types of attrition mechanisms that are observed in practice.

Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/rssa.12677

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jorssa:v:184:y:2021:i:2:p:732-760

Ordering information: This journal article can be ordered from
http://ordering.onli ... 1111/(ISSN)1467-985X

Access Statistics for this article

Journal of the Royal Statistical Society Series A is currently edited by A. Chevalier and L. Sharples

More articles in Journal of the Royal Statistical Society Series A from Royal Statistical Society Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().