An Empirical Unravelling of Lord's Paradox

ZhiMin Xiao, Steve Higgins and Adetayo Kasim
Additional contact information
ZhiMin Xiao: University of Exeter

No auq24, OSF Preprints from Center for Open Science

Abstract: Lord's Paradox occurs when a continuous covariate is statistically controlled for and the relationship between a continuous outcome and group status indicator changes in both magnitude and direction. This phenomenon poses a challenge to the notion of evidence-based policy, where data are supposed to be self-evident. We examined 50 effect size estimates from 34 large-scale educational interventions, and found that impact estimates are affected in magnitude, with or without reversal in sign, when there is substantial baseline imbalance. We also demonstrated that multilevel modelling can ameliorate the divergence in sign and/or magnitude of effect estimation, which, together with project specific knowledge, promises to help those who are presented with conflicting or confusing evidence in decision making.

Date: 2017-08-12
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/598f40f36c613b0259f3f194/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:osfxxx:auq24

DOI: 10.31219/osf.io/auq24

Access Statistics for this paper

More papers in OSF Preprints from Center for Open Science
Bibliographic data for series maintained by OSF ().