The efficacy of propensity score matching for separating selection and measurement effects across different survey modes

Eliud Kibuchi, Patrick Sturgis, Gabriele B. Durrant and Olga Maslovskaya

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: Effective evaluation of data quality between data collected in different modes is complicated by the confounding of selection and measurement effects. This study evaluates the utility of propensity score matching (PSM) as a method that has been proposed as a means of removing selection effects across surveys conducted in different modes. Our results show large differences in estimates for the same variables between parallel face-to-face and online surveys, even after matching on standard demographic variables. Moreover, discrepancies in estimates are still present after matching between surveys conducted in the same (online) mode, where differences in measurement properties can be ruled out a priori. Our findings suggest that PSM has substantial limitations as a method for separating measurement and selection differences across modes and should be used only with caution.

Keywords: face-to-face interviews; online surveys; mixed-mode; propensity score matching; mode effects; as part of Work Package 1 of the ESRC National Centre for Research Methods (2014-2019) (grant number ES/ L008351/1); the ESRC National Centre for Research Methods (2020-2025) (grant number ES/T000066/1; PhD Studentship grant number ES/ J500161/1; ES/L008351/1; ES/T000066/1; ES/J500161/1; MC UU_00022/2 (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 26 pages
Date: 2024-06-01
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in Journal of Survey Statistics and Methodology, 1, June, 2024, 12(3), pp. 764 - 789. ISSN: 2325-0984

Downloads: (external link)
http://eprints.lse.ac.uk/122120/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:122120

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().