When, and how much, might effect estimates be biased when analysing mixed-mode survey data? The roles of mode effects, mode selection, and mode split

Georgia D Tomova, Richard J. Silverwood and Liam Wright

No 6sxbq_v1, SocArXiv from Center for Open Science

Abstract: Survey data are increasingly collected using mixed-mode designs (e.g. personal interview and web questionnaire). Little is known, however, about the extent to which this introduces bias in subsequent analyses, or whether simply including mode as a covariate addresses it. Using data simulations, we identified the conditions under which mode effects (mode measurement differences) and mode selection (respondent differences) introduce bias, complementing this with empirical illustrations using mixed-mode data from the 1958 National Child Development Study. In simulations, absent mode selection, substantial bias arose only from unusually large mode effects, but was amplified when mode selection was present. Controlling for mode under strong mode selection introduced substantial bias, including sign-reversal of the estimate. Bias was more pronounced when the sample was split equally between modes. The direction and size of bias will depend on the direction of all effects. In our empirical illustration, the results were largely unchanged after controlling for mode, possibly reflecting weaker mode selection and mode effects in these data. These findings highlight that, while possible, substantial bias from mixed-mode designs may be unlikely in many practical settings. However, the risk of bias, and the appropriate strategy to address it, should be considered on an analysis-specific basis.

Date: 2026-06-16
References: Add references at CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6a2f64e2bb8eaf75696dbc19/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:6sxbq_v1

DOI: 10.31219/osf.io/6sxbq_v1

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().