Are Factorial Survey Experiments Prone to Survey Mode Effects?

Katrin Auspurg, Thomas Hinz and Sandra Walzenbach

EconStor Open Access Articles and Book Chapters, 2019, 371-392

Abstract: Drawing on data from a nation-wide survey on the fairness of earnings, this piece of work examines the susceptibility of factorial survey experiments to mode effects. It uses multilevel models to compare the results from a completely self-administered questionnaire to those of an experimental condition in which interviewers were present. From a theoretical point of view, interviewers might affect measurement error in different directions. On the one hand, they can provide explanations and assistance if respondents feel unsure about answering the rather complex vignette questions, or even motivate them to put more effort into the response process. As a result, respondents should be less inclined to use satisficing as a response strategy. On the other hand, respondents might feel pressured to provide answers quickly and in line with what is socially desirable. We compare the two survey modes with regard to a set of indicators for data quality: nonresponse, common heuristics (ticking the middle and straight-lining), response consistency throughout the vignette set and social desirability bias. The latter is examined by analysing substantive results from the vignette module, which allows for quantifying the gender wage gap that respondents perceive as just. The study discusses to what extent interviewers can relieve the cognitive burden associated with factorial surveys and positively affect data quality. Do the favourable effects of interviewers outweigh the negative ones? The study can help practitioners to decide if the gains in data quality are great enough to justify investment in a more expensive personal survey mode.

Keywords: factorial survey; vignettes; mode effects; interviewer effects; sensitive questions; gender wage gap (search for similar items in EconPapers)
Date: 2019
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/231408/1/2021-03-04-book-chapter.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:espost:231408

Access Statistics for this article

More articles in EconStor Open Access Articles and Book Chapters from ZBW - Leibniz Information Centre for Economics Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().