A new experiment on the use of images to answer web survey questions

Oriol Bosch Jover, Melanie Revilla, Danish Daniel Qureshi and Jan Höhne

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: Images might provide richer and more objective information than text answers to open-ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open-ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between-subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open-ended questions with text; (2) a treatment group answering open-ended questions with images; and (3) another treatment group answering open-ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants' likelihood of noncompliance as well as their completion times, while worsening their overall survey experience. Including motivational messages, moreover, moderately reduces the likelihood of noncompliance. Finally, the likelihood of noncompliance is similar across devices.

Keywords: breakoff; images; motivational messages; noncompliance; smartphone; web survey (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 26 pages
Date: 2022-07-01
New Economics Papers: this item is included in nep-dem and nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in Journal of the Royal Statistical Society. Series A: Statistics in Society, 1, July, 2022, 185(3), pp. 955 - 980. ISSN: 0964-1998

Downloads: (external link)
http://eprints.lse.ac.uk/114899/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:114899

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager (lseresearchonline@lse.ac.uk).