Instructional Manipulation Checks: A longitudinal analysis with implications for MTurk

Leonard J. Paas, Sara Dolnicar and Logi Karlsson

International Journal of Research in Marketing, 2018, vol. 35, issue 2, 258-269

Abstract: Instructional Manipulation Checks (IMCs) assess respondent behavior by, for example, asking participants not to answer a trick question. We find IMCs can be used to detect problematic response behavior in longitudinal surveys. This is important because a Latent Class Analysis based on IMC-failure in the two reported studies reveals that between 9% and 12% of respondents can be labelled Inattentive Stayers at the first survey-wave and between 13% and 17% at the third wave. The tendency of Inattentive Stayers to remain in the panel is particularly relevant for online panel services such as MTurk, with workers participating in many surveys over longer time-periods. We find IMC-failure to be mitigated by a warning to respondents that their attention will be checked early in the survey, but not by repeated exposure to IMCs, or a timer leaving questions on the screen longer. Respondent personality also plays a role in IMC-failure, which implies persistency of undesirable respondent behavior.

Keywords: Instructional manipulation check; Longitudinal survey; Online survey panel; MTurk; Insufficient effort responding (search for similar items in EconPapers)
Date: 2018
References: Add references at CitEc
Citations: View citations in EconPapers (9)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S016781161830003X
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ijrema:v:35:y:2018:i:2:p:258-269

DOI: 10.1016/j.ijresmar.2018.01.003

Access Statistics for this article

International Journal of Research in Marketing is currently edited by Roland Rust

More articles in International Journal of Research in Marketing from Elsevier
Bibliographic data for series maintained by Catherine Liu ().