Who are you? Inconsistent identity reporting in online labor markets

Stephen Kuselias

Accounting Research Journal, 2020, vol. 33, issue 3, 457-468

Abstract: Purpose - Online labor pools continue to grow in popularity, serving as an inexpensive, readily available source of research data. Despite early skepticism, accounting research has generally found evidence that supports the use of these labor pools. However, one important distinction unique to online labor markets is the pre-screening process that qualifies participants for future studies. As the identity of online participants are generally unknown, researchers rely on participants’ self-reported identities to establish a pool of qualified respondents. This paper aims to provide evidence of the reliability of online participants’ answers to pre-screening questions. Design/methodology/approach - Following the current literature’s recommendations on pre-screening candidates, I employ a multi-stage design using two similar surveys that are taken by each participant. I compare participants' answers on the first survey and the second survey to provide evidence on the consistency of their responses. Findings - My results indicate that online participants are responding with substantial inconsistency to survey questions related to their social identity at a rate that may not be tolerable for many researchers. This has implications for researchers who use these online labor markets to represent a particular population of interest. Originality/value - This study is the first to provide evidence on the consistency of online labor market participant responses. Additionally, it is the first to test the efficacy of current recommended guidelines for identifying populations of interest in the literature.

Keywords: Online labor markets; Screening questions; Mechanical turk (search for similar items in EconPapers)
Date: 2020
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.emerald.com/insight/content/doi/10.110 ... d&utm_campaign=repec (text/html)
https://www.emerald.com/insight/content/doi/10.110 ... d&utm_campaign=repec (application/pdf)
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eme:arjpps:arj-05-2019-0090

DOI: 10.1108/ARJ-05-2019-0090

Access Statistics for this article

Accounting Research Journal is currently edited by Professor Reza Monem

More articles in Accounting Research Journal from Emerald Group Publishing Limited
Bibliographic data for series maintained by Emerald Support ().