Crowd Labor Markets as Platform for Group Decision and Negotiation Research: A Comparison to Laboratory Experiments

Florian Teschner () and Henner Gimpel ()
Additional contact information
Florian Teschner: Karlsruhe Institute of Technology (KIT)
Henner Gimpel: University of Augsburg

Group Decision and Negotiation, 2018, vol. 27, issue 2, No 2, 197-214

Abstract: Abstract Crowd labor markets such as Amazon Mechanical Turk (MTurk) have emerged as popular platforms where researchers can relatively inexpensively and easily run web-based experiments. Some work even suggests that MTurk can be used to run large-scale field experiments in which groups of participants interact synchronously in real-time such as electronic markets. Besides technical issues, several methodological questions arise and lead to the question of how results from MTurk and laboratory experiments compare. Our data shows comparable results between MTurk and a standard lab setting with student subjects in a controlled environment when running rather simple individual decision tasks. However, our data shows stark differences in results between the experimental settings for a rather complex market experiment. Each experimental setting—lab and MTurk—has its own benefits and drawbacks; which of the two settings is better suited for a specific experiment depends on the theory or artifact to be tested. We discuss potential causes for differences (language understanding, education, cognition and context) that we cannot control for and provide guidance for the selection of the appropriate setting for an experiment. In any case, researchers studying complex artifacts like group decisions or markets should not prematurely adopt MTurk based on extant literature regarding comparable results across experimental settings for rather simple tasks.

Keywords: Experiments; Mechanical turk; Electronic markets; Information aggregation (search for similar items in EconPapers)
JEL-codes: C9 D8 (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://link.springer.com/10.1007/s10726-018-9565-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:grdene:v:27:y:2018:i:2:d:10.1007_s10726-018-9565-y

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10726/PS2

DOI: 10.1007/s10726-018-9565-y

Access Statistics for this article

Group Decision and Negotiation is currently edited by Gregory E. Kersten

More articles in Group Decision and Negotiation from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().