Crowd Labor Markets as Platform for Group Decision and Negotiation Research: A Comparison to Laboratory Experiments
Florian Teschner () and
Henner Gimpel ()
Additional contact information
Florian Teschner: Karlsruhe Institute of Technology (KIT)
Henner Gimpel: University of Augsburg
Group Decision and Negotiation, 2018, vol. 27, issue 2, No 2, 197-214
Abstract Crowd labor markets such as Amazon Mechanical Turk (MTurk) have emerged as popular platforms where researchers can relatively inexpensively and easily run web-based experiments. Some work even suggests that MTurk can be used to run large-scale field experiments in which groups of participants interact synchronously in real-time such as electronic markets. Besides technical issues, several methodological questions arise and lead to the question of how results from MTurk and laboratory experiments compare. Our data shows comparable results between MTurk and a standard lab setting with student subjects in a controlled environment when running rather simple individual decision tasks. However, our data shows stark differences in results between the experimental settings for a rather complex market experiment. Each experimental setting—lab and MTurk—has its own benefits and drawbacks; which of the two settings is better suited for a specific experiment depends on the theory or artifact to be tested. We discuss potential causes for differences (language understanding, education, cognition and context) that we cannot control for and provide guidance for the selection of the appropriate setting for an experiment. In any case, researchers studying complex artifacts like group decisions or markets should not prematurely adopt MTurk based on extant literature regarding comparable results across experimental settings for rather simple tasks.
Keywords: Experiments; Mechanical turk; Electronic markets; Information aggregation (search for similar items in EconPapers)
JEL-codes: C9 D8 (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed
Downloads: (external link)
http://link.springer.com/10.1007/s10726-018-9565-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:spr:grdene:v:27:y:2018:i:2:d:10.1007_s10726-018-9565-y
Ordering information: This journal article can be ordered from
Access Statistics for this article
Group Decision and Negotiation is currently edited by Gregory E. Kersten
More articles in Group Decision and Negotiation from Springer
Bibliographic data for series maintained by Sonal Shukla ().