The Experimenters' Dilemma: Inferential Preferences over Populations

Neeraja Gupta, Luca Rigotti and Alistair Wilson

Papers from arXiv.org

Abstract: We compare three populations commonly used in experiments by economists and other social scientists: undergraduate students at a physical location (lab), Amazon's Mechanical Turk (MTurk), and Prolific. The comparison is made along three dimensions: the noise in the data due to inattention, the cost per observation, and the elasticity of response. We draw samples from each population, examining decisions in four one-shot games with varying tensions between the individual and socially efficient choices. When there is no tension, where individual and pro-social incentives coincide, noisy behavior accounts for 60% of the observations on MTurk, 19% on Prolific, and 14% for the lab. Taking costs into account, if noisy data is the only concern Prolific dominates from an inferential power point of view, combining relatively low noise with a cost per observation one fifth of the lab's. However, because the lab population is more sensitive to treatment, across our main PD game comparison the lab still outperforms both Prolific and MTurk.

Date: 2021-07, Revised 2021-07
New Economics Papers: this item is included in nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (20)

Downloads: (external link)
http://arxiv.org/pdf/2107.05064 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2107.05064

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().