The Online Laboratory: Conducting Experiments in a Real Labor Market

John Horton, David G. Rand and Richard Zeckhauser
Additional contact information
David G. Rand: Harvard University

Working Paper Series from Harvard University, John F. Kennedy School of Government

Abstract: Online labor markets have great potential as platforms for conducting experiments, as they provide immediate access to a large and diverse subject pool and allow researchers to conduct randomized controlled trials. We argue that online experiments can be just as valid--both internally and externally--as laboratory and field experiments, while requiring far less money and time to design and to conduct. In this paper, we first describe the benefits of conducting experiments in online labor markets; we then use one such market to replicate three classic experiments and confirm their results. We confirm that subjects (1) reverse decisions in response to how a decision-problem is framed, (2) have pro-social preferences (value payoffs to others positively), and (3) respond to priming by altering their choices. We also conduct a labor supply field experiment in which we confirm that workers have upward sloping labor supply curves. In addition to reporting these results, we discuss the unique threats to validity in an online setting and propose methods for coping with these threats. We also discuss the external validity of results from online domains and explain why online results can have external validity equal to or even better than that of traditional methods, depending on the research question. We conclude with our views on the potential role that online experiments can play within the social sciences, and then recommend software development priorities and best practices.

JEL-codes: C70 C91 C92 C93 J20 (search for similar items in EconPapers)
Date: 2010-04
New Economics Papers: this item is included in nep-cbe, nep-exp, nep-lab and nep-ltv
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (17)

Downloads: (external link)
https://research.hks.harvard.edu/publications/work ... ?PubId=7277&type=WPN

Related works:
Journal Article: The online laboratory: conducting experiments in a real labor market (2011)
Working Paper: The Online Laboratory: Conducting Experiments in a Real Labor Market (2010)
Working Paper: The Online Laboratory: Conducting Experiments in a Real Labor Market (2010)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ecl:harjfk:rwp10-017

Access Statistics for this paper

More papers in Working Paper Series from Harvard University, John F. Kennedy School of Government Contact information at EDIRC.
Bibliographic data for series maintained by ().