Reputation Transferability in Online Labor Markets

Marios Kokkodis () and Panagiotis G. Ipeirotis ()
Additional contact information
Marios Kokkodis: Department of Information, Operations, and Management Sciences, Leonard N. Stern School of Business, New York University, New York, New York 10012
Panagiotis G. Ipeirotis: Department of Information, Operations, and Management Sciences, Leonard N. Stern School of Business, New York University, New York, New York 10012

Management Science, 2016, vol. 62, issue 6, 1687-1706

Abstract: Online workplaces such as oDesk, Amazon Mechanical Turk, and TaskRabbit have been growing in importance over the last few years. In such markets, employers post tasks on which remote contractors work and deliver the product of their work online. As in most online marketplaces, reputation mechanisms play a very important role in facilitating transactions, since they instill trust and are often predictive of the employer’s future satisfaction. However, labor markets are usually highly heterogeneous in terms of available task categories; in such scenarios, past performance may not be an accurate signal of future performance. To account for this natural heterogeneity, in this work, we build models that predict the performance of a worker based on prior, category-specific feedback. Our models assume that each worker has a category-specific quality, which is latent and not directly observable; what is observable, though, is the set of feedback ratings of the worker and of other contractors with similar work histories. Based on this information, we provide a series of models of increasing complexity that successfully estimate the worker’s quality. We start by building a binomial model and a multinomial model under the implicit assumption that the latent qualities of the workers are static. Next, we remove this assumption, and we build linear dynamic systems that capture the evolution of these latent qualities over time. We evaluate our models on a large corpus of over a million transactions (completed tasks) from oDesk, an online labor market with hundreds of millions of dollars in transaction volume. Our results show an improved accuracy of up to 25% compared to feedback baselines and significant improvement over the commonly used collaborative filtering approach. Our study clearly illustrates that reputation systems should present different reputation scores, depending on the context in which the worker has been previously evaluated and the job for which the worker is applying. This paper was accepted by Lorin Hitt, information systems.

Keywords: online labor markets; reputation; Bayesian modeling; linear dynamical systems (search for similar items in EconPapers)
Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (38)

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.2015.2217 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:62:y:2016:i:6:p:1687-1706

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().