Incentives Can Reduce Bias in Online Reviews

Ioana Marinescu, Nadav Klein (), Andrew Chamberlain and Morgan Smart
Additional contact information
Nadav Klein: University of Chicago
Morgan Smart: Glassdoor, Inc.

No 11367, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: Online reviews are a powerful means of propagating the reputations of products, services, and even employers. However, existing research suggests that online reviews often suffer from selection bias – people with extreme opinions are more motivated to share them than people with moderate opinions, resulting in biased distributions of reviews. Providing incentives for reviewing has the potential to reduce this selection bias, because incentives can mitigate the motivational deficit of people who hold moderate opinions. Using data from one of the leading employer review companies, Glassdoor, we show that voluntary reviews have a different distribution from incentivized reviews. The likely bias in the distribution of voluntary reviews can affect workers' choice of employers, because it changes the ranking of industries by average employee satisfaction. Because observational data from Glassdoor are not able to provide a measure of the true distribution of employer reviews, we complement our investigation with a randomized controlled experiment on MTurk. We find that when participants' decision to review their employer is voluntary, the resulting distribution of reviews differs from the distribution of forced reviews. Moreover, providing relatively high monetary rewards or a pro-social cue as incentives for reviewing reduces this bias. We conclude that while voluntary employer reviews often suffer from selection bias, incentives can significantly reduce bias and help workers make more informed employer choices.

Keywords: bias; employer reviews; incentives (search for similar items in EconPapers)
JEL-codes: J2 J28 L14 L86 (search for similar items in EconPapers)
Pages: 46 pages
Date: 2018-02
New Economics Papers: this item is included in nep-exp, nep-hrm, nep-ict and nep-lma
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (8)

Published - published in: Journal of Experimental Psychology: Applied, 2021, 27 (2), 393–407.

Downloads: (external link)
https://docs.iza.org/dp11367.pdf (application/pdf)

Related works:
Working Paper: Incentives Can Reduce Bias in Online Reviews (2018)
Working Paper: Incentives Can Reduce Bias in Online Reviews (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp11367

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().