The Turing test of online reviews: Can we tell the difference between human-written and GPT-4-written online reviews?

Balázs Kovács ()
Additional contact information
Balázs Kovács: Yale University

Marketing Letters, 2024, vol. 35, issue 4, No 10, 666 pages

Abstract: Abstract Online reviews serve as a guide for consumer choice. With advancements in large language models (LLMs) and generative AI, the fast and inexpensive creation of human-like text may threaten the feedback function of online reviews if neither readers nor platforms can differentiate between human-written and AI-generated content. In two experiments, we found that humans cannot recognize AI-written reviews. Even with monetary incentives for accuracy, both Type I and Type II errors were common: human reviews were often mistaken for AI-generated reviews, and even more frequently, AI-generated reviews were mistaken for human reviews. This held true across various ratings, emotional tones, review lengths, and participants’ genders, education levels, and AI expertise. Younger participants were somewhat better at distinguishing between human and AI reviews. An additional study revealed that current AI detectors were also fooled by AI-generated reviews. We discuss the implications of our findings on trust erosion, manipulation, regulation, consumer behavior, AI detection, market structure, innovation, and review platforms.

Keywords: Online reviews; Generative AI; GPT-4; Turing test (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11002-024-09729-3 Abstract (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:kap:mktlet:v:35:y:2024:i:4:d:10.1007_s11002-024-09729-3

Ordering information: This journal article can be ordered from
http://www.springer. ... etailsPage=societies

DOI: 10.1007/s11002-024-09729-3

Access Statistics for this article

Marketing Letters is currently edited by Joel Steckel and Peter Golder

More articles in Marketing Letters from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().