A Turing test of whether AI chatbots are behaviorally similar to humans

Qiaozhu Mei (), Yutong Xie, Walter Yuan and Matthew Jackson
Additional contact information
Qiaozhu Mei: a School of Information , University of Michigan , Ann Arbor , MI 48109
Yutong Xie: a School of Information , University of Michigan , Ann Arbor , MI 48109
Walter Yuan: b MobLab , Pasadena , CA 91107

Proceedings of the National Academy of Sciences, 2024, vol. 121, issue 9, e2313925121

Abstract:

We administer a Turing test to AI chatbots. We examine how chatbots behave in a suite of classic behavioral games that are designed to elicit characteristics such as trust, fairness, risk-aversion, cooperation, etc., as well as how they respond to a traditional Big-5 psychological survey that measures personality traits. ChatGPT-4 exhibits behavioral and personality traits that are statistically indistinguishable from a random human from tens of thousands of human subjects from more than 50 countries. Chatbots also modify their behavior based on previous experience and contexts “as if†they were learning from the interactions and change their behavior in response to different framings of the same strategic situation. Their behaviors are often distinct from average and modal human behaviors, in which case they tend to behave on the more altruistic and cooperative end of the distribution. We estimate that they act as if they are maximizing an average of their own and partner’s payoffs.

Keywords: AI; chatbot; behavioral games; Turing test; personality (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://doi.org/10.1073/pnas.2313925121 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nas:journl:v:121:y:2024:p:e2313925121

Access Statistics for this article

More articles in Proceedings of the National Academy of Sciences from Proceedings of the National Academy of Sciences
Bibliographic data for series maintained by Eric Cain ( this e-mail address is bad, please contact ).