Sentiment Analysis in the Age of Generative AI

Jan Ole Krugmann () and Jochen Hartmann ()
Additional contact information
Jan Ole Krugmann: Technical University of Munich (TUM), TUM School of Management
Jochen Hartmann: Technical University of Munich (TUM), TUM School of Management

Customer Needs and Solutions, 2024, vol. 11, issue 1, No 7, 19 pages

Abstract: Abstract In the rapidly advancing age of Generative AI, Large Language Models (LLMs) such as ChatGPT stand at the forefront of disrupting marketing practice and research. This paper presents a comprehensive exploration of LLMs’ proficiency in sentiment analysis, a core task in marketing research for understanding consumer emotions, opinions, and perceptions. We benchmark the performance of three state-of-the-art LLMs, i.e., GPT-3.5, GPT-4, and Llama 2, against established, high-performing transfer learning models. Despite their zero-shot nature, our research reveals that LLMs can not only compete with but in some cases also surpass traditional transfer learning methods in terms of sentiment classification accuracy. We investigate the influence of textual data characteristics and analytical procedures on classification accuracy, shedding light on how data origin, text complexity, and prompting techniques impact LLM performance. We find that linguistic features such as the presence of lengthy, content-laden words improve classification performance, while other features such as single-sentence reviews and less structured social media text documents reduce performance. Further, we explore the explainability of sentiment classifications generated by LLMs. The findings indicate that LLMs, especially Llama 2, offer remarkable classification explanations, highlighting their advanced human-like reasoning capabilities. Collectively, this paper enriches the current understanding of sentiment analysis, providing valuable insights and guidance for the selection of suitable methods by marketing researchers and practitioners in the age of Generative AI.

Keywords: Generative AI; Large language models; Sentiment analysis; Machine learning; Digital marketing (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40547-024-00143-4 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:custns:v:11:y:2024:i:1:d:10.1007_s40547-024-00143-4

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/40547

DOI: 10.1007/s40547-024-00143-4

Access Statistics for this article

Customer Needs and Solutions is currently edited by Min Ding

More articles in Customer Needs and Solutions from Springer, Institute for Sustainable Innovation and Growth (iSIG)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().