More than a Feeling: Accuracy and Application of Sentiment Analysis

Jochen Hartmann, Mark Heitmann, Christian Siebert and Christina Schamp

International Journal of Research in Marketing, 2023, vol. 40, issue 1, 75-87

Abstract: Sentiment is fundamental to human communication. Countless marketing applications mine opinions from social media communication, news articles, customer feedback, or corporate communication. Various sentiment analysis methods are available and new ones have recently been proposed. Lexicons can relate individual words and expressions to sentiment scores. In contrast, machine learning methods are more complex to interpret, but promise higher accuracy, i.e., fewer false classifications. We propose an empirical framework and quantify these trade-offs for different types of research questions, data characteristics, and analytical resources to enable informed method decisions contingent on the application context. Based on a meta-analysis of 272 datasets and 12 million sentiment-labeled text documents, we find that the recently proposed transfer learning models indeed perform best, but can perform worse than popular leaderboard benchmarks suggest. We quantify the accuracy-interpretability trade-off, showing that, compared to widely established lexicons, transfer learning models on average classify more than 20 percentage points more documents correctly. To form realistic performance expectations, additional context variables, most importantly the desired number of sentiment classes and the text length, should be taken into account. We provide a pre-trained sentiment analysis model (called SiEBERT) with open-source scripts that can be applied as easily as an off-the-shelf lexicon.

Keywords: Sentiment Analysis; Meta-Analysis; Natural Language Processing; Machine Learning; Transfer Learning; Deep Contextual Language Models; Text Mining (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167811622000477
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ijrema:v:40:y:2023:i:1:p:75-87

DOI: 10.1016/j.ijresmar.2022.05.005

Access Statistics for this article

International Journal of Research in Marketing is currently edited by Roland Rust

More articles in International Journal of Research in Marketing from Elsevier
Bibliographic data for series maintained by Catherine Liu ().