Advancing tourism sentiment analysis: a comparative evaluation of traditional machine learning, deep learning, and transformer models on imbalanced datasets

Sawitree Srianan (), Aziz Nanthaamornphong () and Chayanon Phucharoen ()
Additional contact information
Sawitree Srianan: Prince of Songkla University
Aziz Nanthaamornphong: Prince of Songkla University
Chayanon Phucharoen: Prince of Songkla University

Information Technology & Tourism, 2025, vol. 27, issue 4, No 5, 1045 pages

Abstract: Abstract Tourism sentiment analysis faces substantial challenges due to class imbalance and the complex linguistic features of user-generated content. This study systematically compares eight sentiment classification models, spanning traditional machine learning (naïve Bayes, support vector machines, logistic regression), deep learning (convolutional neural networks, long short-term memory networks [LSTMs], gated recurrent units [GRUs]), and transformer-based architectures (RoBERTa in two configurations: pretrained and fine-tuned), using a dataset of 505,980 TripAdvisor reviews. We evaluate model performance under imbalanced class conditions and examine the effectiveness of three oversampling techniques—SMOTE, ADASYN, and RandomOverSampler—in mitigating class bias. The results reveal significant performance disparities across architectures. Deep learning models, particularly LSTM (91.06% accuracy, Cohen’s kappa = 0.6846) and GRU (90.82% accuracy, Cohen’s kappa = 0.6781), consistently outperform traditional approaches. Fine-tuned RoBERTa achieved the highest performance, with 92.31% accuracy, a 95.34% F1-score, and Cohen’s kappa = 0.7321. Traditional models showed notable limitations; for example, naïve Bayes exhibited strong majority-class bias, despite an accuracy of 82.35% (Cohen’s kappa = 0.0054). Among the oversampling methods, SMOTE was the most effective in improving the fairness of traditional models, while RoBERTa’s fine-tuning process inherently mitigated class imbalance. A computational analysis highlights key trade-offs: traditional models train quickly but require oversampling, deep learning offers a balanced trade-off between performance and efficiency, and transformer models provide state-of-the-art accuracy at the cost of high computational resources. These findings offer evidence-based guidance for selecting appropriate models for tourism sentiment analysis.

Keywords: Sentiment analysis; Natural language processing (NLP); Class imbalance; Deep learning; RoBERTa; Online review (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40558-025-00336-0 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:infott:v:27:y:2025:i:4:d:10.1007_s40558-025-00336-0

Ordering information: This journal article can be ordered from
http://www.springer. ... ystems/journal/40558

DOI: 10.1007/s40558-025-00336-0

Access Statistics for this article

Information Technology & Tourism is currently edited by Zheng Xiang

More articles in Information Technology & Tourism from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().