Machine learning framework for country image analysis

Luis N. Zúñiga-Morales (), Jorge Ángel González-Ordiano (), J. Emilio Quiroz-Ibarra () and César Villanueva Rivas ()
Additional contact information
Luis N. Zúñiga-Morales: Universidad Iberoamericana, Ciudad de México
Jorge Ángel González-Ordiano: Universidad Iberoamericana, Ciudad de México
J. Emilio Quiroz-Ibarra: Universidad Iberoamericana, Ciudad de México
César Villanueva Rivas: Universidad Iberoamericana, Ciudad de México

Journal of Computational Social Science, 2024, vol. 7, issue 1, No 20, 523-547

Abstract: Abstract In this work, we compare the performance of a machine learning framework based on a support vector machine (SVM) with fastText embeddings, and a Deep Learning framework consisting on fine-tuning Large Language Models (LLMs) like Bidirectional Encoder Representations from Transformers (BERT), DistilBERT, and Twitter roBERTa Base, to automate the classification of text data to analyze the country image of Mexico in selected data sources, which is described using 18 different classes, based in International Relations theory. To train each model, a data set consisting of tweets from relevant selected Twitter accounts and news headlines from The New York Times is used, based on an initial manual classification of all the entries. However, the data set presents issues in the form of imbalanced classes and few data. Thus, a series of text augmentation techniques are explored: gradual augmentation of the eight less represented classes and an uniform augmentation of the data set. Also, we study the impact of hashtags, user names, stopwords, and emojis as additional text features for the SVM model. The results of the experiments indicate that the SVM reacts negatively to all the data augmentation proposals, while the Deep Learning one shows small benefits from them. The best result of 52.92%, in weighted-average $$F_1$$ F 1 score, is obtained by fine-tuning the Twitter roBERTa Base model without data augmentation.

Keywords: Country image analysis; Imbalanced learning; Support vector machine; Large language models (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s42001-023-00246-3 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:jcsosc:v:7:y:2024:i:1:d:10.1007_s42001-023-00246-3

Ordering information: This journal article can be ordered from
http://www.springer. ... iences/journal/42001

DOI: 10.1007/s42001-023-00246-3

Access Statistics for this article

Journal of Computational Social Science is currently edited by Takashi Kamihigashi

More articles in Journal of Computational Social Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().