Accuracy of a pre-trained sentiment analysis (SA) classification model on tweets related to emergency response and early recovery assessment: the case of 2019 Albanian earthquake

Diana Contreras (), Sean Wilkinson (), Evangeline Alterman () and Javier Hervás ()
Additional contact information
Diana Contreras: Cardiff University
Sean Wilkinson: Newcastle University
Evangeline Alterman: Auckland University

Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, 2022, vol. 113, issue 1, No 18, 403-421

Abstract: Abstract Traditionally, earthquake impact assessments have been made via fieldwork by non-governmental organisations (NGO's) sponsored data collection; however, this approach is time-consuming, expensive and often limited. Recently, social media (SM) has become a valuable tool for quickly collecting large amounts of first-hand data after a disaster and shows great potential for decision-making. Nevertheless, extracting meaningful information from SM is an ongoing area of research. This paper tests the accuracy of the pre-trained sentiment analysis (SA) model developed by the no-code machine learning platform MonkeyLearn using the text data related to the emergency response and early recovery phase of the three major earthquakes that struck Albania on the 26th November 2019. These events caused 51 deaths, 3000 injuries and extensive damage. We obtained 695 tweets with the hashtags: #Albania #AlbanianEarthquake, and #albanianearthquake from the 26th November 2019 to the 3rd February 2020. We used these data to test the accuracy of the pre-trained SA classification model developed by MonkeyLearn to identify polarity in text data. This test explores the feasibility to automate the classification process to extract meaningful information from text data from SM in real-time in the future. We tested the no-code machine learning platform's performance using a confusion matrix. We obtained an overall accuracy (ACC) of 63% and a misclassification rate of 37%. We conclude that the ACC of the unsupervised classification is sufficient for a preliminary assessment, but further research is needed to determine if the accuracy is improved by customising the training model of the machine learning platform.

Keywords: Earthquakes; Social media (SM); Twitter; Sentiment analysis (SA); Machine learning algorithm; Accuracy (ACC) (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s11069-022-05307-w Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:nathaz:v:113:y:2022:i:1:d:10.1007_s11069-022-05307-w

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11069

DOI: 10.1007/s11069-022-05307-w

Access Statistics for this article

Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards is currently edited by Thomas Glade, Tad S. Murty and Vladimír Schenk

More articles in Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards from Springer, International Society for the Prevention and Mitigation of Natural Hazards
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().