A Performance Comparison of Different Cloud-Based Natural Language Understanding Services for an Italian e-Learning Platform

Matteo Zubani, Luca Sigalini, Ivan Serina, Luca Putelli, Alfonso E. Gerevini and Mattia Chiari
Additional contact information
Matteo Zubani: Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy
Luca Sigalini: Mega Italia Media, Via Roncadelle 70A, 25030 Castel Mella, Italy
Ivan Serina: Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy
Luca Putelli: Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy
Alfonso E. Gerevini: Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy
Mattia Chiari: Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy

Future Internet, 2022, vol. 14, issue 2, 1-20

Abstract: During the COVID-19 pandemic, the corporate online training sector has increased exponentially and online course providers had to implement innovative solutions to be more efficient and provide a satisfactory service. This paper considers a real case study in implementing a chatbot, which answers frequently asked questions from learners on an Italian e-learning platform that provides workplace safety courses to several business customers. Having to respond quickly to the increase in the courses activated, the company decided to develop a chatbot using a cloud-based service currently available on the market. These services are based on Natural Language Understanding (NLU) engines, which deal with identifying information such as entities and intentions from the sentences provided as input. To integrate a chatbot in an e-learning platform, we studied the performance of the intent recognition task of the major NLU platforms available on the market with an in-depth comparison, using an Italian dataset provided by the owner of the e-learning platform. We focused on intent recognition, carried out several experiments and evaluated performance in terms of F-score, error rate, response time, and robustness of all the services selected. The chatbot is currently in production, therefore we present a description of the system implemented and its results on the original users’ requests.

Keywords: chatbots; natural language understanding; cloud-based services; machine learning (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/14/2/62/pdf (application/pdf)
https://www.mdpi.com/1999-5903/14/2/62/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:14:y:2022:i:2:p:62-:d:752435

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().