LLM Performance in Low-Resource Languages: Selecting an Optimal Model for Migrant Integration Support in Greek

Alexandros Tassios, Stergios Tegos, Christos Bouas, Konstantinos Manousaridis, Maria Papoutsoglou, Maria Kaltsa, Eleni Dimopoulou, Thanassis Mavropoulos, Stefanos Vrochidis and Georgios Meditskos ()
Additional contact information
Alexandros Tassios: School of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Stergios Tegos: School of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Christos Bouas: School of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Konstantinos Manousaridis: School of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Maria Papoutsoglou: School of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Maria Kaltsa: Department of Theoretical & Applied Linguistics, School of English, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Eleni Dimopoulou: PRAKSIS, 10432 Athens, Greece
Thanassis Mavropoulos: Information Technologies Institute, Center for Research and Technology Hellas, 57001 Thessaloniki, Greece
Stefanos Vrochidis: Information Technologies Institute, Center for Research and Technology Hellas, 57001 Thessaloniki, Greece
Georgios Meditskos: School of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece

Future Internet, 2025, vol. 17, issue 6, 1-24

Abstract: The integration of Large Language Models (LLMs) in chatbot applications gains momentum. However, to successfully deploy such systems, the underlying capabilities of LLMs must be carefully considered, especially when dealing with low-resource languages and specialized fields. This paper presents the results of a comprehensive evaluation of several LLMs conducted in the context of a chatbot agent designed to assist migrants in their integration process. Our aim is to identify the optimal LLM that can effectively process and generate text in Greek and provide accurate information, addressing the specific needs of migrant populations. The design of the evaluation methodology leverages input from experts on social assistance initiatives, social impact and technological solutions, as well as from automated LLM self-evaluations. Given the linguistic challenges specific to the Greek language and the application domain, research findings indicate that Claude 3.7 Sonnet and Gemini 2.0 Flash demonstrate superior performance across all criteria, with Claude 3.7 Sonnet emerging as the leading candidate for the chatbot. Moreover, the results suggest that automated custom evaluations of LLMs can align with human assessments, offering a viable option for preliminary low-cost analysis to assist stakeholders in selecting the optimal LLM based on user and application domain requirements.

Keywords: chatbot; Greek language; large language models; LLM evaluation; migrant integration (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/17/6/235/pdf (application/pdf)
https://www.mdpi.com/1999-5903/17/6/235/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:17:y:2025:i:6:p:235-:d:1664873

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().