Beyond Standard Losses: Redefining Text-to-SQL with Task-Specific Optimization

Iker Azurmendi, Ekaitz Zulueta, Gustavo García, Nekane Uriarte-Arrazola and Jose Manuel Lopez-Guede ()
Additional contact information
Iker Azurmendi: Department of Systems and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (EHU), Nieves Cano, 01006 Vitoria-Gasteiz, Spain
Ekaitz Zulueta: Department of Systems and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (EHU), Nieves Cano, 01006 Vitoria-Gasteiz, Spain
Gustavo García: MC3 Mondragon Componentes Competence Center, Avda. Álava 3, 20550 Aretxabaleta, Spain
Nekane Uriarte-Arrazola: Department of Systems and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (EHU), Nieves Cano, 01006 Vitoria-Gasteiz, Spain
Jose Manuel Lopez-Guede: Department of Systems and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (EHU), Nieves Cano, 01006 Vitoria-Gasteiz, Spain

Mathematics, 2025, vol. 13, issue 14, 1-23

Abstract: In recent years, large language models (LLMs) have shown an impressive ability in translating text to SQL queries. However, in real-world applications, standard loss functions frequently fail to capture the complexity of queries adequately. Therefore, in this study, a dynamic loss function is proposed, which assigns different weights to specific groups of tokens, such as SQL keywords or table names. The objective is to guide the model during training to facilitate the mastery of more fundamental concepts within the SQL. Our custom loss function is composed of four components: cross-entropy with sequence matching loss, focal loss, F-beta loss, and contrastive sequence loss. During the training process, the weights of each component of the loss function are dynamically adjusted to prioritize different aspects of query generation at the appropriate stage. This approach avoids computationally expensive approaches such as SQL validation or detokenization, which improves the efficiency of the learning process compared to alternative methods. We empirically tested this method on several open source LLMs with less than 2 billion parameters, using a customized real vehicle diagnostic dataset. The findings demonstrate that the employment of our dynamic loss function can enhance SQL execution accuracy by up to 20% in comparison with standard cross-entropy loss. It has been demonstrated that customized loss functions for specific tasks can improve the efficiency of LLMs without extending the model or acquiring additional labelled data. The proposed technique is also scalable and adaptable to new domains or more complex weighting schemes, highlighting the importance of custom design of loss functions in real world applications.

Keywords: text-to-SQL; natural language processing; large language models; database querying; custom loss; dynamic weighting (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/14/2315/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/14/2315/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:14:p:2315-:d:1705828

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().