A Survey of Loss Functions in Deep Learning

Caiyi Li, Kaishuai Liu and Shuai Liu ()
Additional contact information
Caiyi Li: School of Educational Science, Hunan Normal University, Changsha 410081, China
Kaishuai Liu: School of Educational Science, Hunan Normal University, Changsha 410081, China
Shuai Liu: School of Educational Science, Hunan Normal University, Changsha 410081, China

Mathematics, 2025, vol. 13, issue 15, 1-50

Abstract: Deep learning (DL), as a cutting-edge technology in artificial intelligence, has significantly impacted fields such as computer vision and natural language processing. Loss function determines the convergence speed and accuracy of the DL model and has a crucial impact on algorithm quality and model performance. However, most of the existing studies focus on the improvement of specific problems of loss function, which lack a systematic summary and comparison, especially in computer vision and natural language processing tasks. Therefore, this paper reclassifies and summarizes the loss functions in DL and proposes a new category of metric loss. Furthermore, this paper conducts a fine-grained division of regression loss, classification loss, and metric loss, elaborating on the existing problems and improvements. Finally, the new trend of compound loss and generative loss is anticipated. The proposed paper provides a new perspective for loss function division and a systematic reference for researchers in the DL field.

Keywords: deep learning; regression loss; classification loss; metric loss (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/15/2417/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/15/2417/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:15:p:2417-:d:1711153

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().