A Hybrid Sparrow Search Algorithm of the Hyperparameter Optimization in Deep Learning

Yanyan Fan, Yu Zhang, Baosu Guo, Xiaoyuan Luo, Qingjin Peng and Zhenlin Jin ()
Additional contact information
Yanyan Fan: School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China
Yu Zhang: School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China
Baosu Guo: School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China
Xiaoyuan Luo: School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China
Qingjin Peng: Department of Mechanical Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada
Zhenlin Jin: School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China

Mathematics, 2022, vol. 10, issue 16, 1-23

Abstract: Deep learning has been widely used in different fields such as computer vision and speech processing. The performance of deep learning algorithms is greatly affected by their hyperparameters. For complex machine learning models such as deep neural networks, it is difficult to determine their hyperparameters. In addition, existing hyperparameter optimization algorithms easily converge to a local optimal solution. This paper proposes a method for hyperparameter optimization that combines the Sparrow Search Algorithm and Particle Swarm Optimization, called the Hybrid Sparrow Search Algorithm. This method takes advantages of avoiding the local optimal solution in the Sparrow Search Algorithm and the search efficiency of Particle Swarm Optimization to achieve global optimization. Experiments verified the proposed algorithm in simple and complex networks. The results show that the Hybrid Sparrow Search Algorithm has the strong global search capability to avoid local optimal solutions and satisfactory search efficiency in both low and high-dimensional spaces. The proposed method provides a new solution for hyperparameter optimization problems in deep learning models.

Keywords: deep learning; hyperparameter optimization; Hybrid Sparrow Search; global optimization (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/16/3019/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/16/3019/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:16:p:3019-:d:894277

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().