A Review of Neural Network Lightweighting Techniques

Ziyi Gong (), Huifu Zhang, Hao Yang, Fangjun Liu and Fan Luo
Additional contact information
Ziyi Gong: Department of Computer Sciences and Engineering, Hunan University of Science and Technology, Xiangtan, Hunan, China
Huifu Zhang: Department of Computer Sciences and Engineering, Hunan University of Science and Technology, Xiangtan, Hunan, China
Hao Yang: Department of Computer Sciences and Engineering, Hunan University of Science and Technology, Xiangtan, Hunan, China
Fangjun Liu: Department of Computer Sciences and Engineering, Hunan University of Science and Technology, Xiangtan, Hunan, China
Fan Luo: Department of Computer Sciences and Engineering, Hunan University of Science and Technology, Xiangtan, Hunan, China

Innovation & Technology Advances, 2023, vol. 1, issue 2, 1-24

Abstract: The application of portable devices based on deep learning has become increasingly widespread, which has made the deployment of complex neural networks on embedded devices a hot research topic. Neural network lightweighting is one of the key technologies for applying neural networks to embedded devices. This paper elaborates and analyzes neural network lightweighting techniques from two aspects: model pruning and network structure design. For model pruning, a comparison of methods from different periods is conducted, highlighting their advantages and limitations. Regarding network structure design, the principles of four classical lightweight network designs are described from a mathematical perspective, and the latest optimization methods for these networks are reviewed. Finally, potential research directions for lightweight neural network pruning and structure design optimization are discussed.

Keywords: Lightweighting Techniques for Neural Networks; Model Pruning; Network Structure Design; Convolutional Structure Optimization (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://bergersci.com/index.php/jta/article/view/36/23 (application/pdf)
https://bergersci.com/index.php/jta/article/view/36 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cwi:itadva:v:1:y:2023:i:2:p:1-24

DOI: 10.61187/ita.v1i2.36

Access Statistics for this article

More articles in Innovation & Technology Advances from Berger Science Press
Bibliographic data for series maintained by Berger Science Press ().