Optimization Based Layer-Wise Pruning Threshold Method for Accelerating Convolutional Neural Networks

Yunlong Ding () and Di-Rong Chen
Additional contact information
Yunlong Ding: School of Mathematical Science, Beihang University, Beijing 100191, China
Di-Rong Chen: School of Mathematical Science, Beihang University, Beijing 100191, China

Mathematics, 2023, vol. 11, issue 15, 1-13

Abstract: Among various network compression methods, network pruning has developed rapidly due to its superior compression performance. However, the trivial pruning threshold limits the compression performance of pruning. Most conventional pruning threshold methods are based on well-known hard or soft techniques that rely on time-consuming handcrafted tests or domain experience. To mitigate these issues, we propose a simple yet effective general pruning threshold method from an optimization point of view. Specifically, the pruning threshold problem is formulated as a constrained optimization program that minimizes the size of each layer. More importantly, our pruning threshold method together with conventional pruning works achieves a better performance across various pruning scenarios on many advanced benchmarks. Notably, for the L 1 -norm pruning algorithm with VGG-16, our method achieves higher FLOPs reductions without utilizing time-consuming sensibility analysis. The compression ratio boosts from 34% to 53%, which is a huge improvement. Similar experiments with ResNet-56 reveal that, even for compact networks, our method achieves competitive compression performance even without skipping any sensitive layers.

Keywords: model compression; neural network; pruning; pruning metric; pruning threshold (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/15/3311/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/15/3311/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:15:p:3311-:d:1204437

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().