Layer-Wise Compressive Training for Convolutional Neural Networks

Matteo Grimaldi, Valerio Tenace and Andrea Calimera
Additional contact information
Matteo Grimaldi: Department of Control and Computer Engineering, Politecnico di Torino, Turin 10129, Italy
Valerio Tenace: Department of Control and Computer Engineering, Politecnico di Torino, Turin 10129, Italy
Andrea Calimera: Department of Control and Computer Engineering, Politecnico di Torino, Turin 10129, Italy

Future Internet, 2018, vol. 11, issue 1, 1-15

Abstract: Convolutional Neural Networks (CNNs) are brain-inspired computational models designed to recognize patterns. Recent advances demonstrate that CNNs are able to achieve, and often exceed, human capabilities in many application domains. Made of several millions of parameters, even the simplest CNN shows large model size. This characteristic is a serious concern for the deployment on resource-constrained embedded-systems, where compression stages are needed to meet the stringent hardware constraints. In this paper, we introduce a novel accuracy-driven compressive training algorithm. It consists of a two-stage flow: first, layers are sorted by means of heuristic rules according to their significance; second, a modified stochastic gradient descent optimization is applied on less significant layers such that their representation is collapsed into a constrained subspace. Experimental results demonstrate that our approach achieves remarkable compression rates with low accuracy loss (<1%).

Keywords: deep learning; machine learning; neural networks on-chip; optimization (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2018
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/1999-5903/11/1/7/pdf (application/pdf)
https://www.mdpi.com/1999-5903/11/1/7/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:11:y:2018:i:1:p:7-:d:193662

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().