Spatial Channel Attention for Deep Convolutional Neural Networks

Tonglai Liu, Ronghai Luo, Longqin Xu, Dachun Feng, Liang Cao, Shuangyin Liu and Jianjun Guo
Additional contact information
Tonglai Liu: College of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, China
Ronghai Luo: School of Information and Communication, Guilin University of Electronic Technology, Guilin 541004, China
Longqin Xu: College of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, China
Dachun Feng: College of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, China
Liang Cao: College of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, China
Shuangyin Liu: College of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, China
Jianjun Guo: College of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, China

Mathematics, 2022, vol. 10, issue 10, 1-10

Abstract: Recently, the attention mechanism combining spatial and channel information has been widely used in various deep convolutional neural networks (CNNs), proving its great potential in improving model performance. However, this usually uses 2D global pooling operations to compress spatial information or scaling methods to reduce the computational overhead in channel attention. These methods will result in severe information loss. Therefore, we propose a Spatial channel attention mechanism that captures cross-dimensional interaction, which does not involve dimensionality reduction and brings significant performance improvement with negligible computational overhead. The proposed attention mechanism can be seamlessly integrated into any convolutional neural network since it is a lightweight general module. Our method achieves a performance improvement of 2.08% on ResNet and 1.02% on MobileNetV2 in top-one error rate on the ImageNet dataset.

Keywords: attention mechanism; image classification; deep learning; cross-dimensional interaction (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/10/1750/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/10/1750/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:10:p:1750-:d:820162

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().