Quantized Graph Neural Networks for Image Classification

Xinbiao Xu, Liyan Ma (), Tieyong Zeng and Qinghua Huang
Additional contact information
Xinbiao Xu: School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
Liyan Ma: School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
Tieyong Zeng: Department of Mathematics, The Chinese University of Hong Kong, Hong Kong 999077, China
Qinghua Huang: School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China

Mathematics, 2023, vol. 11, issue 24, 1-16

Abstract: Researchers have resorted to model quantization to compress and accelerate graph neural networks (GNNs). Nevertheless, several challenges remain: (1) quantization functions overlook outliers in the distribution, leading to increased quantization errors; (2) the reliance on full-precision teacher models results in higher computational and memory overhead. To address these issues, this study introduces a novel framework called quantized graph neural networks for image classification (QGNN-IC), which incorporates a novel quantization function, Pauta quantization (PQ), and two innovative self-distillation methods, attention quantization distillation (AQD) and stochastic quantization distillation (SQD). Specifically, PQ utilizes the statistical characteristics of distribution to effectively eliminate outliers, thereby promoting fine-grained quantization and reducing quantization errors. AQD enhances the semantic information extraction capability by learning from beneficial channels via attention. SQD enhances the quantization robustness through stochastic quantization. AQD and SQD significantly improve the performance of the quantized model with minimal overhead. Extensive experiments show that QGNN-IC not only surpasses existing state-of-the-art quantization methods but also demonstrates robust generalizability.

Keywords: graph neural network; model quantization; knowledge distillation; image classification (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/24/4927/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/24/4927/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:24:p:4927-:d:1298079

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().