Interpreting Deep Graph Convolutional Networks with Spectrum Perspective

Sisi Zhang (), Fan Li, Tiancheng Zhang and Ge Yu
Additional contact information
Sisi Zhang: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
Fan Li: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
Tiancheng Zhang: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
Ge Yu: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China

Mathematics, 2023, vol. 11, issue 10, 1-17

Abstract: Graph convolutional network (GCN) architecture is the basis of many neural networks and has been widely used in processing graph-structured data. When dealing with large and sparse data, deeper GCN models are often required. However, the models suffer from performance degradation as the number of layers increases. The mainstream attribution of the current research is over-smoothing, and there are also gradient vanishing, training difficulties, etc., so a consensus cannot be reached. In this paper, we theoretically analyze the degradation problem by adopting spectral graph theory to globally consider the propagation and transformation components of the GCN architecture, and conclude that the over-smoothing problem caused by the propagation matrices is not the key factor for performance degradation. Afterwards, in addition to using conventional experimental methods, we proposed an experimental analysis strategy under the guidance of random matrix theory to analyze the singular value distribution of the model weight matrix. We concluded that the key factor leading to the degradation of model performance is the transformation component. In the context of a lack of consensus on the problem of model performance degradation, the paper proposes a systematic analysis strategy, as well as theoretical and empirical evidence.

Keywords: mining graphs; graph learning; deep graph convolutional networks; semi-supervised learning (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/10/2256/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/10/2256/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:10:p:2256-:d:1144728

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().