Graph Information Vanishing Phenomenon in Implicit Graph Neural Networks

Silu He, Jun Cao, Hongyuan Yuan, Zhe Chen, Shijuan Gao () and Haifeng Li
Additional contact information
Silu He: School of Geosciences and Info-Physics, Central South University, Changsha 410083, China
Jun Cao: School of Geosciences and Info-Physics, Central South University, Changsha 410083, China
Hongyuan Yuan: School of Geosciences and Info-Physics, Central South University, Changsha 410083, China
Zhe Chen: School of Geosciences and Info-Physics, Central South University, Changsha 410083, China
Shijuan Gao: School of Geosciences and Info-Physics, Central South University, Changsha 410083, China
Haifeng Li: School of Geosciences and Info-Physics, Central South University, Changsha 410083, China

Mathematics, 2024, vol. 12, issue 17, 1-19

Abstract: Graph neural networks (GNNs) have been highly successful in graph representation learning. The goal of GNNs is to enrich node representations by aggregating information from neighboring nodes. Much work has attempted to improve the quality of aggregation by introducing a variety of graph information with representational capabilities. The class of GNNs that improves the quality of aggregation by encoding graph information with representational capabilities into the weights of neighboring nodes through different learnable transformation structures (LTSs) are referred to as implicit GNNs. However, we argue that LTSs only transform graph information into the weights of neighboring nodes in the direction that minimizes the loss function during the learning process and does not actually utilize the effective properties of graph information, a phenomenon that we refer to as graph information vanishing (GIV). To validate this point, we perform thousands of experiments on seven node classification benchmark datasets. We first replace the graph information utilized by five implicit GNNs with random values and surprisingly observe that the variation range of accuracies is less than ± 0.3%. Then, we quantitatively characterize the similarity of the weights generated from graph information and random values by cosine similarity, and the cosine similarities are greater than 0.99. The empirical experiments show that graph information is equivalent to initializing the input of LTSs. We believe that graph information as an additional supervised signal to constrain the training of GNNs can effectively solve GIV. Here, we propose GinfoNN, which utilizes both labels and discrete graph curvature as supervised signals to jointly constrain the training of the model. The experimental results show that the classification accuracies of GinfoNN improve by two percentage points over baselines on large and dense datasets.

Keywords: graph neural network; graph information; joint training; graph curvature (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/17/2659/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/17/2659/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:17:p:2659-:d:1465173

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().