Understanding the message passing in graph neural networks via power iteration clustering

DOI10.1016/J.NEUNET.2021.02.025zbMATH Open1521.68138arXiv2006.00144MaRDI QIDQ6078751FDOQ6078751

Authors: Xue Li, Yuanzhi Cheng 

Publication date: 28 September 2023

Published in: Neural Networks (Search for Journal in Brave)

Abstract: The mechanism of message passing in graph neural networks (GNNs) is still mysterious. Apart from convolutional neural networks, no theoretical origin for GNNs has been proposed. To our surprise, message passing can be best understood in terms of power iteration. By fully or partly removing activation functions and layer weights of GNNs, we propose subspace power iteration clustering (SPIC) models that iteratively learn with only one aggregator. Experiments show that our models extend GNNs and enhance their capability to process random featured networks. Moreover, we demonstrate the redundancy of some state-of-the-art GNNs in design and define a lower limit for model evaluation by a random aggregator of message passing. Our findings push the boundaries of the theoretical understanding of neural networks.

Full work available at URL: https://arxiv.org/abs/2006.00144

Recommendations

• Mean-field theory of graph neural networks in graph partitioning
• \(k\)-hop graph neural networks
• Fast Haar transforms for graph neural networks
• Pseudoinverse graph convolutional networks. Fast filters tailored for large eigengaps of dense graphs and hypergraphs
• Variational models for signal processing with graph neural networks

zbMATH Keywords

power iterationmessage passinggraph neural networkssubspace power iteration clustering

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20)

Cites Work

• Drawing graphs by eigenvectors: theory and practice
• Laplacians and the Cheeger inequality for directed graphs

Cited In (13)

• Variational models for signal processing with graph neural networks
• Polarized message-passing in graph neural networks
• Lower and upper bounds for numbers of linear regions of graph convolutional networks
• Mean-field theory of graph neural networks in graph partitioning
• Bosonic random walk neural networks for graph learning
• Interpreting the basis path set in neural networks
• Revisiting graph neural networks from hybrid regularized graph signal reconstruction
• Geometric deep learning: a temperature based analysis of graph neural networks
• Theory of graph neural networks: representation and learning
• A charge-preserving method for solving graph neural diffusion networks
• \(k\)-hop graph neural networks
• Learning stability on graphs
• Graph routing between capsules

This page was built for publication: Understanding the message passing in graph neural networks via power iteration clustering

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6078751)