Understanding and Extending Subgraph GNNs by Rethinking Their Symmetries (2206.11140v3)

Abstract: Subgraph GNNs are a recent class of expressive Graph Neural Networks (GNNs) which model graphs as collections of subgraphs. So far, the design space of possible Subgraph GNN architectures as well as their basic theoretical properties are still largely unexplored. In this paper, we study the most prominent form of subgraph methods, which employs node-based subgraph selection policies such as ego-networks or node marking and deletion. We address two central questions: (1) What is the upper-bound of the expressive power of these methods? and (2) What is the family of equivariant message passing layers on these sets of subgraphs?. Our first step in answering these questions is a novel symmetry analysis which shows that modelling the symmetries of node-based subgraph collections requires a significantly smaller symmetry group than the one adopted in previous works. This analysis is then used to establish a link between Subgraph GNNs and Invariant Graph Networks (IGNs). We answer the questions above by first bounding the expressive power of subgraph methods by 3-WL, and then proposing a general family of message-passing layers for subgraph methods that generalises all previous node-based Subgraph GNNs. Finally, we design a novel Subgraph GNN dubbed SUN, which theoretically unifies previous architectures while providing better empirical performance on multiple benchmarks.

• The paper introduces a novel symmetry analysis that uses a joint permutation group to accurately model node and subgraph relations.
• It demonstrates that the expressive power of node-based subgraph GNNs is bounded by the 3-WL test, aligning with third-order invariant graph networks.
• The paper presents SUN, a unified architecture combining local and global operations that outperforms benchmarks on molecular datasets.

Understanding and Extending Subgraph GNNs by Rethinking Their Symmetries

This paper focuses on the paper of Subgraph Graph Neural Networks (GNNs), a class of models that have shown promising results in representing graph structures through collections of subgraphs. Subgraph GNNs utilize node-based subgraph selection policies, which are critical in enhancing the expressiveness of GNNs beyond classic Message Passing Neural Networks (MPNNs). The research seeks to address two central questions: determining the expressive power limit of node-based subgraph GNNs and characterizing the family of equivariant message-passing layers within this setup.

Key Insights and Analytical Framework

1. Symmetry Analysis: The paper introduces a symmetry analysis to model the symmetries inherent in node-based subgraph collections. Prior frameworks considered independent symmetries for nodes and subgraphs, often employing a symmetry group larger (direct product) than necessary. This paper argues that, due to the inherent bijection between nodes and subgraphs induced by node-based policies, a single joint permutation group is more efficient and accurate. This insight aligns Subgraph GNNs with Invariant Graph Networks (IGNs), particularly those considering third-order tensors.
2. Expressive Power: The authors demonstrate that the expressive power of Subgraph GNNs using node-based selection policies is bounded by the Weisfeiler-Lehman 3-IGN (3-WL) test. They confirm this by showing that any Subgraph GNN can be implemented as a 3-IGN, leveraging the fact that both share the same symmetry constraints and representational capacity.
3. Layer Framework - ReIGN(#1){2}: The paper extends the 2-IGN formalism with ReIGN(#1){2}, incorporating local and global operations in subgraph-node representations. This expanded framework not only corroborates existing subgraph models but also presents new potential designs encompassing more complex local interactions and aggregated operations between subgraphs and nodes.
4. Subgraph Union Network (SUN): A new architecture, SUN, is proposed as a unifying and generalizing model over existing node-based Subgraph GNNs, incorporating both global and local pooling, specifically differentiating between updates for root and non-root nodes. SUN is designed to outperform existing models in various benchmarks, demonstrating its empirical effectiveness while being computationally tractable.

Empirical and Theoretical Implications

• Empirical Results: SUN exhibits superior performance across synthetic and real-world datasets, like ZINC molecular property prediction and OGB molecular datasets; highlighting not only its expressive power but also its generalization capabilities over training data subsets. The architecture effectively balances rich expressiveness with feasible implementation complexity, overcoming challenges typical of subgraph methods.
• Theoretical Perspectives: This work provides a robust framework for understanding the representational capabilities of Subgraph GNNs. It establishes practical boundaries within theoretical concepts, advocating for further exploration into both node-based policy extensions and alternative layer designs that might challenge current expressive thresholds.

Forward-Looking Ideas

The insights brought forth by this paper can guide future research into higher-order node-based policies, potentially broadening the depth and breadth of expressiveness in GNNs. Moreover, it raises intriguing questions about the interplay between learned subgraph selections and predefined ones, opening pathways for adaptive policies suited to diverse graph-related tasks.

This exploration sets a foundation for leveraging symmetry considerations and extending current architectures, driving progress in areas where graph representations are pivotal, such as computational chemistry, social network analysis, and more generalized graph-based machine learning applications.