Higher-order Graph Convolutional Network with Flower-Petals Laplacians on Simplicial Complexes (2309.12971v2)
Abstract: Despite the recent successes of vanilla Graph Neural Networks (GNNs) on various tasks, their foundation on pairwise networks inherently limits their capacity to discern latent higher-order interactions in complex systems. To bridge this capability gap, we propose a novel approach exploiting the rich mathematical theory of simplicial complexes (SCs) - a robust tool for modeling higher-order interactions. Current SC-based GNNs are burdened by high complexity and rigidity, and quantifying higher-order interaction strengths remains challenging. Innovatively, we present a higher-order Flower-Petals (FP) model, incorporating FP Laplacians into SCs. Further, we introduce a Higher-order Graph Convolutional Network (HiGCN) grounded in FP Laplacians, capable of discerning intrinsic features across varying topological scales. By employing learnable graph filters, a parameter group within each FP Laplacian domain, we can identify diverse patterns where the filters' weights serve as a quantifiable measure of higher-order interaction strengths. The theoretical underpinnings of HiGCN's advanced expressiveness are rigorously demonstrated. Additionally, our empirical investigations reveal that the proposed model accomplishes state-of-the-art performance on a range of graph tasks and provides a scalable and flexible solution to explore higher-order interactions in graphs. Codes and datasets are available at https://github.com/Yiminghh/HiGCN.
- Diffusion-convolutional neural networks. Advances in neural information processing systems, 29.
- The physics of higher-order interactions in complex systems. Nature Physics, 17(10): 1093–1098.
- Networks beyond pairwise interactions: Structure and dynamics. Physics Reports, 874: 1–92.
- Simplicial closure and higher-order link prediction. Proceedings of the National Academy of Sciences.
- Simplex2vec embeddings for community detection in simplicial complexes. arXiv:1906.09068.
- Weisfeiler and lehman go topological: Message passing simplicial networks. In International Conference on Machine Learning (ICML), 1026–1037.
- The maximum clique problem. In Handbook of combinatorial optimization, 1–74. Springer.
- Protein function prediction via graph kernels. Bioinformatics, 21(suppl_1): i47–i56.
- Algorithm 457: Finding All Cliques of an Undirected Graph. Commun. ACM, 16(9): 575–577.
- An optimal lower bound on the number of variables for graph identification. Combinatorica, 12(4): 398–140.
- Centola, D. 2010. The Spread of Behavior in an Online Social Network Experiment. Science, 329(5996): 1194–1197.
- BScNets: Block Simplicial Complex Neural Networks. volume 36, 6333–6341.
- Arboricity and subgraph listing algorithms. SIAM J. Comput., 14(1): 210–223.
- Adaptive Universal Generalized PageRank Graph Neural Network. In International Conference on Learning Representations.
- Natural graph networks. Advances in neural information processing systems, 33: 3636–3646.
- Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, 34(2): 786–797.
- Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 29: 3838–3845.
- Simplicial Neural Networks. In NeurIPS 2020 Workshop on Topological Data Analysis and Beyond.
- Fast graph representation learning with PyTorch Geometric. arXiv:1903.02428.
- Sparse low-order interaction network underlies a highly correlated and learnable neural population code. Proceedings of the National Academy of Sciences, 108(23): 9679–9684.
- Hypergraph Learning: Methods and Practices. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5): 2548–2566.
- On graph kernels: Hardness results and efficient alternatives. In Learning Theory and Kernel Machines, 129–143. Springer.
- Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In International Conference on Learning Representations.
- Higher-order interactions stabilize dynamics in competitive network models. Nature, 548(7666): 210–213.
- Hacker, C. 2020. k-simplex2vec: a simplicial extension of node2vec. arXiv:2010.05636.
- Simplicial complex representation learning. In Machine Learning on Graphs (MLoG) Workshop at 15th ACM International WSDM Conference.
- Hatcher, A. 2002. Algebraic topology. Cambridge University Press.
- Bernnet: Learning arbitrary graph spectral filters via bernstein approximation. Advances in Neural Information Processing Systems, 34: 14239–14251.
- Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33: 22118–22133.
- Kendall, M. G. 1938. A new measure of rank correlation. Biometrika, 30(1/2): 81–93.
- Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
- SGAT: Simplicial Graph Attention Network. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22), 3192–3200.
- Finding global homophily in graph neural networks when meeting heterophily. In International Conference on Machine Learning, 13242–13256. PMLR.
- Expertise and dynamics within crowdsourced musical knowledge curation: A case study of the genius platform. In Proceedings of the International AAAI Conference on Web and Social Media, volume 15, 373–384.
- Revisiting heterophily for graph neural networks. Advances in neural information processing systems, 35: 1362–1375.
- Provably powerful graph networks. Advances in neural information processing systems, 32.
- Invariant and Equivariant Graph Networks. In International Conference on Learning Representations.
- Milgram, S. 1967. The small world problem. Psychology today, 2(1): 60–67.
- Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01): 4602–4609.
- Propagation kernels: efficient graph kernels from propagated information. Machine Learning, 102: 209–245.
- Quantifying randomness in real networks. Nature communications, 6(1): 8627.
- Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations.
- Principled Simplicial Neural Networks for Trajectory Prediction. In Proceedings of the 38th International Conference on Machine Learning, volume 139, 9020–9029. PMLR.
- Multi-scale attributed node embedding. Journal of Complex Networks, 9(2): 1–22.
- Random walks on simplicial complexes and the normalized Hodge 1-Laplacian. SIAM Review, 62(2): 353–391.
- Pitfalls of Graph Neural Network Evaluation. CoRR, abs/1811.05868.
- Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(9).
- Efficient graphlet kernels for large graph comparison. In Artificial intelligence and statistics, 488–495. PMLR.
- The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3): 83–98.
- An Overview of Microsoft Academic Service (MAS) and Applications. In Proceedings of the 24th International Conference on World Wide Web. ACM Press.
- Statistical evaluation of the predictive toxicology challenge 2000–2001. Bioinformatics, 19(10): 1183–1193.
- Social structure of facebook networks. Physica A: Statistical Mechanics and its Applications, 391(16): 4165–4180.
- Graph Attention Networks. In International Conference on Learning Representations (ICLR).
- How Powerful are Spectral Graph Neural Networks. In International Conference on Machine Learning (ICML).
- The reduction of a graph to canonical form and the algebra which appears therein. NTI, Series, 2(9): 12–16.
- Simplifying Graph Convolutional Networks. In Chaudhuri, K.; and Salakhutdinov, R., eds., Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, 6861–6871. PMLR.
- How powerful are graph neural networks? In International Conference on Learning Representations (ICLR).
- Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks. arXiv:2102.06462.
- Deep Graph Kernels. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, 1365–1374. New York, NY, USA: Association for Computing Machinery. ISBN 9781450336642.
- Simplicial convolutional neural networks. In ICASSP, 8847–8851.
- A generalized simplicial model and its application. arXiv:2309.02851.
- Revisiting Semi-Supervised Learning with Graph Embeddings. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, 40–48. New York, New York, USA: PMLR.
- Identifying vital nodes through augmented random walks on higher-order networks. arXiv:2305.06898.
- Influential Simplices Mining via Simplicial Convolutional Network. arXiv:2307.05841.
- Hyper-Null Models and Their Applications. Entropy, 25(10): 1390.
- DRGCN: Dynamic Evolving Initial Residual for Deep Graph Convolutional Networks. arXiv:2302.05083.
- An end-to-end deep learning architecture for graph classification. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
- Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33: 7793–7804.