Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Improving Expressivity of Graph Neural Networks using Localization (2305.19659v3)

Published 31 May 2023 in cs.LG and cs.DS

Abstract: In this paper, we propose localized versions of Weisfeiler-Leman (WL) algorithms in an effort to both increase the expressivity, as well as decrease the computational overhead. We focus on the specific problem of subgraph counting and give localized versions of $k-$WL for any $k$. We analyze the power of Local $k-$WL and prove that it is more expressive than $k-$WL and at most as expressive as $(k+1)-$WL. We give a characterization of patterns whose count as a subgraph and induced subgraph are invariant if two graphs are Local $k-$WL equivalent. We also introduce two variants of $k-$WL: Layer $k-$WL and recursive $k-$WL. These methods are more time and space efficient than applying $k-$WL on the whole graph. We also propose a fragmentation technique that guarantees the exact count of all induced subgraphs of size at most 4 using just $1-$WL. The same idea can be extended further for larger patterns using $k>1$. We also compare the expressive power of Local $k-$WL with other GNN hierarchies and show that given a bound on the time-complexity, our methods are more expressive than the ones mentioned in Papp and Wattenhofer[2022a].

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Finding and counting given length cycles. Algorithmica, 17(3):209–223, Mar 1997. ISSN 1432-0541. doi: 10.1007/BF02523189. URL https://doi.org/10.1007/BF02523189.
  2. Beyond 1-wl with local ego-network encodings, 2022. URL https://arxiv.org/abs/2211.14906.
  3. Graph isomorphism, color refinement, and compactness. computational complexity, 26(3):627–685, 2017.
  4. On weisfeiler-leman invariance: Subgraph counts and related graph properties. Journal of Computer and System Sciences, 113:42–59, 2020.
  5. László Babai. Graph isomorphism in quasipolynomial time. In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, pages 684–697, 2016.
  6. Random graph isomorphism. SIaM Journal on computing, 9(3):628–635, 1980.
  7. Graph neural networks with local graph parameters, 2021. URL https://arxiv.org/abs/2106.06707.
  8. Equivariant subgraph aggregation networks, 2021. URL https://arxiv.org/abs/2110.02910.
  9. Béla Bollobás. Distinguishing vertices of random graphs. In North-Holland Mathematics Studies, volume 62, pages 33–49. Elsevier, 1982.
  10. Improving graph neural network expressivity via subgraph isomorphism counting, 2020. URL https://arxiv.org/abs/2006.09252.
  11. Marco Bressan. Faster algorithms for counting subgraphs in sparse graphs, 2018. URL https://arxiv.org/abs/1805.02089.
  12. An optimal lower bound on the number of variables for graph identification. Combinatorica, 12(4):389–410, 1992.
  13. Can graph neural networks count substructures? Advances in neural information processing systems, 33:10383–10395, 2020.
  14. Homomorphisms are a good basis for counting small subgraphs. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pages 210–223, 2017.
  15. Lovász meets weisfeiler and leman. In ICALP, 2018.
  16. The protein folding problem. Annual Review of Biophysics, 37(1):289–316, 2008. doi: 10.1146/annurev.biophys.37.092707.153558. URL https://doi.org/10.1146/annurev.biophys.37.092707.153558. PMID: 18573083.
  17. Understanding and extending subgraph gnns by rethinking their symmetries, 2022. URL https://arxiv.org/abs/2206.11140.
  18. Martin Fürer. On the combinatorial power of the weisfeiler-lehman algorithm. In Algorithms and Complexity: 10th International Conference, CIAC 2017, Athens, Greece, May 24-26, 2017, Proceedings, pages 260–271. Springer, 2017.
  19. A linear upper bound on the weisfeiler-leman dimension of graphs of bounded genus. arXiv preprint arXiv:1904.07216, 2019.
  20. Isomorphism, canonization, and definability for graphs of bounded rank width. 2021.
  21. Inductive representation learning on large graphs, 2018.
  22. Boosting the cycle counting power of graph neural networks with i22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT-gnns, 2023.
  23. The k𝑘kitalic_k-dimensional weisfeiler-leman algorithm, 2019.
  24. Sandra Kiefer. Power and limits of the Weisfeiler-Leman algorithm. PhD thesis, Dissertation, RWTH Aachen University, 2020, 2020.
  25. The power of the weisfeiler-leman algorithm to decompose graphs. CoRR, abs/1908.05268, 2019.
  26. Graphs identified by logics with counting. In International Symposium on Mathematical Foundations of Computer Science, pages 319–330. Springer, 2015.
  27. The weisfeiler–leman dimension of planar graphs is at most 3. Journal of the ACM (JACM), 66(6):1–31, 2019.
  28. Semi-supervised classification with graph convolutional networks, 2017.
  29. Finding and counting patterns in sparse graphs. In Petra Berenbrink, Patricia Bouyer, Anuj Dawar, and Mamadou Moustapha Kanté, editors, 40th International Symposium on Theoretical Aspects of Computer Science, STACS 2023, March 7-9, 2023, Hamburg, Germany, volume 254 of LIPIcs, pages 40:1–40:20. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2023. doi: 10.4230/LIPIcs.STACS.2023.40. URL https://doi.org/10.4230/LIPIcs.STACS.2023.40.
  30. Tuukka Korhonen. A single-exponential time 2-approximation algorithm for treewidth. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pages 184–192. IEEE, 2022.
  31. An improved parameterized algorithm for treewidth. arXiv preprint arXiv:2211.07154, 2022.
  32. Neural subgraph isomorphism counting, 2019. URL https://arxiv.org/abs/1912.11589.
  33. Provably powerful graph networks, 2019. URL https://arxiv.org/abs/1905.11136.
  34. Harry L Morgan. The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service. Journal of chemical documentation, 5(2):107–113, 1965.
  35. Weisfeiler and leman go neural: Higher-order graph neural networks. CoRR, abs/1810.02244, 2018. URL http://arxiv.org/abs/1810.02244.
  36. Weisfeiler and leman go neural: Higher-order graph neural networks. AAAI’19/IAAI’19/EAAI’19. AAAI Press, 2019. ISBN 978-1-57735-809-1. doi: 10.1609/aaai.v33i01.33014602. URL https://doi.org/10.1609/aaai.v33i01.33014602.
  37. Weisfeiler and leman go machine learning: The story so far, 2021. URL https://arxiv.org/abs/2112.09992.
  38. Sang-il Oum. Rank-width: Algorithmic and structural results. Discrete Applied Mathematics, 231:15–24, 2017.
  39. A theoretical comparison of graph neural network extensions. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 17323–17345. PMLR, 17–23 Jul 2022a. URL https://proceedings.mlr.press/v162/papp22a.html.
  40. A theoretical comparison of graph neural network extensions, 2022b. URL https://arxiv.org/abs/2201.12884.
  41. Efficient graphlet kernels for large graph comparison. In Artificial intelligence and statistics, pages 488–495. PMLR, 2009.
  42. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(77):2539–2561, 2011. URL http://jmlr.org/papers/v12/shervashidze11a.html.
  43. Graph attention networks, 2018.
  44. The reduction of a graph to canonical form and the algebra which appears therein. NTI, Series, 2(9):12–16, 1968.
  45. Douglas Brent West et al. Introduction to graph theory, volume 2. Prentice hall Upper Saddle River, 2001.
  46. How powerful are graph neural networks? In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=ryGs6iA5Km.
  47. Gnnexplainer: Generating explanations for graph neural networks, 2019. URL https://arxiv.org/abs/1903.03894.
  48. Identity-aware graph neural networks, 2021. URL https://arxiv.org/abs/2101.10320.
  49. Nested graph neural networks. Advances in Neural Information Processing Systems, 34:15734–15747, 2021.
  50. From stars to subgraphs: Uplifting any GNN with local structure awareness. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=Mspk_WYKoEH.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now