Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Simplifying Subgraph Representation Learning for Scalable Link Prediction (2301.12562v4)

Published 29 Jan 2023 in cs.LG and cs.SI

Abstract: Link prediction on graphs is a fundamental problem. Subgraph representation learning approaches (SGRLs), by transforming link prediction to graph classification on the subgraphs around the links, have achieved state-of-the-art performance in link prediction. However, SGRLs are computationally expensive, and not scalable to large-scale graphs due to expensive subgraph-level operations. To unlock the scalability of SGRLs, we propose a new class of SGRLs, that we call Scalable Simplified SGRL (S3GRL). Aimed at faster training and inference, S3GRL simplifies the message passing and aggregation operations in each link's subgraph. S3GRL, as a scalability framework, accommodates various subgraph sampling strategies and diffusion operators to emulate computationally-expensive SGRLs. We propose multiple instances of S3GRL and empirically study them on small to large-scale graphs. Our extensive experiments demonstrate that the proposed S3GRL models scale up SGRLs without significant performance compromise (even with considerable gains in some cases), while offering substantially lower computational footprints (e.g., multi-fold inference and training speedup).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. N-gcn: Multi-scale Graph Convolution for Semi-supervised Node Classification. In Uncertainty in Artificial Intelligence.
  2. Lada A Adamic and Eytan Adar. 2003. Friends and Neighbors on the Web. Social Networks 25, 3 (2003), 211–230.
  3. Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations.
  4. Lei Cai and Shuiwang Ji. 2020. A Multi-Scale Approach for Graph Link Prediction. In AAAI Conference on Artificial Intelligence.
  5. Graph Neural Networks for Link Prediction with Subgraph Sketching. In International Conference on Learning Representations.
  6. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In International Conference on Learning Representations.
  7. Friend Recommendation based on Multi-social Graph Convolutional Network. IEEE Access 8 (2020), 43618–43629.
  8. Hyperspherical Variational Auto-Encoders. In Uncertainty in Artificial Intelligence.
  9. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Advances in Neural Information Processing Systems. 9 pages.
  10. Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
  11. SIGN: Scalable Inception Graph Neural Networks. In ICML 2020 Workshop on Graph Representation Learning and Beyond.
  12. Diffusion Improves Graph Learning. In Advances in Neural Information Processing Systems.
  13. Mark Granovetter. 1983. The strength of weak ties: A network theory revisited. Sociological theory (1983), 201–233.
  14. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In International Conference on Knowledge Discovery and Data mining.
  15. Linkless Link Prediction via Relational Distillation. arXiv preprint arXiv:2210.05801 (2022).
  16. William L Hamilton. 2020. Graph representation learning. Synthesis Lectures on Artifical Intelligence and Machine Learning 14, 3 (2020), 1–159.
  17. Inductive Representation Learning on Large Graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS’17). Curran Associates Inc., Red Hook, NY, USA, 1025–1035.
  18. Ogb-lsc: A large-scale challenge for machine learning on graphs. arXiv preprint arXiv:2103.09430 (2021).
  19. Open Graph Benchmark: Datasets for Machine Learning on Graphs. arXiv preprint arXiv:2005.00687 (2020).
  20. SkipGNN: predicting molecular interactions with skip-graph networks. Scientific reports 10, 1 (2020), 1–16.
  21. Boosting the Cycle Counting Power of Graph Neural Networks with I2superscriptI2\text{I}^{2}I start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT-GNNs. arXiv preprint arXiv:2210.13978 (2022).
  22. Stochastic Subgraph Neighborhood Pooling for Subgraph Classification. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (Birmingham, United Kingdom) (CIKM ’23). Association for Computing Machinery, 3963–3967.
  23. Leo Katz. 1953. A New Status Index derived from Sociometric Analysis. Psychometrika 18, 1 (1953), 39–43.
  24. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations.
  25. Thomas N Kipf and Max Welling. 2016. Variational Graph Auto-encoders. arXiv preprint arXiv:1611.07308 (2016).
  26. Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations.
  27. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
  28. Distance encoding: Design provably more powerful neural networks for graph representation learning. Advances in Neural Information Processing Systems 33 (2020), 4465–4478.
  29. David Liben-Nowell and Jon Kleinberg. 2003. The link prediction problem for social networks. In International Conference on Information and Knowledge Management.
  30. Sampling Enclosing Subgraphs for Link Prediction. In International Conference on Information & Knowledge Management.
  31. Costas Mavromatis and G. Karypis. 2021. Graph InfoClust: Maximizing Coarse-Grain Mutual Information in Graphs. In PAKDD.
  32. The PageRank citation ranking: Bringing order to the web. Technical Report. Stanford InfoLab.
  33. Neural Link Prediction with Walk Pooling. In International Conference on Learning Representations.
  34. Adversarially Regularized Graph Autoencoder for Graph Embedding. In International Joint Conference on Artificial Intelligence.
  35. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems. Article 721, 12 pages.
  36. Deepwalk: Online learning of social representations. In International Conference on Knowledge Discovery and Data mining.
  37. Patrick Pho and Alexander V. Mantzaris. 2022. Link Prediction with Simple Graph Convolution and Regularized Simple Graph Convolution. In International Conference on Information System and Data Mining. 5 pages.
  38. Attention is All You Need. In Advances in Neural Information Processing Systems. 11 pages.
  39. Simplifying Graph Convolutional Networks. In International Conference on Machine Learning.
  40. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of medicinal chemistry 63, 16 (2019), 8749–8760.
  41. Representation learning on graphs with jumping knowledge networks. In International Conference on Machine Learning.
  42. Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning. Proceedings of the VLDB Endowment 15, 11 (2022), 2788–2796.
  43. Graph convolutional neural networks for web-scale recommender systems. In International Conference on Knowledge Discovery and Data mining.
  44. Decoupling the Depth and Scope of Graph Neural Networks. In Advances in Neural Information Processing Systems.
  45. GraphSAINT: Graph Sampling Based Inductive Learning Method. In International Conference on Learning Representations.
  46. Muhan Zhang and Yixin Chen. 2017. Weisfeiler-lehman Neural Machine for Link Prediction. In International Conference on Knowledge Discovery and Data Mining.
  47. Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. In Advances in Neural Information Processing Systems.
  48. An end-to-end deep learning architecture for graph classification. In AAAI Conference on Artificial Intelligence.
  49. Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning. In Advances in Neural Information Processing Systems, Vol. 34.
  50. Relational graph neural network with hierarchical attention for knowledge graph completion. In AAAI Conference on Artificial Intelligence.
  51. Long-distance dependency combined multi-hop graph neural networks for protein–protein interactions prediction. BMC bioinformatics 23, 1 (2022), 1–21.
  52. Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics 34, 13 (2018), i457–i466.
  53. Layer-dependent importance sampling for training deep and large graph convolutional networks. In Advances in Neural Information Processing Systems.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now