Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Accelerating Dynamic Network Embedding with Billions of Parameter Updates to Milliseconds (2306.08967v1)

Published 15 Jun 2023 in cs.SI and cs.LG

Abstract: Network embedding, a graph representation learning method illustrating network topology by mapping nodes into lower-dimension vectors, is challenging to accommodate the ever-changing dynamic graphs in practice. Existing research is mainly based on node-by-node embedding modifications, which falls into the dilemma of efficient calculation and accuracy. Observing that the embedding dimensions are usually much smaller than the number of nodes, we break this dilemma with a novel dynamic network embedding paradigm that rotates and scales the axes of embedding space instead of a node-by-node update. Specifically, we propose the Dynamic Adjacency Matrix Factorization (DAMF) algorithm, which achieves an efficient and accurate dynamic network embedding by rotating and scaling the coordinate system where the network embedding resides with no more than the number of edge modifications changes of node embeddings. Moreover, a dynamic Personalized PageRank is applied to the obtained network embeddings to enhance node embeddings and capture higher-order neighbor information dynamically. Experiments of node classification, link prediction, and graph reconstruction on different-sized dynamic graphs suggest that DAMF advances dynamic network embedding. Further, we unprecedentedly expand dynamic network embedding experiments to billion-edge graphs, where DAMF updates billion-level parameters in less than 10ms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Watch your step: Learning node embeddings via graph attention. Advances in neural information processing systems 31 (2018).
  2. Nino Arsov and Georgina Mirceva. 2019. Network Embedding: An Overview. ArXiv abs/1911.11726 (2019).
  3. Scaling graph neural networks with approximate pagerank. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2464–2473.
  4. Deep neural networks for learning graph representations. In Proceedings of the AAAI conference on artificial intelligence, Vol. 30.
  5. Chen Chen and Hanghang Tong. 2015. Fast eigen-functions tracking on dynamic graphs. In Proceedings of the 2015 SIAM international conference on data mining. SIAM, 559–567.
  6. A Survey of Knowledge Graph Embedding and Their Applications. ArXiv abs/2107.07842 (2021).
  7. A survey on network embedding. IEEE transactions on knowledge and data engineering 31, 5 (2018), 833–852.
  8. Dynamic network embedding: An extended approach for skip-gram based network embedding.. In IJCAI, Vol. 2018. 2086–2092.
  9. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997 (2018).
  10. Exploring temporal information for dynamic network embedding. IEEE Transactions on Knowledge and Data Engineering 34, 8 (2020), 3754–3764.
  11. Dyngem: Deep embedding method for dynamic graphs. arXiv preprint arXiv:1805.11273 (2018).
  12. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855–864.
  13. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review 53, 2 (2011), 217–288.
  14. Representation learning on graphs: Methods and applications. arXiv preprint arXiv:1709.05584 (2017).
  15. Glodyne: Global topology preserving dynamic network embedding. IEEE Transactions on Knowledge and Data Engineering 34, 10 (2020), 4826–4837.
  16. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL-HLT. 4171–4186.
  17. What is Twitter, a social network or a news media?. In Proceedings of the 19th international conference on World wide web. 591–600.
  18. Attributed network embedding for learning in a dynamic environment. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 387–396.
  19. Real-time streaming graph embedding through local actions. In Companion Proceedings of The 2019 World Wide Web Conference. 285–293.
  20. Depthlgp: Learning embeddings of out-of-sample nodes in dynamic networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
  21. dynnode2vec: Scalable dynamic network embedding. In 2018 IEEE International Conference on Big Data (Big Data). IEEE, 3762–3765.
  22. Matt Mahoney. 2011. Large text compression benchmark.
  23. Measurement and analysis of online social networks. In Proceedings of the 7th ACM SIGCOMM conference on Internet measurement. 29–42.
  24. To embed or not: network embedding as a paradigm in computational biology. Frontiers in genetics 10 (2019), 381.
  25. Continuous-time dynamic network embeddings. In Companion proceedings of the the web conference 2018. 969–976.
  26. The PageRank citation ranking: Bringing order to the web. Technical Report. Stanford InfoLab.
  27. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 5363–5370.
  28. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701–710.
  29. Lightne: A lightweight graph processing system for network embedding. In Proceedings of the 2021 international conference on management of data. 2281–2289.
  30. Netsmf: Large-scale network embedding as sparse matrix factorization. In The World Wide Web Conference. 1509–1520.
  31. Network embedding as matrix factorization: Unifying deepwalk, line, pte, and node2vec. In Proceedings of the eleventh ACM international conference on web search and data mining. 459–467.
  32. Heterogeneous information network embedding for recommendation. IEEE Transactions on Knowledge and Data Engineering 31, 2 (2018), 357–370.
  33. Node embedding over temporal graphs. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 4605–4612.
  34. Network embedding in biomedical data science. Briefings in bioinformatics 21, 1 (2020), 182–197.
  35. Line: Large-scale information network embedding. In Proceedings of the 24th international conference on world wide web. 1067–1077.
  36. Lei Tang and Huan Liu. 2009a. Relational learning via latent social dimensions. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. 817–826.
  37. Lei Tang and Huan Liu. 2009b. Scalable learning of collective behavior based on sparse social dimensions. In Proceedings of the 18th ACM conference on Information and knowledge management. 1107–1116.
  38. Verse: Versatile graph embeddings from similarity measures. In Proceedings of the 2018 world wide web conference. 539–548.
  39. Approximate graph propagation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1686–1696.
  40. Graphgan: Graph representation learning with generative adversarial nets. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
  41. Network embedding based recommendation method in social networks. In Companion Proceedings of the The Web Conference 2018. 11–12.
  42. Homogeneous network embedding for massive graphs via reweighted personalized PageRank. Proceedings of the VLDB Endowment 13, 5 (2020), 670–683.
  43. Yuan Yin and Zhewei Wei. 2019a. Scalable graph embeddings via sparse transpose proximities. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1429–1437.
  44. Yuan Yin and Zhewei Wei. 2019b. Scalable graph embeddings via sparse transpose proximities. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1429–1437.
  45. Netwalk: A flexible deep embedding approach for anomaly detection in dynamic networks. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 2672–2681.
  46. Hongyuan Zha and Horst D Simon. 1999. On updating problems in latent semantic indexing. SIAM Journal on Scientific Computing 21, 2 (1999), 782–791.
  47. Approximate personalized pagerank on dynamic graphs. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1315–1324.
  48. ProNE: Fast and Scalable Network Representation Learning.. In IJCAI, Vol. 19. 4278–4284.
  49. Billion-scale network embedding with iterative random projection. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 787–796.
  50. Timers: Error-bounded svd restart on dynamic networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
  51. Arbitrary-order proximity preserved network embedding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 2778–2786.
  52. Instant Graph Neural Networks for Dynamic Graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’22). 2605–2615.
  53. Dynamic network embedding by modeling triadic closure process. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
  54. High-order proximity preserved embedding for dynamic networks. IEEE Transactions on Knowledge and Data Engineering 30, 11 (2018), 2134–2144.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Haoran Deng (13 papers)
  2. Yang Yang (884 papers)
  3. Jiahe Li (16 papers)
  4. Haoyang Cai (4 papers)
  5. Shiliang Pu (106 papers)
  6. Weihao Jiang (12 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now