Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunities (2302.01018v4)

Published 2 Feb 2023 in cs.LG and cs.AI

Abstract: Graph Neural Networks (GNNs) have become the leading paradigm for learning on (static) graph-structured data. However, many real-world systems are dynamic in nature, since the graph and node/edge attributes change over time. In recent years, GNN-based models for temporal graphs have emerged as a promising area of research to extend the capabilities of GNNs. In this work, we provide the first comprehensive overview of the current state-of-the-art of temporal GNN, introducing a rigorous formalization of learning settings and tasks and a novel taxonomy categorizing existing approaches in terms of how the temporal aspect is represented and processed. We conclude the survey with a discussion of the most relevant open challenges for the field, from both research and application perspectives.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (106)
  1. DeepEye: Link prediction in dynamic networks based on non-negative matrix factorization. Big Data Mining and Analytics, 2018.
  2. Global explainability of GNNs via logic combination of learned concepts. arXiv preprint arXiv:2210.07147, 2022.
  3. A survey on embedding dynamic graphs. ACM CSUR, 2021.
  4. Weisfeiler–Lehman goes dynamic: An analysis of the expressive power of graph neural networks for attributed and dynamic graphs. arXiv preprint arXiv:2210.03990, 2022.
  5. F.M. Bianchi and V. Lachi. The expressive power of pooling in graph neural networks. arXiv preprint arXiv:2304.01575, 2023.
  6. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in GNNs. In ICLR, 2022.
  7. A multi-scale approach for graph link prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3308–3315, 2020.
  8. Digital proximity tracing on empirical contact networks for pandemic control. Nature Communications, 2021.
  9. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  10. Scalable spatiotemporal graph neural networks. arXiv preprint arXiv:2209.06520, 2022.
  11. Scientific machine learning through physics–informed neural networks: where we are and what’s next. Journal of Scientific Computing, 2022.
  12. Disease persistence on temporal contact networks accounting for heterogeneous infectious periods. Royal Society open science, 6(1):181404, 2019.
  13. Network-based assessment of the vulnerability of italian regions to bovine brucellosis. Preventive veterinary medicine, 158:25–34, 2018.
  14. Hyte: Hyperplane-based temporally aware knowledge graph embedding. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2001–2011, 2018.
  15. Learning dynamic context graphs for predicting social events. In ACM SIGKDD, 2019.
  16. A new perspective on the approximation capability of GNNs. arXiv preprint arXiv:2106.08992, 2021.
  17. J. Enright and R.K. Rowland. Epidemics on dynamic networks. Epidemics, 2018.
  18. J. H. Faghmous and V. Kumar. A Big Data Guide to Understanding Climate Change: The Case for Theory-Guided Data Science. Big Data, 2(3), 2014.
  19. Graph neural networks for social recommendation. WWW ’19, New York, NY, USA, 2019. Association for Computing Machinery.
  20. Graph neural networks for recommender system. WSDM ’22, page 1623–1625, New York, NY, USA, 2022. Association for Computing Machinery.
  21. Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 2022.
  22. Large-scale learnable graph convolutional networks. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1416–1424, 2018.
  23. Detecting the critical states during disease development based on temporal network flow entropy. Briefings in Bioinformatics, 2022.
  24. S. Gao. Spatio-temporal analytics for exploring human mobility patterns and urban dynamics in the mobile age. Spatial Cognition & Computation, 15(2):86–114, 2015.
  25. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997, 2018.
  26. Dyngem: Deep embedding method for dynamic graphs. arXiv preprint arXiv:1805.11273, 2018.
  27. DynaGraph: dynamic graph neural networks at scale. In ACM SIGMOD22 GRADES-NDA, 2022.
  28. S. Gupta and S. Bedathur. A survey on temporal graph representation learning and generative modeling. arXiv preprint arXiv:2208.12126, 2022.
  29. Variational graph recurrent neural networks. NeurIPS, 32, 2019.
  30. Inductive representation learning on large graphs. NeurIPS, 30, 2017.
  31. An explainer for temporal graph neural networks. In GLOBECOM - IEEE Global Communications Conference 2022, pages 6384–6389. IEEE, 2022.
  32. Aberrant frontal and temporal complex network structure in schizophrenia: A graph theoretical analysis. Journal of Neuroscience, 2010.
  33. Disease spreading modeling and analysis: A survey. Briefings in Bioinformatics, 23(4):bbac230, 2022.
  34. Open Graph Benchmark: Datasets for machine learning on graphs. NeurIPS, 33:22118–22133, 2020.
  35. Exposure reconstruction using space-time information technology. Encyclopedia of Environmental Health, pages 793–804, 2019.
  36. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications, 207:117921, 2022.
  37. Representation learning for dynamic graphs: A survey. Journal of Machine Learning Research, 2020.
  38. R. Keisler. Forecasting global weather with graph neural networks. arXiv preprint arXiv:2202.07575, 2022.
  39. T.N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICLR 2016.
  40. T.N. Kipf and M. Welling. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016.
  41. Contact-based model for epidemic spreading on temporal networks. Physical Review X, 9(3):031017, 2019.
  42. Learning dynamic embeddings from temporal interactions. arXiv preprint arXiv:1812.02289, 2018.
  43. PAW Lewis. Multivariate point processes. In Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability, volume 1, page 401. University of California Press, 1972.
  44. Towards fine-grained temporal network representation via time-reinforced random walk. In AAAI, volume 34, pages 4973–4980, 2020.
  45. Analysis of complex customer networks: A real-world banking example. In 2022 45th Jubilee International Convention on Information, Communication and Electronic Technology (MIPRO), pages 321–326. IEEE, 2022.
  46. Explaining the explainers in graph neural networks: a comparative study. arXiv preprint arXiv:2210.15304, 2022.
  47. Neighbourhood matching creates realistic surrogate temporal networks. arXiv preprint arXiv:2205.08820, 2022.
  48. An efficient procedure for mining egocentric temporal motifs. Data Mining and Knowledge Discovery, 2022.
  49. Parameterized explainer for graph neural network. Advances in Neural Information Processing Systems, 33:19620–19631, 2020.
  50. Y. Luo and P. Li. Neighborhood-aware scalable temporal network representation learning. arXiv preprint arXiv:2209.01084, 2022.
  51. Streaming graph neural networks. In ACM SIGIR, 2020.
  52. Generating mobility networks with generative adversarial networks. EPJ Data Science, 11(1):58, 2022.
  53. Earthquake location and magnitude estimation with graph neural networks. In IEEE ICIP, 2022.
  54. Spatial cluster analysis of early stage breast cancer: a method for public health practice using cancer registry data. Cancer Causes & Control, 20:1061–1069, 2009.
  55. A. Micheli and D. Tortorella. Discrete-time dynamic graph echo state networks. Neurocomputing, 496:85–95, 2022.
  56. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5115–5124, 2017.
  57. Weisfeiler and Leman go neural: Higher-order graph neural networks. In AAAI, 2019.
  58. Temporal network analysis using zigzag persistence. EPJ Data Science, 12(1):6, 2023.
  59. Spatio-temporal deep graph infomax. arXiv preprint arXiv:1904.06316, 2019.
  60. Cluster detection methods applied to the upper cape cod cancer data. Environmental Health, 4:1–9, 2005.
  61. EvolveGCN: Evolving graph convolutional networks for dynamic graphs. In AAAI, 2020.
  62. Learning mesh-based simulation with graph networks. In ICLR, 2021.
  63. M. Qin and D Yeung. Temporal link prediction: A unified framework, taxonomy, and review. arXiv preprint arXiv:2210.08765, 2022.
  64. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In NeurIPS, 2008.
  65. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561, 2017.
  66. Temporal graph networks for deep learning on dynamic graphs. arXiv preprint arXiv:2006.10637, 2020.
  67. Exploring customer data using spatio-temporal analysis: Case study of fixed broadband provider. International Journal of Applied Science and Engineering, 16(2):133–147, 2019.
  68. Pytorch Geometric Temporal: Spatiotemporal signal processing with neural machine learning models. In ACM CIKM, 2021.
  69. Inferring patient zero on temporal networks via graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9632–9640, 2023.
  70. Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In WSDM, 2020.
  71. Dyane: dynamics-aware node embedding for temporal networks. arXiv preprint arXiv:1909.05976, 2019.
  72. R. Sato. A survey on the expressive power of graph neural networks. arXiv preprint arXiv:2003.04078, 2020.
  73. Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey. IEEE Access, 9:79143–79168, 2021.
  74. Visualizing covid-19 pandemic risk through network connectedness. International Journal of Infectious Diseases, 96:558–561, Jul 2020.
  75. Provably expressive temporal graph networks. NeurIPS, 2022.
  76. Learning to represent the evolution of dynamic graphs with recurrent models. In Companion proceedings of the 2019 world wide web conference, pages 301–307, 2019.
  77. Graph neural networks designed for different graph types: A survey. arXiv preprint arXiv:2204.03080, 2022.
  78. Understanding over-squashing and bottlenecks on graphs via curvature. arXiv preprint arXiv:2111.14522, 2021.
  79. Dyrep: Learning representations over dynamic graphs. In International Conference on Learning Representations, 2019.
  80. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In international conference on machine learning, pages 3462–3471. PMLR, 2017.
  81. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.
  82. On the limit of explaining black-box temporal graph neural networks. arXiv preprint arXiv:2212.00952, 2022.
  83. APAN: Asynchronous propagation attention network for real-time temporal graph embedding. In SIGMOD, 2021.
  84. Inductive representation learning in temporal networks via causal anonymous walks. arXiv preprint arXiv:2101.05974, 2021.
  85. David C Wheeler. A comparison of spatial clustering and cluster detection techniques for childhood leukemia incidence in ohio, 1996–2003. International journal of health geographics, 6(1):1–16, 2007.
  86. Integrating scientific knowledge with machine learning for engineering and environmental systems. ACM Computing Surveys, 55(4), 2022.
  87. Simplifying graph convolutional networks. In International conference on machine learning, pages 6861–6871. PMLR, 2019.
  88. Graph neural networks in recommender systems: a survey. ACM CSUR, 2022.
  89. Graph learning: A survey. IEEE Transactions on Artificial Intelligence, 2(2):109–127, 2021.
  90. Explaining temporal graph models through an explorer-navigator framework. In The Eleventh International Conference on Learning Representations.
  91. A survey on dynamic network embedding. arXiv preprint arXiv:2006.08093, 2020.
  92. Inductive representation learning on temporal graphs. arXiv preprint arXiv:2002.07962, 2020.
  93. Spatio-temporal attentive rnn for node classification in temporal attributed graphs. In IJCAI, pages 3947–3953, 2019.
  94. Transformer-style relational reasoning with dynamic memory updating for temporal network modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4546–4554, 2021.
  95. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
  96. Dynamic network embedding survey. Neurocomputing, 2022.
  97. Featurenorm: L2 feature normalization for dynamic graph embedding. In ICDM, 2020.
  98. M. Yin and M. Zhou. Semi-implicit variational inference. In ICML, 2018.
  99. Gnnexplainer: Generating explanations for graph neural networks. Advances in Neural Information Processing Systems, 32, 2019.
  100. ROLAND: graph learning framework for dynamic graphs. In ACM SIGKDD, 2022.
  101. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875, 2017.
  102. M. Zhang and Y. Chen. Link prediction based on graph neural networks. NeurIPS, 2018.
  103. Weisfeiler-lehman neural machine for link prediction. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 575–583, 2017.
  104. Deep learning on graphs: A survey. IEEE Transactions on Knowledge and Data Engineering, 34(1):249–270, 2020.
  105. Node classification in temporal graphs through stochastic sparsification and temporal structural convolution. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part III, pages 330–346. Springer, 2021.
  106. TGL: A general framework for temporal GNN training on billion-scale graphs. arXiv preprint arXiv:2203.14883, 2022.
Citations (42)
View on Semantic Scholar

Summary

  • The paper provides a comprehensive taxonomy and formal framework distinguishing static and temporal graphs in GNN research.
  • It analyzes snapshot-based and event-based models to capture dynamic node and edge interactions over time.
  • The study identifies critical challenges such as benchmarking, expressiveness, and scalability, and outlines future research directions.

Graph Neural Networks for Temporal Graphs: An Overview of State-of-the-Art, Challenges, and Opportunities

The paper "Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunities" delivers an exhaustive survey on the evolving field of Graph Neural Networks (GNNs) applied to temporal graphs. The work is conducted by Antonio Longa, Veronica Lachi, Gabriele Santin, Monica Bianchini, Bruno Lepri, Pietro Liò, and Franco Scarselli. This paper identifies the burgeoning significance of temporal graphs in various real-world domains and sets itself the goal of organizing existing research efforts into a coherent structure while shedding light on the most pressing challenges and prospective innovations in the domain.

Overview and Taxonomy

The exploration begins with the observation of the dynamic nature of many real-world graph scenarios, which contrasts with the largely static graph settings typically addressed by traditional GNNs. Temporal graphs introduce complexities as nodes, edges, and their attributes alter over time. To navigate this landscape, researchers propose a rigorous formalization of learning settings and tasks.

The paper argues for a formal distinction between static and temporal graphs through well-defined mathematical frameworks, highlighting essential differences in terms of node and edge representations and updates. Temporal graphs are theoretically defined, allowing for a categorization that can accommodate both discrete and continuous time scenarios. Consequently, this leads to defining Discrete Time Temporal Graphs (DTTGs) and Continuous-Time Temporal Graphs (CTTGs). A novel taxonomy is introduced, bifurcating existing GNN approaches for temporal graphs into two primary domains: snapshot-based and event-based models.

  • Snapshot-based Models: These models treat temporal graphs as sequences of graph snapshots over time. They are further subdivided into Model Evolution and Embedding Evolution methods. Model Evolution involves updating the model parameters over time, while Embedding Evolution updates the node embeddings without altering the model architecture itself.
  • Event-based Models: Address continuous interactions and are suited for dynamic updates on node states in real-time, accommodating changes through attention mechanisms and temporal message-passing paradigms.

Tasks and Evaluation

The analysis categorizes supervised and unsupervised learning tasks specific to temporal graphs, including temporal node and edge classification, temporal link prediction, clustering, and low-dimensional embeddings. This systematic categorization reveals that much of the present work has predominantly concentrated on node classification and link prediction within future prediction settings, leaving substantial gaps in other potential applications and outcome predictions, such as clustering and visualization.

Challenges and Future Directions

The authors stress several areas in need of further research and development:

  • Dataset and Benchmarking: The field lacks standardized datasets and evaluation benchmarks analogous to the ones existing for traditional GNNs. This absence complicates objective comparison and evaluation of method efficacy.
  • Expressiveness and Theoretical Foundations: The expressiveness of TGNNs and rigorous theoretical underpinnings need significant enhancement. Delineating the expressive boundaries of these models can lead to the development of more powerful architectures, potentially guiding the field similar to how expressiveness studies contributed to static GNNs.
  • Scalability and Efficiency: Ensuring models scale to larger graphs without sacrificing temporal precision is crucial. Practical applications often involve vast, intricate networks where computational efficiency and resource management become pivotal.
  • Real-World Applications: Promising application areas such as epidemic modeling, climate science, and physics-informed network predictions remain underexplored, offering potential high-impact avenues for applying temporal GNNs. These domains highlight the need for models capable of intricate spatio-temporal reasoning and high predictive accuracy.

Conclusion

The paper contributes to the literature by synthesizing current research thrusts and identifying gaps within temporal graph analysis powered by GNNs. By illuminating pathways for future exploration, it offers valuable insights which, if pursued, can advance theoretical knowledge, enhance practical capabilities, and broaden application domains for temporal graph analysis.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube