Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GinAR: An End-To-End Multivariate Time Series Forecasting Model Suitable for Variable Missing (2405.11333v1)

Published 18 May 2024 in cs.LG and cs.AI

Abstract: Multivariate time series forecasting (MTSF) is crucial for decision-making to precisely forecast the future values/trends, based on the complex relationships identified from historical observations of multiple sequences. Recently, Spatial-Temporal Graph Neural Networks (STGNNs) have gradually become the theme of MTSF model as their powerful capability in mining spatial-temporal dependencies, but almost of them heavily rely on the assumption of historical data integrity. In reality, due to factors such as data collector failures and time-consuming repairment, it is extremely challenging to collect the whole historical observations without missing any variable. In this case, STGNNs can only utilize a subset of normal variables and easily suffer from the incorrect spatial-temporal dependency modeling issue, resulting in the degradation of their forecasting performance. To address the problem, in this paper, we propose a novel Graph Interpolation Attention Recursive Network (named GinAR) to precisely model the spatial-temporal dependencies over the limited collected data for forecasting. In GinAR, it consists of two key components, that is, interpolation attention and adaptive graph convolution to take place of the fully connected layer of simple recursive units, and thus are capable of recovering all missing variables and reconstructing the correct spatial-temporal dependencies for recursively modeling of multivariate time series data, respectively. Extensive experiments conducted on five real-world datasets demonstrate that GinAR outperforms 11 SOTA baselines, and even when 90% of variables are missing, it can still accurately predict the future values of all variables.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (82)
  1. Imputation of clinical covariates in time series. Machine Learning 110 (2021), 185–248.
  2. Selective imputation for multivariate time series datasets with missing values. IEEE Transactions on Knowledge and Data Engineering (2023).
  3. Spectral temporal graph neural network for multivariate time-series forecasting. Advances in neural information processing systems 33 (2020), 17766–17778.
  4. Brits: Bidirectional recurrent imputation for time series. Advances in neural information processing systems 31 (2018).
  5. Vest: Automatic feature engineering for forecasting. Machine Learning (2021), 1–23.
  6. Multi-Variate Time Series Forecasting on Variable Subsets. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 76–86.
  7. Multi-scale adaptive graph neural network for multivariate time series forecasting. IEEE Transactions on Knowledge and Data Engineering (2023).
  8. Low-rank autoregressive tensor completion for spatiotemporal traffic data imputation. IEEE Transactions on Intelligent Transportation Systems 23, 8 (2021), 12301–12310.
  9. Biased Temporal Convolution Graph Network for Time Series Forecasting with Missing Values.. In The Twelfth International Conference on Learning Representations.
  10. Higher-Order Spatio-Temporal Neural Networks for Covid-19 Forecasting. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5.
  11. Yong Chen and Xiqun Michael Chen. 2022. A novel reinforced dynamic graph convolutional network model with data imputation for network-wide traffic flow prediction. Transportation Research Part C: Emerging Technologies 143 (2022), 103820.
  12. Adaptive graph recurrent network for multivariate time series imputation. In Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings, Part V. Springer, 64–73.
  13. Traffic flow imputation using parallel data and generative adversarial networks. IEEE Transactions on Intelligent Transportation Systems 21, 4 (2019), 1624–1630.
  14. Gatgpt: A pre-trained large language model with graph attention network for spatiotemporal imputation. arXiv preprint arXiv:2311.14332 (2023).
  15. A multi-factor driven spatiotemporal wind power prediction model based on ensemble deep graph attention reinforcement learning networks. Energy 263 (2023), 126034.
  16. Tarnet: Task-aware reconstruction for time-series transformer. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 212–220.
  17. Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks. In International Conference on Learning Representations.
  18. Towards spatio-temporal aware traffic time series forecasting. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 2900–2913.
  19. The pulse of urban transport: Exploring the co-evolving pattern for spatio-temporal forecasting. ACM Transactions on Knowledge Discovery from Data (TKDD) 15, 6 (2021), 1–25.
  20. St-norm: Spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 269–278.
  21. A multi-view multi-task learning framework for multi-variate time series forecasting. IEEE Transactions on Knowledge and Data Engineering (2022).
  22. Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting. IEEE Transactions on Knowledge and Data Engineering (2024).
  23. Saits: Self-attention-based imputation for time series. Expert Systems with Applications 219 (2023), 119619.
  24. Gp-vae: Deep probabilistic time series imputation. In International conference on artificial intelligence and statistics. PMLR, 1651–1661.
  25. MPA-RNN: a novel attention-based recurrent neural networks for total nitrogen prediction. IEEE Transactions on Industrial Informatics 18, 10 (2022), 6516–6525.
  26. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  27. MGCN: Dynamic Spatio-Temporal Multi-Graph Convolutional Neural Network. In 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–9.
  28. Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observations. In 36th Conference on Neural Information Processing Systems (NeurIPS 2022). 1–17.
  29. Spatio-temporal meta-graph learning for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8078–8086.
  30. Dl-traff: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings of the 30th ACM international conference on information & knowledge management. 4515–4525.
  31. Anomaly detection in time series with robust variational quasi-recurrent autoencoders. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 1342–1354.
  32. Thomas N Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations.
  33. Multivariate Transfer Passenger Flow Forecasting with Data Imputation by Joint Deep Learning and Matrix Factorization. Applied Sciences 13, 9 (2023), 5625.
  34. Dynamic Frequency Domain Graph Convolutional Network for Traffic Forecasting. arXiv preprint arXiv:2312.11933 (2023).
  35. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In International Conference on Learning Representations.
  36. Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure. IEEE Transactions on Knowledge and Data Engineering (2023).
  37. Reasoning over different types of knowledge graphs: Static, temporal and multi-modal. arXiv preprint arXiv:2212.05767 (2022), 7576–7584.
  38. Abslearn: a gnn-based framework for aliasing and buffer-size information retrieval. Pattern Analysis and Applications (2023), 1–19.
  39. A new hybrid ensemble deep reinforcement learning model for wind speed short term forecasting. Energy 202 (2020), 117794.
  40. Stochastic iterative graph matching. In International Conference on Machine Learning. PMLR, 6815–6825.
  41. RT-GCN: Gaussian-based spatiotemporal graph convolutional network for robust traffic prediction. Information Fusion 102 (2024), 102078.
  42. E2gan: End-to-end generative adversarial network for multivariate time series imputation. In Proceedings of the 28th international joint conference on artificial intelligence. AAAI Press Palo Alto, CA, USA, 3094–3100.
  43. Soumen Pachal and Avinash Achar. 2022. Sequence Prediction under Missing Data: An RNN Approach without Imputation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 1605–1614.
  44. AdapTraj: A Multi-Source Domain Generalization Framework for Multi-Agent Trajectory Prediction. arXiv preprint arXiv:2312.14394 (2023).
  45. DAMR: Dynamic Adjacency Matrix Representation Learning for Multivariate Time Series Imputation. Proceedings of the ACM on Management of Data 1, 2 (2023), 1–25.
  46. Nrtsi: Non-recurrent time series imputation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5.
  47. Discrete Graph Structure Learning for Forecasting Multiple Time Series. In International Conference on Learning Representations.
  48. A new ensemble deep graph reinforcement learning network for spatio-temporal traffic volume forecasting in a freeway network. Digital Signal Processing 123 (2022), 103419.
  49. Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis. arXiv preprint arXiv:2310.06119 (2023).
  50. Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting. In Proceedings of the 31st ACM International Conference on Information and Knowledge Management. 4454–4458.
  51. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1567–1577.
  52. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. Proceedings of the VLDB Endowment 15, 11 (2022), 2733–2746.
  53. A novel AQI forecasting method based on fusing temporal correlation forecasting with spatial correlation forecasting. Atmospheric Pollution Research 14, 4 (2023), 101717.
  54. Human mobility identification by deep behavior relevant location representation. In International Conference on Database Systems for Advanced Applications. Springer, 439–454.
  55. A new ensemble spatio-temporal PM2. 5 prediction method based on graph attention recursive networks and reinforcement learning. Chaos, Solitons & Fractals 162 (2022), 112405.
  56. A Recurrent Neural Network based Generative Adversarial Network for Long Multivariate Time Series Forecasting. In Proceedings of the 2023 ACM International Conference on Multimedia Retrieval. 181–189.
  57. Joint modeling of local and global temporal dynamics for multivariate time series forecasting with missing values. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 5956–5963.
  58. AI-enhanced spatial-temporal data-mining technology: New chance for next-generation urban computing. The Innovation 4, 2 (2023).
  59. A fingerprint database reconstruction method based on ordinary kriging algorithm for indoor localization. In 2019 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS). IEEE, 224–227.
  60. A multi-view bidirectional spatiotemporal graph network for urban traffic flow imputation. International Journal of Geographical Information Science 36, 6 (2022), 1231–1257.
  61. Learning Dynamic Temporal Relations with Continuous Graph for Multivariate Time Series Forecasting (Student Abstract). In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 16358–16359.
  62. LSTM-autoencoder-based anomaly detection for indoor air quality time-series data. IEEE Sensors Journal 23, 4 (2023), 3787–3800.
  63. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In The Eleventh International Conference on Learning Representations.
  64. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems 34 (2021), 22419–22430.
  65. Inductive graph neural networks for spatiotemporal kriging. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4478–4485.
  66. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 753–763.
  67. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 1907–1913.
  68. Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9632–9643.
  69. Artificial intelligence: A powerful paradigm for scientific research. The Innovation 2, 4 (2021), 100179.
  70. Artificial intelligence for science—bridging data to wisdom. The Innovation 4, 6 (2023).
  71. Spatial-temporal traffic data imputation via graph attention convolutional network. In International Conference on Artificial Neural Networks. Springer, 241–252.
  72. FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective. In Thirty-seventh Conference on Neural Information Processing Systems.
  73. Wireless sensor network survey. Computer networks 52, 12 (2008), 2292–2330.
  74. MTMGNN: Multi-time multi-graph neural network for metro passenger flow prediction. GeoInformatica 27, 1 (2023), 77–105.
  75. Gain: Missing data imputation using generative adversarial nets. In International conference on machine learning. PMLR, 5689–5698.
  76. Dsformer: A double sampling transformer for multivariate time series long-term prediction. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 3062–3072.
  77. MRIformer: A multi-resolution interactive transformer for wind speed multi-step prediction. Information Sciences 661 (2024), 120150.
  78. TriD-MAE: A Generic Pre-trained Model for Multivariate Time Series with Missing Values. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 3164–3173.
  79. T-gcn: A temporal graph convolutional network for traffic prediction. IEEE transactions on intelligent transportation systems 21, 9 (2019), 3848–3858.
  80. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 1234–1241.
  81. SLOTH: Structured Learning and Task-Based Optimization for Time Series Forecasting on Hierarchies. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 11417–11425.
  82. One Fits All: Universal Time Series Analysis by Pretrained LM and Specially Designed Adaptors. arXiv preprint arXiv:2311.14782 (2023).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Chengqing Yu (8 papers)
  2. Fei Wang (574 papers)
  3. Zezhi Shao (14 papers)
  4. Tangwen Qian (5 papers)
  5. Zhao Zhang (250 papers)
  6. Wei Wei (426 papers)
  7. Yongjun Xu (81 papers)
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets