AdapTraj: A Multi-Source Domain Generalization Framework for Multi-Agent Trajectory Prediction (2312.14394v1)
Abstract: Multi-agent trajectory prediction, as a critical task in modeling complex interactions of objects in dynamic systems, has attracted significant research attention in recent years. Despite the promising advances, existing studies all follow the assumption that data distribution observed during model learning matches that encountered in real-world deployments. However, this assumption often does not hold in practice, as inherent distribution shifts might exist in the mobility patterns for deployment environments, thus leading to poor domain generalization and performance degradation. Consequently, it is appealing to leverage trajectories from multiple source domains to mitigate such discrepancies for multi-agent trajectory prediction task. However, the development of multi-source domain generalization in this task presents two notable issues: (1) negative transfer; (2) inadequate modeling for external factors. To address these issues, we propose a new causal formulation to explicitly model four types of features: domain-invariant and domain-specific features for both the focal agent and neighboring agents. Building upon the new formulation, we propose AdapTraj, a multi-source domain generalization framework specifically tailored for multi-agent trajectory prediction. AdapTraj serves as a plug-and-play module that is adaptable to a variety of models. Extensive experiments on four datasets with different domains demonstrate that AdapTraj consistently outperforms other baselines by a substantial margin.
- S. Wang, Z. Bao, J. S. Culpepper, and G. Cong, “A survey on trajectory data management, analytics, and learning,” ACM Comput. Surv., vol. 54, no. 2, pp. 39:1–39:36, 2022.
- A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, F. Li, and S. Savarese, “Social LSTM: human trajectory prediction in crowded spaces,” in CVPR, 2016, pp. 961–971.
- A. Gupta, J. Johnson, F. Li, S. Savarese, and A. Alahi, “Social GAN: socially acceptable trajectories with generative adversarial networks,” in CVPR, 2018, pp. 2255–2264.
- Y. Huang, H. Bi, Z. Li, T. Mao, and Z. Wang, “Stgat: Modeling spatial-temporal interactions for human trajectory prediction,” in ICCV, 2019, pp. 6272–6281.
- K. Mangalam, H. Girase, S. Agarwal, K. Lee, E. Adeli, J. Malik, and A. Gaidon, “It is not the journey but the destination: Endpoint conditioned trajectory prediction,” in ECCV, 2020.
- G. Chen, J. Li, J. Lu, and J. Zhou, “Human trajectory prediction via counterfactual analysis,” in ICCV, 2021, pp. 9824–9833.
- Y. Liu, R. Cadei, J. Schweizer, S. Bahmani, and A. Alahi, “Towards robust and adaptive motion forecasting: A causal representation perspective,” in CVPR, 2022, pp. 17 081–17 092.
- T. Zhong, Z. Chi, L. Gu, Y. Wang, Y. Yu, and J. Tang, “Meta-dmoe: Adapting to domain shift by meta-distillation from mixture-of-experts,” in NeurIPS, 2022.
- K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy, “Domain generalization: A survey,” TPAMI, vol. 45, no. 4, pp. 4396–4415, 2023.
- R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,” Neural Comput., vol. 3, no. 1, pp. 79–87, 1991.
- D. Helbing and P. Molnar, “Social force model for pedestrian dynamics,” Physical review E, vol. 51, no. 5, p. 4282, 1995.
- B. Pang, T. Zhao, X. Xie, and Y. N. Wu, “Trajectory prediction with latent belief energy-based model,” in CVPR, 2021, pp. 11 814–11 824.
- J. Li, F. Yang, M. Tomizuka, and C. Choi, “Evolvegraph: Multi-agent trajectory prediction with dynamic relational reasoning,” NeurIPS, vol. 33, pp. 19 783–19 794, 2020.
- S. Hu, K. Zhang, Z. Chen, and L. Chan, “Domain generalization via multidomain discriminant analysis,” in UAI, vol. 115, 2019, pp. 292–302.
- J. Wang, C. Lan, C. Liu, Y. Ouyang, and T. Qin, “Generalizing to unseen domains: A survey on domain generalization,” in IJCAI, 2021, pp. 4627–4635.
- K. Bousmalis, G. Trigeorgis, N. Silberman, D. Krishnan, and D. Erhan, “Domain separation networks,” in NeurIPS, 2016, pp. 343–351.
- Y. Wang, H. Wu, J. Zhang, Z. Gao, J. Wang, P. S. Yu, and M. Long, “Predrnn: A recurrent neural network for spatiotemporal predictive learning,” TPAMI, vol. 45, no. 2, pp. 2208–2225, 2023.
- F. Giuliari, I. Hasan, M. Cristani, and F. Galasso, “Transformer networks for trajectory forecasting,” in ICPR, 2020, pp. 10 335–10 342.
- K. Shao, Y. Wang, Z. Zhou, X. Xie, and G. Wang, “Trajforesee: How limited detailed trajectories enhance large-scale sparse information to predict vehicle trajectories?” in ICDE, 2021, pp. 2189–2194.
- N. Dryden and T. Hoefler, “Spatial mixture-of-experts,” NeurIPS, vol. 35, pp. 11 697–11 713, 2022.
- S. Bucci, A. D’Innocente, Y. Liao, F. M. Carlucci, B. Caputo, and T. Tommasi, “Self-supervised learning across domains,” TPAMI, vol. 44, no. 9, pp. 5516–5528, 2022.
- D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” in NeurIPS, 2014, pp. 2366–2374.
- Z. Xiao, Y. Jiang, G. Tang, L. Liu, S. Xu, Y. Xiao, and W. Yan, “Adversarial mixture of experts with category hierarchy soft constraint,” in ICDE, 2021, pp. 2453–2463.
- P. Kothari, S. Kreiss, and A. Alahi, “Human trajectory forecasting in crowds: A deep learning perspective,” TITS, vol. 23, no. 7, pp. 7386–7400, 2021.
- S. Pellegrini, A. Ess, K. Schindler, and L. V. Gool, “You’ll never walk alone: Modeling social behavior for multi-target tracking,” in ICCV, 2009, pp. 261–268.
- A. Lerner, Y. Chrysanthou, and D. Lischinski, “Crowds by example,” in Computer graphics forum, vol. 26, no. 3, 2007, pp. 655–664.
- L. Sun, Z. Yan, S. M. Mellado, M. Hanheide, and T. Duckett, “3dof pedestrian trajectory prediction learned from long-term autonomous mobile robot deployment data,” ICRA, pp. 1–7, 2017.
- S. Yi, H. Li, and X. Wang, “Understanding pedestrian behaviors from stationary crowd groups,” in CVPR, 2015, pp. 3488–3496.
- A. Robicquet, A. Sadeghian, A. Alahi, and S. Savarese, “Learning social etiquette: Human trajectory understanding in crowded scenes,” in ECCV, vol. 9912, 2016, pp. 549–565.
- J. Zhang, C. Wang, J. Wang, and J. X. Yu, “Inferring continuous dynamic social influence and personal preference for temporal behavior prediction,” Proc. VLDB Endow., vol. 8, no. 3, pp. 269–280, 2014.
- R. Volpi, H. Namkoong, O. Sener, J. C. Duchi, V. Murino, and S. Savarese, “Generalizing to unseen domains via adversarial data augmentation,” in NeurIPS, 2018, pp. 5339–5349.
- Y. Zhang, C. Li, I. W. Tsang, H. Xu, L. Duan, H. Yin, W. Li, and J. Shao, “Diverse preference augmentation with multiple domains for cold-start recommendations,” in ICDE, 2022, pp. 2942–2955.
- W. Li, Z. Xu, D. Xu, D. Dai, and L. V. Gool, “Domain generalization and adaptation using low rank exemplar svms,” TPAMI, vol. 40, no. 5, pp. 1114–1127, 2018.
- V. Piratla, P. Netrapalli, and S. Sarawagi, “Efficient domain generalization via common-specific low-rank decomposition,” in ICML, vol. 119, 2020, pp. 7728–7738.
- Y. Xu, X. Liu, X. Cao, C. Huang, E. Liu, S. Qian, X. Liu, Y. Wu, F. Dong, C.-W. Qiu et al., “Artificial intelligence: A powerful paradigm for scientific research,” The Innovation, vol. 2, no. 4, 2021.
- T. Kahveci, A. K. Singh, and A. Gürel, “An efficient index structure for shift and scale invariant search of multi-attribute time sequences,” in ICDE, 2002, p. 266.
- W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,” JMLR, vol. 23, pp. 120:1–120:39, 2022.
- D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen, “Gshard: Scaling giant models with conditional computation and automatic sharding,” in ICLR, 2021.
- G. A. Carpenter and S. Grossberg, “Adaptive resonance theory,” in Encyclopedia of Machine Learning, 2010, pp. 22–35.