Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hierarchical Control for Cooperative Teams in Competitive Autonomous Racing (2204.13070v3)

Published 27 Apr 2022 in cs.MA and cs.GT

Abstract: We investigate the problem of autonomous racing among teams of cooperative agents that are subject to realistic racing rules. Our work extends previous research on hierarchical control in head-to-head autonomous racing by considering a generalized version of the problem while maintaining the two-level hierarchical control structure. A high-level tactical planner constructs a discrete game that encodes the complex rules using simplified dynamics to produce a sequence of target waypoints. The low-level path planner uses these waypoints as a reference trajectory and computes high-resolution control inputs by solving a simplified formulation of a racing game with a simplified representation of the realistic racing rules. We explore two approaches for the low-level path planner: training a multi-agent reinforcement learning (MARL) policy and solving a linear-quadratic Nash game (LQNG) approximation. We evaluate our controllers on simple and complex tracks against three baselines: an end-to-end MARL controller, a MARL controller tracking a fixed racing line, and an LQNG controller tracking a fixed racing line. Quantitative results show our hierarchical methods outperform the baselines in terms of race wins, overall team performance, and compliance with the rules. Qualitatively, we observe the hierarchical controllers mimic actions performed by expert human drivers such as coordinated overtaking, defending against multiple opponents, and long-term planning for delayed advantages.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. A. Faisal, M. Kamruzzaman, T. Yigitcanlar, and G. Currie, “Understanding autonomous vehicles,” Journal of transport and land use, vol. 12, no. 1, pp. 45–72, 2019.
  2. J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, and R. Mangharam, “Autonomous vehicles on the edge: A survey on autonomous vehicle racing,” IEEE Open Journal of Intelligent Transportation Systems, vol. 3, pp. 458–488, 2022.
  3. S. Edelstein, “The technologies your car inherited from race cars,” https://www.digitaltrends.com/cars/racing-tech-in-your-current-car/, 2019, accessed: 02-17-2022.
  4. M. Wang, Z. Wang, J. Talbot, J. C. Gerdes, and M. Schwager, “Game theoretic planning for self-driving cars in competitive scenarios,” in Robotics: Science and Systems XV, University of Freiburg, Freiburg im Breisgau, Germany, June 22-26, 2019, A. Bicchi, H. Kress-Gazit, and S. Hutchinson, Eds., 2019. [Online]. Available: https://doi.org/10.15607/RSS.2019.XV.048
  5. ——, “Game-theoretic planning for self-driving cars in multivehicle competitive scenarios,” IEEE Transactions on Robotics, pp. 1–13, 2021. [Online]. Available: https://doi.org/10.1109/tro.2020.3047521
  6. N. Li, E. Goubault, L. Pautet, and S. Putot, “Autonomous racecar control in head-to-head competition using mixed-integer quadratic programming,” in 2021 International Conference on Robotics and Automation (ICRA 2021) - Workshop Opportunities and Challenges With Autonomous Racing.   IEEE, 2021. [Online]. Available: https://linklab-uva.github.io/icra-autonomous-racing/contributed_papers/paper2.pdf
  7. S. He, J. Zeng, and K. Sreenath, “Autonomous racing with multiple vehicles using a parallelized optimization with safety guarantee using control barrier functions,” arXiv preprint arXiv:2112.06435, 2022. [Online]. Available: https://arxiv.org/abs/2112.06435
  8. T. Martin, “The guide to road racing, part 8: Passing etiquette,” https://www.windingroad.com/articles/blogs/the-road-racers-guide-to-passing-etiquette/, 2020, accessed: 02-17-2022.
  9. R. S. Thakkar, A. S. Samyal, D. Fridovich-Keil, Z. Xu, and U. Topcu, “Hierarchical control for head-to-head autonomous racing,” Field Robotics, vol. 4, pp. 46–69, 2024. [Online]. Available: https://fieldrobotics.net/Field_Robotics/Volume_4_files/Vol4_02.pdf
  10. J. Xu, C. Yan, Y. Xu, J. Shi, K. Sheng, and X. Xu, “A hierarchical game theory based demand optimization method for grid-interaction of energy flexible buildings,” Frontiers in Energy Research, p. 500, 2021.
  11. C. Tomlin, G. Pappas, J. Lygeros, D. Godbole, S. Sastry, and G. Meyer, “Hybrid control in air traffic management systems1,” IFAC Proceedings Volumes, vol. 29, no. 1, pp. 5512–5517, 1996, 13th World Congress of IFAC, 1996, San Francisco USA, 30 June - 5 July. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1474667017585596
  12. J.-H. Hou and T. Wang, “The development of a simulated car racing controller based on monte-carlo tree search,” in 2016 Conference on Technologies and Applications of Artificial Intelligence (TAAI), 2016, pp. 104–109.
  13. J. L. Vazquez, M. Bruhlmeier, A. Liniger, A. Rupenyan, and J. Lygeros, “Optimization-based hierarchical motion planning for autonomous racing,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, Oct. 2020. [Online]. Available: https://doi.org/10.1109/iros45743.2020.9341731
  14. T. Stahl, A. Wischnewski, J. Betz, and M. Lienkamp, “Multilayer graph-based trajectory planning for race vehicles in dynamic scenarios,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC).   IEEE, Oct. 2019. [Online]. Available: https://doi.org/10.1109/itsc.2019.8917032
  15. J. Kabzan, L. Hewing, A. Liniger, and M. N. Zeilinger, “Learning-based model predictive control for autonomous racing,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3363–3370, Oct. 2019. [Online]. Available: https://doi.org/10.1109/lra.2019.2926677
  16. A. Remonda, S. Krebs, E. E. Veas, G. Luzhnica, and R. Kern, “Formula RL: deep reinforcement learning for autonomous racing using telemetry data,” CoRR, vol. abs/2104.11106, 2021. [Online]. Available: https://arxiv.org/abs/2104.11106
  17. T. de Bruin, J. Kober, K. Tuyls, and R. Babuska, “Integrating state representation learning into deep reinforcement learning,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1394–1401, Jul. 2018. [Online]. Available: https://doi.org/10.1109/lra.2018.2800101
  18. T. Weiss and M. Behl, “Deepracing: Parameterized trajectories for autonomous racing,” arXiv preprint arXiv:2005.05178, 2020. [Online]. Available: https://arxiv.org/abs/2005.05178
  19. R. Spica, E. Cristofalo, Z. Wang, E. Montijano, and M. Schwager, “A real-time game theoretic planner for autonomous two-player drone racing,” IEEE Transactions on Robotics, vol. 36, no. 5, pp. 1389–1403, 2020.
  20. W. Schwarting, T. Seyde, I. Gilitschenski, L. Liebenwein, R. Sander, S. Karaman, and D. Rus, “Deep latent competition: Learning to race using visual control policies in latent space,” arXiv preprint arXiv:2102.09812, 2021. [Online]. Available: https://arxiv.org/abs/2102.09812
  21. Y. Song, H. Lin, E. Kaufmann, P. Duerr, and D. Scaramuzza, “Autonomous overtaking in gran turismo sport using curriculum reinforcement learning,” arXiv preprint arXiv:2103.14666, 2021. [Online]. Available: https://arxiv.org/abs/2103.14666
  22. P. R. Wurman, S. Barrett, K. Kawamoto, J. MacGlashan, K. Subramanian, T. J. Walsh, R. Capobianco, A. Devlic, F. Eckert, F. Fuchs et al., “Outracing champion gran turismo drivers with deep reinforcement learning,” Nature, vol. 602, no. 7896, pp. 223–228, 2022.
  23. A. Liniger, “Path planning and control for autonomous racing,” Ph.D. dissertation, ETH Zürich, 2018.
  24. J. F. Fisac, E. Bronstein, E. Stefansson, D. Sadigh, S. S. Sastry, and A. D. Dragan, “Hierarchical game-theoretic planning for autonomous vehicles,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 9590–9596.
  25. M. Moghadam and G. H. Elkaim, “A hierarchical architecture for sequential decision-making in autonomous driving using deep reinforcement learning,” arXiv preprint arXiv:1906.08464, 2019.
  26. M. Abdoos, “A cooperative multiagent system for traffic signal control using game theory and reinforcement learning,” IEEE Intelligent Transportation Systems Magazine, vol. 13, no. 4, pp. 6–16, 2021.
  27. K.-S. Hwang, S.-W. Tan, and C.-C. Chen, “Cooperative strategy based on adaptive q-learning for robot soccer systems,” IEEE Transactions on Fuzzy Systems, vol. 12, no. 4, pp. 569–576, 2004.
  28. R. Coulom, “Efficient selectivity and backup operators in monte-carlo tree search,” in International conference on computers and games.   Springer, 2006, pp. 72–83.
  29. U. Technologies, “Unity technologies karting microgame template,” https://assetstore.unity.com/packages/templates/karting-microgame-150956, 2021, accessed: 10-2021.
  30. A. Juliani, V.-P. Berges, E. Teng, A. Cohen, J. Harper, C. Elion, C. Goy, Y. Gao, H. Henry, M. Mattar et al., “Unity: A general platform for intelligent agents,” arXiv preprint arXiv:1809.02627, 2018.
  31. A. Cohen, E. Teng, V.-P. Berges, R.-P. Dong, H. Henry, M. Mattar, A. Zook, and S. Ganguly, “On the use and misuse of absorbing states in multi-agent reinforcement learning,” arXiv preprint arXiv:2111.05992, 2021.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now