Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections (2403.13674v2)

Published 20 Mar 2024 in cs.RO

Abstract: In this work, we present a reward-driven automated curriculum reinforcement learning approach for interaction-aware self-driving at unsignalized intersections, taking into account the uncertainties associated with surrounding vehicles (SVs). These uncertainties encompass the uncertainty of SVs' driving intention and also the quantity of SVs. To deal with this problem, the curriculum set is specifically designed to accommodate a progressively increasing number of SVs. By implementing an automated curriculum selection mechanism, the importance weights are rationally allocated across various curricula, thereby facilitating improved sample efficiency and training outcomes. Furthermore, the reward function is meticulously designed to guide the agent towards effective policy exploration. Thus the proposed framework could proactively address the above uncertainties at unsignalized intersections by employing the automated curriculum learning technique that progressively increases task difficulty, and this ensures safe self-driving through effective interaction with SVs. Comparative experiments are conducted in $Highway_Env$, and the results indicate that our approach achieves the highest task success rate, attains strong robustness to initialization parameters of the curriculum selection module, and exhibits superior adaptability to diverse situational configurations at unsignalized intersections. Furthermore, the effectiveness of the proposed method is validated using the high-fidelity CARLA simulator.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. B. Paden, M. Čáp, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Transactions on Intelligent Vehicles, vol. 1, no. 1, pp. 33–55, 2016.
  2. K. Muhammad, A. Ullah, J. Lloret, J. Del Ser, and V. H. C. de Albuquerque, “Deep learning for safe autonomous driving: Current challenges and future directions,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 7, pp. 4316–4336, 2020.
  3. S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving,” Journal of Field Robotics, vol. 37, no. 3, pp. 362–386, 2020.
  4. S. Mozaffari, O. Y. Al-Jarrah, M. Dianati, P. Jennings, and A. Mouzakitis, “Deep learning-based vehicle behavior prediction for autonomous driving applications: A review,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 33–47, 2020.
  5. F. Li, X. Li, J. Luo, S. Fan, and H. Zhang, “Open-set intersection intention prediction for autonomous driving,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 13092–13098, IEEE, 2021.
  6. L. Zheng, R. Yang, Z. Peng, M. Y. Wang, and J. Ma, “Spatiotemporal receding horizon control with proactive interaction towards autonomous driving in dense traffic,” arXiv preprint arXiv:2308.05929, 2023.
  7. L. Wei, Z. Li, J. Gong, C. Gong, and J. Li, “Autonomous driving strategies at intersections: Scenarios, state-of-the-art, and future outlooks,” in Proceedings of the IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 44–51, IEEE, 2021.
  8. A. Aksjonov and V. Kyrki, “Rule-based decision-making system for autonomous vehicles at intersections with mixed traffic environment,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 660–666, IEEE, 2021.
  9. R. Tian, N. Li, I. Kolmanovsky, Y. Yildiz, and A. R. Girard, “Game-theoretic modeling of traffic in unsignalized intersection network for autonomous vehicle control verification and validation,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 3, pp. 2211–2226, 2020.
  10. L. Riegger, M. Carlander, N. Lidander, N. Murgovski, and J. Sjöberg, “Centralized MPC for autonomous intersection crossing,” in Proceedings of the IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 1372–1377, IEEE, 2016.
  11. M. Kneissl, A. Molin, H. Esen, and S. Hirche, “A feasible MPC-based negotiation algorithm for automated intersection crossing,” in 2018 European Control Conference (ECC), pp. 1282–1288, IEEE, 2018.
  12. D. Isele, R. Rahimi, A. Cosgun, K. Subramanian, and K. Fujimura, “Navigating occluded intersections with autonomous vehicles using deep reinforcement learning,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2034–2039, IEEE, 2018.
  13. B. R. Kiran, I. Sobh, V. Talpaert, P. Mannion, A. A. Al Sallab, S. Yogamani, and P. Pérez, “Deep reinforcement learning for autonomous driving: A survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 4909–4926, 2021.
  14. C. Xia, M. Xing, and S. He, “Interactive planning for autonomous driving in intersection scenarios without traffic signs,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 24818–24828, 2022.
  15. Z. Qiao, J. Schneider, and J. M. Dolan, “Behavior planning at urban intersections through hierarchical reinforcement learning,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2667–2673, IEEE, 2021.
  16. Y. Huang, S. Yang, L. Wang, K. Yuan, H. Zheng, and H. Chen, “An efficient self-evolution method of autonomous driving for any given algorithm,” IEEE Transactions on Intelligent Transportation Systems, 2023.
  17. Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in Proceedings of the 26th International Conference on Machine Learning (ICML), pp. 41–48, 2009.
  18. Y. Song, H. Lin, E. Kaufmann, P. Dürr, and D. Scaramuzza, “Autonomous overtaking in gran turismo sport using curriculum reinforcement learning,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 9403–9409, IEEE, 2021.
  19. Y. Wang, Y. Li, Z. Peng, H. Ghazzai, and J. Ma, “Chance-aware lane change with high-level model predictive control through curriculum reinforcement learning,” 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024.
  20. Z. Peng, X. Zhou, Y. Wang, L. Zheng, M. Liu, and J. Ma, “Curriculum proximal policy optimization with stage-decaying clipping for self-driving at unsignalized intersections,” in Proceedings of the IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 5027–5033, IEEE, 2023.
  21. A. Graves, M. G. Bellemare, J. Menick, R. Munos, and K. Kavukcuoglu, “Automated curriculum learning for neural networks,” in Proceedings of the 34th International Conference on Machine Learning (ICML), pp. 1311–1320, 2017.
  22. Z. Qiao, K. Muelling, J. M. Dolan, P. Palanisamy, and P. Mudalige, “Automatically generated curriculum based reinforcement learning for autonomous vehicles in urban environment,” in Proceedings of the IEEE Intelligent Vehicles Symposium (IV), pp. 1233–1238, IEEE, 2018.
  23. S. Khaitan and J. M. Dolan, “State dropout-based curriculum reinforcement learning for self-driving at unsignalized intersections,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 12219–12224, IEEE, 2022.
  24. A. Slivkins et al., “Introduction to multi-armed bandits,” Foundations and Trends® in Machine Learning, vol. 12, no. 1-2, pp. 1–286, 2019.
  25. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire, “The nonstochastic multiarmed bandit problem,” SIAM Journal on Computing, vol. 32, no. 1, pp. 48–77, 2002.
  26. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
  27. E. Leurent, “An environment for autonomous driving decision-making.” https://github.com/eleurent/highway-env, 2018.
  28. M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,” Physical Review E, vol. 62, no. 2, p. 1805, 2000.
  29. A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Proceedings of the Conference on Robot Learning, pp. 1–16, PMLR, 2017.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now