Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization (2403.02882v2)

Published 5 Mar 2024 in eess.SY, cs.LG, cs.RO, and cs.SY

Abstract: Most of the current studies on autonomous vehicle decision-making and control tasks based on reinforcement learning are conducted in simulated environments. The training and testing of these studies are carried out under rule-based microscopic traffic flow, with little consideration of migrating them to real or near-real environments to test their performance. It may lead to a degradation in performance when the trained model is tested in more realistic traffic scenes. In this study, we propose a method to randomize the driving style and behavior of surrounding vehicles by randomizing certain parameters of the car-following model and the lane-changing model of rule-based microscopic traffic flow in SUMO. We trained policies with deep reinforcement learning algorithms under the domain randomized rule-based microscopic traffic flow in freeway and merging scenes, and then tested them separately in rule-based microscopic traffic flow and high-fidelity microscopic traffic flow. Results indicate that the policy trained under domain randomization traffic flow has significantly better success rate and calculative reward compared to the models trained under other microscopic traffic flows.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Autonomous cars: The tension between occupant experience and intersection capacity. Transportation Research Part C: Emerging Technologies 2015, 52, 1–14.
  2. Deep reinforcement learning framework for autonomous driving. arXiv preprint arXiv:1704.02532 2017.
  3. Automated speed and lane change decision making using deep reinforcement learning. In Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2018, pp. 2148–2155.
  4. Automated vehicle’s behavior decision making using deep reinforcement learning and high-fidelity simulation environment. Transportation Research Part C: Emerging Technologies 2019, 107, 155–170.
  5. Automated lane change strategy using proximal policy optimization-based deep reinforcement learning. In Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020, pp. 1746–1752.
  6. Anti-jerk on-ramp merging using deep reinforcement learning. In Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020, pp. 7–14.
  7. An Integrated Model for Autonomous Speed and Lane Change Decision-Making Based on Deep Reinforcement Learning. IEEE Transactions on Intelligent Transportation Systems 2022, 23, 21848–21860.
  8. Reinforcement learning for autonomous maneuvering in highway scenarios. In Proceedings of the Workshop for Driving Assistance Systems and Autonomous Driving, 2017, pp. 32–41.
  9. Congested traffic states in empirical observations and microscopic simulations. Physical review E 2000, 62, 1805.
  10. Deep structured reactive planning. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 4897–4904.
  11. TIP: Task-informed motion prediction for intelligent vehicles. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 11432–11439.
  12. Analysis and comparison of microscopic traffic flow models with real traffic microscopic data. Transportation Research Record 2005, 1934, 53–63.
  13. Action robust reinforcement learning and applications in continuous control. In Proceedings of the International Conference on Machine Learning. PMLR, 2019, pp. 6215–6224.
  14. Learning to reinforcement learn. arXiv preprint arXiv:1611.05763 2016.
  15. Learning dexterous in-hand manipulation. The International Journal of Robotics Research 2020, 39, 3–20.
  16. Using data-driven domain randomization to transfer robust control policies to mobile robots. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 3224–3230.
  17. LimSim: A long-term interactive multi-scenario traffic simulator. In Proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2023, pp. 1255–1262.
  18. Microscopic traffic flow simulator VISSIM. Fundamentals of traffic simulation 2010, pp. 63–93.
  19. Traffic simulation with aimsun. Fundamentals of traffic simulation 2010, pp. 173–232.
  20. Microscopic traffic simulation using sumo. In Proceedings of the 2018 21st international conference on intelligent transportation systems (ITSC). IEEE, 2018, pp. 2575–2582.
  21. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Proceedings of the Field and Service Robotics: Results of the 11th International Conference. Springer, 2018, pp. 621–635.
  22. CARLA: An open urban driving simulator. In Proceedings of the Conference on robot learning. PMLR, 2017, pp. 1–16.
  23. Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning. IEEE transactions on pattern analysis and machine intelligence 2022, 45, 3461–3475.
  24. InterSim: Interactive traffic simulation via explicit relation modeling. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 11416–11423.
  25. Trafficgen: Learning to generate diverse and realistic traffic scenarios. In Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 3567–3575.
  26. CitySim: A drone-based vehicle trajectory dataset for safety oriented research and digital twins. arXiv preprint arXiv:2208.11036 2022.
  27. Solving the optimal path planning of a mobile robot using improved Q-learning. Robotics and Autonomous Systems 2019, 115, 143–161.
  28. Multi-layer path planning control for the simulation of manipulation tasks: Involving semantics and topology. Robotics and Computer-Integrated Manufacturing 2019, 57, 17–28.
  29. Analysis and initial observations on varying penetration rates of automated vehicles in mixed traffic flow utilizing sumo. In Proceedings of the 2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE). IEEE, 2019, pp. 1–7.
  30. Recent development and applications of SUMO-Simulation of Urban MObility. International journal on advances in systems and measurements 2012, 5.
  31. TraCI: an interface for coupling road traffic and network simulators. In Proceedings of the Proceedings of the 11th communications and networking simulation symposium, 2008, pp. 155–163.
  32. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905 2018.
  33. Discretionary Lane-Change Decision and Control via Parameterized Soft Actor-Critic for Hybrid Action Space. arXiv preprint arXiv:2402.15790 2024.
  34. Deep reinforcement learning in parameterized action space. arXiv preprint arXiv:1511.04143 2015.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Yuan Lin (55 papers)
  2. Antai Xie (6 papers)
  3. Xiao Liu (402 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets