Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Simulation-based reinforcement learning for real-world autonomous driving (1911.12905v4)

Published 29 Nov 2019 in cs.LG, cs.AI, and cs.RO

Abstract: We use reinforcement learning in simulation to obtain a driving system controlling a full-size real-world vehicle. The driving policy takes RGB images from a single camera and their semantic segmentation as input. We use mostly synthetic data, with labelled real-world data appearing only in the training of the segmentation network. Using reinforcement learning in simulation and synthetic data is motivated by lowering costs and engineering effort. In real-world experiments we confirm that we achieved successful sim-to-real policy transfer. Based on the extensive evaluation, we analyze how design decisions about perception, control, and training impact the real-world performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst. In Robotics: Science and Systems XV, University of Freiburg, Freiburg im Breisgau, Germany, June 22-26, 2019., 2019.
  2. Learning to drive from simulation without real world labels. In International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, Canada, May 20-24, 2019, pages 4818–4824, 2019.
  3. Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, pages 4243–4250, 2018.
  4. Model-free deep reinforcement learning for urban autonomous driving. CoRR, abs/1904.09503, 2019.
  5. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1724–1734, 2014.
  6. Quantifying generalization in reinforcement learning. In International Conference on Machine Learning, pages 1282–1289, 2019.
  7. On offline evaluation of vision-based driving models. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XV, pages 246–262, 2018.
  8. End-to-end driving via conditional imitation learning. CoRR, abs/1710.02410, 2017.
  9. A survey on policy search for robotics. Foundations and Trends in Robotics, 2(1-2):1–142, 2013.
  10. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017.
  11. OpenAI Baselines. https://github.com/openai/baselines, 2017.
  12. Motion prediction of traffic actors for autonomous driving using deep convolutional networks. CoRR, abs/1808.05819, 2018.
  13. Carla: An open urban driving simulator. arXiv oloreprint arXiv:1711.03938, 2017.
  14. IMPALA: scalable distributed deep-rl with importance weighted actor-learner architectures. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, pages 1406–1415, 2018.
  15. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, pages 1856–1865, 2018.
  16. Emergence of locomotion behaviours in rich environments. CoRR, abs/1707.02286, 2017.
  17. Reinforcement learning with unsupervised auxiliary tasks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
  18. Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task. CoRR, abs/1707.02267, 2017.
  19. Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 12627–12637, 2019.
  20. Model-based reinforcement learning for atari. CoRR, abs/1903.00374, 2019.
  21. Generalization through simulation: Integrating simulated and real data into deep reinforcement learning for vision-based autonomous flight. In International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, Canada, May 20-24, 2019, pages 6008–6014, 2019.
  22. Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. In British Machine Vision Conference 2017, BMVC 2017, London, UK, September 4-7, 2017, 2017.
  23. Learning to run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments. In Sergio Escalera and Markus Weimer, editors, The NIPS ’17 Competition: Building Intelligent Systems, pages 121–153, Cham, 2018. Springer International Publishing.
  24. Plan online, learn offline: Efficient learning and exploration via model-based control. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
  25. Driving policy transfer via modularity and abstraction. In 2nd Annual Conference on Robot Learning, CoRL 2018, Zürich, Switzerland, 29-31 October 2018, Proceedings, pages 1–15, 2018.
  26. The mapillary vistas dataset for semantic understanding of street scenes. In International Conference on Computer Vision (ICCV), 2017.
  27. Learning dexterous in-hand manipulation. arXiv preprint arXiv:1808.00177, 2018.
  28. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, pages 1–8, 2018.
  29. Asymmetric actor critic for image-based robot learning. In Robotics: Science and Systems XIV, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, June 26-30, 2018, 2018.
  30. Dean Pomerleau. ALVINN: an autonomous land vehicle in a neural network. In Advances in Neural Information Processing Systems 1, [NIPS Conference, Denver, Colorado, USA, 1988], pages 305–313, 1988.
  31. Deep imitative models for flexible inference, planning, and control. CoRR, abs/1810.06544, 2018.
  32. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 - 9, 2015, Proceedings, Part III, pages 234–241, 2015.
  33. Sim-to-real robot learning from pixels with progressive nets. In 1st Annual Conference on Robot Learning, CoRL 2017, Mountain View, California, USA, November 13-15, 2017, Proceedings, pages 262–270, 2017.
  34. Fereshteh Sadeghi. Divis: Domain invariant visual servoing for collision-free goal reaching. In Robotics: Science and Systems XV, University of Freiburg, Freiburg im Breisgau, Germany, June 22-26, 2019., 2019.
  35. CAD2RL: real single-image flight without a single real image. In Robotics: Science and Systems XIII, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA, July 12-16, 2017, 2017.
  36. Sim2real viewpoint invariant visual servoing by recurrent control. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 4691–4699, 2018.
  37. Deep reinforcement learning framework for autonomous driving. Electronic Imaging, 2017(19):70–76, Jan 2017.
  38. Conditional affordance learning for driving in urban environments. In 2nd Annual Conference on Robot Learning, CoRL 2018, Zürich, Switzerland, 29-31 October 2018, Proceedings, pages 237–252, 2018.
  39. Horovod: fast and easy distributed deep learning in tensorflow. arXiv preprint arXiv:1802.05799, 2018.
  40. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and Service Robotics, Results of the 11th International Conference, FSR 2017, Zurich, Switzerland, 12-15 September 2017, pages 621–635, 2017.
  41. The limits and potentials of deep learning for robotics. I. J. Robotics Res., 37(4-5):405–420, 2018.
  42. Autonomous driving in reality with reinforcement learning and image translation. CoRR, abs/1801.05299, 2018.
  43. Sim-to-real: Learning agile locomotion for quadruped robots. In Tom Howard Hadas Kress-Gazit, Siddhartha Srinivasa and Nikolay Atanasov, editors, Robotics: Science and System XIV, 2018.
  44. Domain randomization and generative models for robotic grasping. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, Madrid, Spain, October 1-5, 2018, pages 3482–3489, 2018.
  45. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012, Vilamoura, Algarve, Portugal, October 7-12, 2012, pages 5026–5033, 2012.
Citations (113)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets