Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dimension-variable Mapless Navigation with Deep Reinforcement Learning (2002.06320v3)

Published 15 Feb 2020 in cs.RO

Abstract: Deep reinforcement learning (DRL) has exhibited considerable promise in the training of control agents for mapless robot navigation. However, DRL-trained agents are limited to the specific robot dimensions used during training, hindering their applicability when the robot's dimension changes for task-specific requirements. To overcome this limitation, we propose a dimension-variable robot navigation method based on DRL. Our approach involves training a meta agent in simulation and subsequently transferring the meta skill to a dimension-varied robot using a technique called dimension-variable skill transfer (DVST). During the training phase, the meta agent for the meta robot learns self-navigation skills with DRL. In the skill-transfer phase, observations from the dimension-varied robot are scaled and transferred to the meta agent, and the resulting control policy is scaled back to the dimension-varied robot. Through extensive simulated and real-world experiments, we demonstrated that the dimension-varied robots could successfully navigate in unknown and dynamic environments without any retraining. The results show that our work substantially expands the applicability of DRL-based navigation methods, enabling them to be used on robots with different dimensions without the limitation of a fixed dimension. The video of our experiments can be found in the supplementary file.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Y. Yue, C. Zhao, Z. Wu, C. Yang, Y. Wang, and D. Wang, “Collaborative semantic understanding and mapping framework for autonomous systems,” IEEE/ASME Trans. Mechatronics, vol. 26, no. 2, pp. 978–989, 2021.
  2. Q. Zou, Q. Sun, L. Chen, B. Nie, and Q. Li, “A comparative analysis of lidar slam-based indoor navigation for autonomous vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 6907–6921, 2022.
  3. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015.
  4. H.-C. Wang, S.-C. Huang, P.-J. Huang, K.-L. Wang, Y.-C. Teng, Y.-T. Ko, D. Jeon, and I.-C. Wu, “Curriculum reinforcement learning from avoiding collisions to navigating among movable obstacles in diverse environments,” IEEE Robot. Autom. Lett., vol. 8, no. 5, pp. 2740–2747, 2023.
  5. B. Li, Z. Huang, T. W. Chen, T. Dai, Y. Zang, W. Xie, B. Tian, and K. Cai, “Msn: Mapless short-range navigation based on time critical deep reinforcement learning,” IEEE Trans. Intell. Transp. Syst., pp. 1–10, 2022.
  6. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
  7. Z. Zhou, Z. Zeng, L. Lang, W. Yao, H. Lu, Z. Zheng, and Z. Zhou, “Navigating robots in dynamic environment with deep reinforcement learning,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 12, pp. 25 201–25 211, 2022.
  8. J. Lim, S. Ha, and J. Choi, “Prediction of reward functions for deep reinforcement learning via gaussian process regression,” IEEE/ASME Trans. Mechatronics, vol. 25, no. 4, pp. 1739–1746, 2020.
  9. K. Wu, H. Wang, M. A. Esfahani, and S. Yuan, “Learn to navigate autonomously through deep reinforcement learning,” IEEE Trans. Ind. Electron., vol. 69, no. 5, pp. 5342–5352, 2022.
  10. L. Tai, G. Paolo, and M. Liu, “Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation,” in IEEE Int. Conf. Intell. Robots Syst., vol. 2017-Septe, 2017, pp. 31–36.
  11. H. Shi, L. Shi, M. Xu, and K.-S. Hwang, “End-to-end navigation strategy with deep reinforcement learning for mobile robots,” IEEE Trans. Ind. Informat., vol. 16, no. 4, pp. 2393–2402, 2020.
  12. L. Xie, Y. Miao, S. Wang, P. Blunsom, Z. Wang, C. Chen, A. Markham, and N. Trigoni, “Learning with stochastic guidance for robot navigation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 1, pp. 166–176, 2021.
  13. K. Rana, V. Dasagi, J. Haviland, B. Talbot, M. Milford, and N. Sünderhauf, “Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics,” Int. J. Rob. Res., vol. 42, no. 3, pp. 123–146, 2023.
  14. Y. Jang, J. Baek, and S. Han, “Hindsight intermediate targets for mapless navigation with deep reinforcement learning,” IEEE Trans. Ind. Electron., vol. 69, no. 11, pp. 11 816–11 825, 2022.
  15. F. Leiva and J. Ruiz-del Solar, “Robust rl-based map-less local planning: Using 2d point clouds as observations,” IEEE Robot. Autom. Lett., vol. 5, no. 4, pp. 5787–5794, 2020.
  16. M. Pfeiffer, S. Shukla, M. Turchetta, C. Cadena, A. Krause, R. Siegwart, and J. Nieto, “Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior Demonstrations,” IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 4423–4430, 2018.
  17. W. Yu, J. Tan, Y. Bai, E. Coumans, and S. Ha, “Learning fast adaptation with meta strategy optimization,” IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 2950–2957, 2020.
  18. A. Ghadirzadeh, X. Chen, P. Poklukar, C. Finn, M. Björkman, and D. Kragic, “Bayesian meta-learning for few-shot policy adaptation across robotic platforms,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021, pp. 1274–1280.
  19. A. Loquercio, E. Kaufmann, R. Ranftl, A. Dosovitskiy, V. Koltun, and D. Scaramuzza, “Deep drone racing: From simulation to reality with domain randomization,” IEEE Trans. Robot., vol. 36, no. 1, pp. 1–14, 2020.
  20. A. Ishfaque and B. Kim, “Real-time sound source localization in robots using fly ormia ochracea inspired mems directional microphone,” IEEE Sens. Lett., vol. 7, no. 1, pp. 1–4, 2023.
  21. J. Ding, Y. Wang, H. Si, S. Gao, and J. Xing, “Three-dimensional indoor localization and tracking for mobile target based on wifi sensing,” IEEE Internet Things J., vol. 9, no. 21, pp. 21 687–21 701, 2022.
  22. T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in 35th International Conference on Machine Learning, ICML 2018, vol. 5, 2018, pp. 2976–2989.
  23. J. C. de Jesus, V. A. Kich, A. H. Kolling, R. B. Grando, M. A. d. S. L. Cuadros, and D. F. T. Gamarra, “Soft actor-critic for navigation of mobile robots,” J. Intell. Robot. Syst., vol. 102, no. 2, p. 31, 2021.
  24. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
  25. W. Zhang, Y. Zhang, N. Liu, K. Ren, and P. Wang, “IPAPRec: A promising tool for learning high-performance mapless navigation skills with deep reinforcement learning,” IEEE/ASME Trans. Mechatronics, vol. 27, no. 6, pp. 5451–5461, 2022.
  26. Z. Liu, X. Li, B. Kang, and T. Darrell, “Regularization matters in policy optimization - an empirical study on continuous control,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=yr1mzrH3IC
  27. Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in Proceedings of the 26th Annual International Conference on Machine Learning.   Association for Computing Machinery, 2009, p. 41–48.
  28. D. Fox, W. Burgard, and S. Thrun, “The dynamic window approach to collision avoidance,” IEEE Robot. Autom. Mag., vol. 4, no. 1, pp. 23–33, 1997.
  29. R. Vaughan, “Massively multi-robot simulation in stage,” Swarm Intell., vol. 2, pp. 189–208, 2008.
  30. “AMCL.” [Online]. Available: http://wiki.ros.org/amcl
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Wei Zhang (1489 papers)
  2. Yunfeng Zhang (45 papers)
  3. Ning Liu (199 papers)
  4. Kai Ren (18 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now