2000 character limit reached
Learning Quadrupedal Locomotion with Impaired Joints Using Random Joint Masking (2403.00398v1)
Published 1 Mar 2024 in cs.RO
Abstract: Quadrupedal robots have played a crucial role in various environments, from structured environments to complex harsh terrains, thanks to their agile locomotion ability. However, these robots can easily lose their locomotion functionality if damaged by external accidents or internal malfunctions. In this paper, we propose a novel deep reinforcement learning framework to enable a quadrupedal robot to walk with impaired joints. The proposed framework consists of three components: 1) a random joint masking strategy for simulating impaired joint scenarios, 2) a joint state estimator to predict an implicit status of current joint condition based on past observation history, and 3) progressive curriculum learning to allow a single network to conduct both normal gait and various joint-impaired gaits. We verify that our framework enables the Unitree's Go1 robot to walk under various impaired joint conditions in real-world indoor and outdoor environments.
- Advances in real-world applications for legged robots. Journal of Field Robotics, 35(8):1311–1326, 2018.
- Alphred: A multi-modal operations quadruped robot for package delivery applications. IEEE Robotics and Automation Letters, 5(4):5409–5416, 2020.
- Autonomous social distancing in urban environments using a quadruped robot. IEEE Access, 9:8392–8403, 2021.
- Cerberus in the darpa subterranean challenge. Science Robotics, 7(66):eabp9742, 2022.
- Robust rough-terrain locomotion with a quadrupedal robot. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 5761–5768. IEEE, 2018.
- Learning quadrupedal locomotion over challenging terrain. Science robotics, 5(47):eabc5986, 2020.
- Cheap rapid locomotion of a quadruped robot: Self-stabilization of bounding gait. In Intelligent autonomous systems, volume 8, pages 642–649. IOS Press Amsterdam, 2004.
- Robust recovery controller for a quadrupedal robot using deep reinforcement learning. arXiv preprint arXiv:1901.07517, 2019.
- Jung-Min Yang. Kinematic constraints on fault-tolerant gaits for a locked joint failure. Journal of Intelligent and Robotic Systems, 45:323–342, 2006.
- Fault-tolerant motion planning and generation of quadruped robots synthesised by posture optimization and whole body control. Complex & Intelligent Systems, 8(4):2991–3003, 2022.
- Meta-learning for fast adaptive locomotion with uncertainties in environments and robot dynamics. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4568–4575. IEEE, 2021.
- Reinforcement learning with adaptive curriculum dynamics randomization for fault-tolerant robot control. arXiv preprint arXiv:2111.10005, 2021.
- Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, pages 91–100. PMLR, 2022.
- Dynamic locomotion in the mit cheetah 3 through convex model-predictive control. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 1–9. IEEE, 2018.
- Practice makes perfect: An optimization-based approach to controlling agile motions for a quadruped robot. IEEE Robotics & Automation Magazine, 23(1):34–43, 2016.
- Rapid locomotion via reinforcement learning. arXiv preprint arXiv:2205.02824, 2022.
- Learning quadrupedal locomotion on deformable terrain. Science Robotics, 8(74):eade2256, 2023.
- Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, pages 119–126, 2010.
- Leveraging multiple simulators for crossing the reality gap. In 2012 12th international conference on control automation robotics & vision (ICARCV), pages 1113–1119. IEEE, 2012.
- Sim-to-real: Learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804.10332, 2018.
- Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021.
- Learning agile and dynamic motor skills for legged robots. Science Robotics, 4(26):eaau5872, 2019.
- Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 23–30. IEEE, 2017.
- Active domain randomization. In Conference on Robot Learning, pages 1162–1176. PMLR, 2020.
- Fault-tolerant gaits of quadruped robot on a constant-slope terrain. In 2008 IEEE International Conference on Automation, Quality and Testing, Robotics, volume 1, pages 222–226. IEEE, 2008.
- Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
- Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
- Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning, pages 22–31. PMLR, 2023.
- Pseudo-rehearsal: Achieving deep reinforcement learning without catastrophic forgetting. Neurocomputing, 428:291–307, 2021.
- Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017.
- Understanding catastrophic forgetting and remembering in continual learning with optimal relevance mapping. arXiv preprint arXiv:2102.11343, 2021.
- Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021.