Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Quadruped-Frog: Rapid Online Optimization of Continuous Quadruped Jumping (2403.06954v1)

Published 11 Mar 2024 in cs.RO

Abstract: Legged robots are becoming increasingly agile in exhibiting dynamic behaviors such as running and jumping. Usually, such behaviors are either optimized and engineered offline (i.e. the behavior is designed for before it is needed), either through model-based trajectory optimization, or through deep learning-based methods involving millions of timesteps of simulation interactions. Notably, such offline-designed locomotion controllers cannot perfectly model the true dynamics of the system, such as the motor dynamics. In contrast, in this paper, we consider a quadruped jumping task that we rapidly optimize online. We design foot force profiles parameterized by only a few parameters which we optimize for directly on hardware with Bayesian Optimization. The force profiles are tracked at the joint level, and added to Cartesian PD impedance control and Virtual Model Control to stabilize the jumping motions. After optimization, which takes only a handful of jumps, we show that this control architecture is capable of diverse and omnidirectional jumps including forward, lateral, and twist (turning) jumps, even on uneven terrain, enabling the Unitree Go1 quadruped to jump 0.5 m high, 0.5 m forward, and jump-turn over 2 rad. Video results can be found at https://youtu.be/SvfVNQ90k_w.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. C. D. Fitzgibbon and J. H. Fanshawe, “Stotting in thomson’s gazelles: an honest signal of condition,” Behavioral Ecology and Sociobiology, vol. 23, pp. 69–74, 1988.
  2. T. Caro, “The functions of stotting: a review of the hypotheses,” Animal Behaviour, vol. 34, no. 3, pp. 649–662, 1986.
  3. N. Dominici, Y. P. Ivanenko, G. Cappellini, A. d’Avella, V. Mondì, M. Cicchese, A. Fabiano, T. Silei, A. D. Paolo, C. Giannini, R. E. Poppele, and F. Lacquaniti, “Locomotor primitives in newborn babies and their development,” Science, vol. 334, no. 6058, pp. 997–999, 2011.
  4. M. Garwicz, M. Christensson, and E. Psouni, “A unifying model for timing of walking onset in humans and other mammals,” Proceedings of the National Academy of Sciences, vol. 106, no. 51, pp. 21 889–21 893, 2009.
  5. A. J. Ijspeert, “Central pattern generators for locomotion control in animals and robots: A review,” Neural Networks, vol. 21, no. 4, pp. 642–653, 2008, robotics and Neuroscience.
  6. Q. Nguyen, M. J. Powell, B. Katz, J. D. Carlo, and S. Kim, “Optimized jumping on the mit cheetah 3 robot,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 7448–7454.
  7. C. V. Nguyen and Q. Nguyen, “Contact-timing and trajectory optimization for 3d jumping on quadruped robots,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 11 994–11 999.
  8. B. Katz, J. Di Carlo, and S. Kim, “Mini cheetah: A platform for pushing the limits of dynamic quadruped control,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 6295–6301.
  9. M. Chignoli, S. Morozov, and S. Kim, “Rapid and reliable quadruped motion planning with omnidirectional jumping,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 6621–6627.
  10. M. Chignoli and S. Kim, “Online trajectory optimization for dynamic aerial motions of a quadruped robot,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 7693–7699.
  11. C. V. Nguyen, L. Bao, and Q. Nguyen, “Continuous jumping for legged robots on stepping stones via trajectory optimization and model predictive control,” in 2022 IEEE 61st Conference on Decision and Control (CDC), 2022, pp. 93–99.
  12. J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim, “Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 1–9.
  13. D. Kim, J. Di Carlo, B. Katz, G. Bledt, and S. Kim, “Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control,” arXiv preprint arXiv:1909.06586, 2019.
  14. M. Sombolestan, Y. Chen, and Q. Nguyen, “Adaptive force-based control for legged robots,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021, pp. 7440–7447.
  15. C. D. Bellicoso, F. Jenelten, C. Gehring, and M. Hutter, “Dynamic locomotion through online nonlinear motion optimization for quadrupedal robots,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2261–2268, 2018.
  16. G. Ji, J. Mun, H. Kim, and J. Hwangbo, “Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4630–4637, 2022.
  17. G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal, “Rapid locomotion via reinforcement learning,” arXiv preprint arXiv:2205.02824, 2022.
  18. G. Bellegarda, Y. Chen, Z. Liu, and Q. Nguyen, “Robust high-speed running for quadruped robots via deep reinforcement learning,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 10 364–10 370.
  19. N. Rudin, D. Hoeller, M. Bjelonic, and M. Hutter, “Advanced skills by learning locomotion and local navigation end-to-end,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 2497–2503.
  20. T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,” Science Robotics, 2022.
  21. Z. Li, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Robust and Versatile Bipedal Jumping Control through Reinforcement Learning,” in Proceedings of Robotics: Science and Systems, Daegu, Republic of Korea, July 2023.
  22. L. Smith, J. C. Kew, T. Li, L. Luu, X. B. Peng, S. Ha, J. Tan, and S. Levine, “Learning and adapting agile locomotion skills by transferring experience,” arXiv preprint arXiv:2304.09834, 2023.
  23. G. Bellegarda and K. Byl, “An online training method for augmenting mpc with deep reinforcement learning,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 5453–5459.
  24. G. Bellegarda and K. Byl, “Training in task space to speed up and guide reinforcement learning,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2019, pp. 2693–2699.
  25. G. Bellegarda and Q. Nguyen, “Robust quadruped jumping via deep reinforcement learning,” arXiv preprint arXiv:2011.07089, 2020.
  26. G. B. Margolis, T. Chen, K. Paigwar, X. Fu, D. Kim, S. bae Kim, and P. Agrawal, “Learning to jump from pixels,” in 5th Annual Conference on Robot Learning, 2021.
  27. W. Yu, D. Jain, A. Escontrela, A. Iscen, P. Xu, E. Coumans, S. Ha, J. Tan, and T. Zhang, “Visual-locomotion: Learning to walk on complex terrains with vision,” in 5th Annual Conference on Robot Learning, 2021.
  28. K.-H. Lee, O. Nachum, T. Zhang, S. Guadarrama, J. Tan, and W. Yu, “Pi-ars: Accelerating evolution-learned visual-locomotion with predictive information representations,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 1447–1454.
  29. Y. Yang, G. Shi, X. Meng, W. Yu, T. Zhang, J. Tan, and B. Boots, “Cajun: Continuous adaptive jumping using a learned centroidal controller,” arXiv preprint arXiv:2306.09557, 2023.
  30. Y. Yang, X. Meng, W. Yu, T. Zhang, J. Tan, and B. Boots, “Continuous versatile jumping using learned action residuals,” in Proceedings of The 5th Annual Learning for Dynamics and Control Conference, ser. Proceedings of Machine Learning Research, N. Matni, M. Morari, and G. J. Pappas, Eds., vol. 211.   PMLR, 15–16 Jun 2023, pp. 770–782.
  31. G. Bellegarda and A. Ijspeert, “CPG-RL: Learning central pattern generators for quadruped locomotion,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 12 547–12 554, 2022.
  32. G. Bellegarda and A. Ijspeert, “Visual CPG-RL: Learning central pattern generators for visually-guided quadruped navigation,” arXiv preprint arXiv:2212.14400, 2022.
  33. M. Shafiee, G. Bellegarda, and A. Ijspeert, “Manyquadrupeds: Learning a single locomotion policy for diverse quadruped robots,” arXiv preprint arXiv:2310.10486, 2023.
  34. M. Shafiee, G. Bellegarda, and A. Ijspeert, “Puppeteer and marionette: Learning anticipatory quadrupedal locomotion based on interactions of a central pattern generator and supraspinal drive,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 1112–1119.
  35. M. Shafiee, G. Bellegarda, and A. Ijspeert, “Deeptransition: Viability leads to the emergence of gait transitions in learning anticipatory quadrupedal locomotion skills,” arXiv preprint arXiv:2306.07419, 2023.
  36. L. Smith, J. C. Kew, X. B. Peng, S. Ha, J. Tan, and S. Levine, “Legged robots that keep on learning: Fine-tuning locomotion policies in the real world,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 1593–1599.
  37. A. Rai, R. Antonova, F. Meier, and C. G. Atkeson, “Using simulation to improve sample-efficiency of bayesian optimization for bipedal robots,” The Journal of Machine Learning Research, vol. 20, no. 1, pp. 1844–1867, 2019.
  38. A. Rai, R. Antonova, S. Song, W. Martin, H. Geyer, and C. Atkeson, “Bayesian optimization using domain knowledge on the atrias biped,” in 2018 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2018, pp. 1771–1778.
  39. R. Calandra, A. Seyfarth, J. Peters, and M. P. Deisenroth, “An experimental comparison of bayesian optimization for bipedal locomotion,” in 2014 IEEE international conference on robotics and automation (ICRA).   IEEE, 2014, pp. 1951–1958.
  40. R. Calandra, A. Seyfarth, J. Peters, and M. P. Deisenroth, “Bayesian optimization for learning gaits under uncertainty: An experimental comparison on a dynamic bipedal walker,” Annals of Mathematics and Artificial Intelligence, vol. 76, pp. 5–23, 2016.
  41. F. Ruppert and A. Badri-Spröwitz, “Learning plastic matching of robot dynamics in closed-loop central pattern generators,” Nature Machine Intelligence, vol. 4, no. 7, pp. 652–660, 2022.
  42. D. Widmer, D. Kang, B. Sukhija, J. Hübotter, A. Krause, and S. Coros, “Tuning legged locomotion controllers via safe bayesian optimization,” arXiv preprint arXiv:2306.07092, 2023.
  43. S. Nauwelaerts and P. Aerts, “Take-off and landing forces in jumping frogs,” Journal of Experimental Biology, vol. 209, no. 1, pp. 66–77, 2006.
  44. K. Matsuoka, “Mechanisms of frequency and pattern control in the neural rhythm generators,” Biological cybernetics, vol. 56, no. 5, pp. 345–353, 1987.
  45. L. Righetti and A. J. Ijspeert, “Pattern generators with sensory feedback for the control of quadruped locomotion,” in IEEE International Conference on Robotics and Automation, 2008, pp. 819–824.
  46. A. J. Ijspeert, A. Crespi, D. Ryczko, and J.-M. Cabelguen, “From swimming to walking with a salamander robot driven by a spinal cord model,” Science, vol. 315, no. 5817, pp. 1416–1420, 2007.
  47. A. Spröwitz, A. Tuleu, M. Vespignani, M. Ajallooeian, E. Badri, and A. J. Ijspeert, “Towards dynamic trot gait locomotion: Design, control, and experiments with cheetah-cub, a compliant quadruped robot,” The International Journal of Robotics Research, vol. 32, no. 8, pp. 932–950, 2013.
  48. M. Ajallooeian, S. Pouya, A. Sproewitz, and A. J. Ijspeert, “Central pattern generators augmented with virtual model control for quadruped rough terrain locomotion,” in 2013 IEEE International Conference on Robotics and Automation, 2013, pp. 3321–3328.
  49. M. Ajallooeian, S. Gay, A. Tuleu, A. Spröwitz, and A. J. Ijspeert, “Modular control of limit cycle locomotion over unperceived rough terrain,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 3390–3397.
  50. T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
  51. Unitree Robotics. Go1. https://www.unitree.com/products/go1/.
  52. J. Fan, Q. Du, Z. Dong, J. Zhao, and T. Xu, “Design of the jump mechanism for a biomimetic robotic frog,” Biomimetics, vol. 7, no. 4, p. 142, 2022.
  53. S. H. Jeon, S. Kim, and D. Kim, “Online optimal landing control of the mit mini cheetah,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 178–184.
  54. F. Roscia, M. Focchi, A. Del Prete, D. G. Caldwell, and C. Semini, “Reactive landing controller for quadruped robots,” arXiv preprint arXiv:2305.07748, 2023.
  55. M. S. Ashtiani, A. Aghamaleki Sarvestani, and A. Badri-Spröwitz, “Hybrid parallel compliance allows robots to operate with sensorimotor delays and low control frequencies,” Frontiers in Robotics and AI, vol. 8, p. 645748, 2021.
  56. S. Cox, L. Ekstrom, and G. Gillis, “The influence of visual, vestibular, and hindlimb proprioceptive ablations on landing preparation in cane toads,” Integrative and Comparative Biology, vol. 58, no. 5, pp. 894–905, 2018.
Citations (6)
View on Semantic Scholar

Summary

  • The paper introduces an online Bayesian optimization method that rapidly converges on effective force profiles for quadruped jumping in about 20 trials.
  • It integrates Cartesian PD impedance control and Virtual Model Control to execute multidirectional jumps, including lateral and twist maneuvers on uneven terrain.
  • Experimental results demonstrate robust performance with jumps achieving 0.5 m height and lateral rotations up to 2 radians.

Online Optimization of Quadruped Jumping Control

The paper "Quadruped-Frog: Rapid Online Optimization of Continuous Quadruped Jumping" by Guillaume Bellegarda et al. presents a novel approach to designing and optimizing jumping behaviors in quadruped robots using online fast-adapting control strategies. The methodology involves Bayesian Optimization for parameterizing and fine-tuning force profiles that are executed directly on the hardware. This paper discusses the integration of these optimized force profiles with existing control architectures such as Cartesian Proportional-Derivative (PD) impedance control and Virtual Model Control (VMC) to achieve not just forward jumping but lateral and twist (turning) jumps as well, even across uneven terrain.

Research Context and Methodology

Traditional approaches in legged robotics frequently rely on offline optimization techniques, employing either model-based trajectory optimization or large-scale deep learning simulations to generate locomotive behaviors like jumping. These methods often fall short in capturing the complexities of real-world dynamics such as motor characteristics and frictional interactions. The paper eschews these constraints by facilitating online optimization directly via hardware, employing Bayesian Optimization as a principal technique. This optimization is based on the measurement and adjustment of a minimal set of parameters governing the force profiles at the quadruped's feet.

The jumping behavior is parameterized as a set of desired foot force profiles. These are adjusted through Bayesian Optimization to improve jumping performance. This optimization cycle occurs rapidly, needing only a limited number of attempts to converge on an effective jumping strategy. Importantly, the optimization utilizes Tree-Parzen Estimator (TPE) for refining parameters such as the frequency and magnitude of foot-ground impact forces.

Key Results

The experimental findings demonstrate the proposed control architecture's ability to achieve various jumping maneuvers, including omnidirectional jumps. Notable results are highlighted:

  • The quadruped robot, Unitree Go1, accomplished jumps acknowledging heights up to 0.5 m and forward advances of 0.5 m.
  • Lateral (sideway) jumps and twist jumps reaching up to 2 radians were realized.
  • The system displayed robustness to external environmental challenges, such as uneven terrains due to the stabilizing effects of Virtual Model Control.
  • Initial hardware validation revealed compelling evidence for rapid parameter convergence in about 20 trials.

Implications and Future Directions

The implications of this research are twofold. Practically, this work suggests a pathway for robots to dynamically adapt their locomotion abilities in situ, facilitating robust movement across varied real-world scenarios with minimal prior tuning. Theoretically, the successful use of Bayesian Optimization posits a broader applicability in autonomous robotic systems for nuanced and complex behavior adaption without resorting to extensive computational pre-training or dynamics modeling.

Future research could explore more advanced whole-body controllers that may further enhance stability and adaptability during jumping. Integrating deep reinforcement learning is another potential avenue, permitting the modulation of joint-level actions based on the learned action residuals. Additionally, studying the synergies between model-based and learning-based controllers could unlock further performance improvements.

This paper becomes a relevant reference point as the robotics community continues to balance the trade-offs between optimization efficiency, computational cost, and practical application. The demonstrated effectiveness of online optimization approaches like this one could inspire similar techniques in other areas of robotics and dynamic motion planning.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube