Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
140 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Self Model for Embodied Intelligence: Modeling Full-Body Human Musculoskeletal System and Locomotion Control with Hierarchical Low-Dimensional Representation (2312.05473v5)

Published 9 Dec 2023 in cs.AI

Abstract: Modeling and control of the human musculoskeletal system is important for understanding human motor functions, developing embodied intelligence, and optimizing human-robot interaction systems. However, current human musculoskeletal models are restricted to a limited range of body parts and often with a reduced number of muscles. There is also a lack of algorithms capable of controlling over 600 muscles to generate reasonable human movements. To fill this gap, we build a musculoskeletal model (MS-Human-700) with 90 body segments, 206 joints, and 700 muscle-tendon units, allowing simulation of full-body dynamics and interaction with various devices. We develop a new algorithm using low-dimensional representation and hierarchical deep reinforcement learning to achieve state-of-the-art full-body control. We validate the effectiveness of our model and algorithm in simulations with real human locomotion data. The musculoskeletal model, along with its control algorithm, will be made available to the research community to promote a deeper understanding of human motion control and better design of interactive robots. Project page: https://lnsgroup.cc/research/MS-Human-700

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. L. H. Ting, S. A. Chvatal, S. A. Safavynia, and J. Lucas McKay, “Review and perspective: neuromechanical considerations for predicting muscle activation patterns for movement,” International journal for numerical methods in biomedical engineering, vol. 28, no. 10, pp. 1003–1014, 2012.
  2. N. Bernstein, “The co-ordination and regulation of movements,” The co-ordination and regulation of movements, 1966.
  3. F. E. Zajac, “Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control.” Critical reviews in biomedical engineering, vol. 17, no. 4, pp. 359–411, 1989.
  4. D. M. Wolpert and Z. Ghahramani, “Computational principles of movement neuroscience,” Nature neuroscience, vol. 3, no. 11, pp. 1212–1217, 2000.
  5. E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ international conference on intelligent robots and systems.   IEEE, 2012, pp. 5026–5033.
  6. S. L. Delp, F. C. Anderson, A. S. Arnold, P. Loan, A. Habib, C. T. John, E. Guendelman, and D. G. Thelen, “Opensim: open-source software to create and analyze dynamic simulations of movement,” IEEE transactions on biomedical engineering, vol. 54, no. 11, pp. 1940–1950, 2007.
  7. S.-H. Lee, E. Sifakis, and D. Terzopoulos, “Comprehensive biomechanical modeling and simulation of the upper body,” ACM Transactions on Graphics (TOG), vol. 28, no. 4, pp. 1–17, 2009.
  8. K. R. Saul, X. Hu, C. M. Goehler, M. E. Vidt, M. Daly, A. Velisar, and W. M. Murray, “Benchmarking of dynamic simulation predictions in two software platforms using an upper limb musculoskeletal model,” Computer methods in biomechanics and biomedical engineering, vol. 18, no. 13, pp. 1445–1458, 2015.
  9. D. C. McFarland, E. M. McCain, M. N. Poppo, and K. R. Saul, “Spatial dependency of glenohumeral joint stability during dynamic unimanual and bimanual pushing and pulling,” Journal of Biomechanical Engineering, vol. 141, no. 5, 2019.
  10. S. L. Delp, J. P. Loan, M. G. Hoy, F. E. Zajac, E. L. Topp, and J. M. Rosen, “An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures,” IEEE Transactions on Biomedical engineering, vol. 37, no. 8, pp. 757–767, 1990.
  11. E. M. Arnold, S. R. Ward, R. L. Lieber, and S. L. Delp, “A model of the lower limb for analysis of human movement,” Annals of biomedical engineering, vol. 38, no. 2, pp. 269–279, 2010.
  12. A. Rajagopal, C. L. Dembia, M. S. DeMers, D. D. Delp, J. L. Hicks, and S. L. Delp, “Full-body musculoskeletal model for muscle-driven simulation of human gait,” IEEE transactions on biomedical engineering, vol. 63, no. 10, pp. 2068–2079, 2016.
  13. Y. Sui, K. ho Kim, and J. W. Burdick, “Quantifying performance of bipedal standing with multi-channel emg,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2017, pp. 3891–3896.
  14. A. K. Lai, A. S. Arnold, and J. M. Wakeling, “Why are antagonist muscles co-activated in my simulation? a musculoskeletal model for analysing human locomotor tasks,” Annals of biomedical engineering, vol. 45, pp. 2762–2774, 2017.
  15. A. G. Bruno, M. L. Bouxsein, and D. E. Anderson, “Development and validation of a musculoskeletal model of the fully articulated thoracolumbar spine and rib cage,” Journal of biomechanical engineering, vol. 137, no. 8, p. 081003, 2015.
  16. S. Schmid, K. A. Burkhart, B. T. Allaire, D. Grindle, and D. E. Anderson, “Musculoskeletal full-body models including a detailed thoracolumbar spine for children and adolescents aged 6–18 years,” Journal of biomechanics, vol. 102, p. 109305, 2020.
  17. S. Lee, M. Park, K. Lee, and J. Lee, “Scalable muscle-actuated human simulation and control,” ACM Transactions On Graphics (TOG), vol. 38, no. 4, pp. 1–13, 2019.
  18. J. Chang, D. Chablat, F. Bennis, and L. Ma, “A full-chain opensim model and its application on posture analysis of an overhead drilling task,” in Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Human Body and Motion: 10th International Conference, DHM 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings, Part I 21.   Springer, 2019, pp. 33–44.
  19. M. Damsgaard, J. Rasmussen, S. T. Christensen, E. Surma, and M. De Zee, “Analysis of musculoskeletal systems in the anybody modeling system,” Simulation Modelling Practice and Theory, vol. 14, no. 8, pp. 1100–1111, 2006.
  20. E. Coumans and Y. Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” 2016.
  21. J. Lee, M. X. Grey, S. Ha, T. Kunz, S. Jain, Y. Ye, S. S. Srinivasa, M. Stilman, and C. Karen Liu, “Dart: Dynamic animation and robotics toolkit,” The Journal of Open Source Software, vol. 3, no. 22, p. 500, 2018.
  22. J. Hwangbo, J. Lee, and M. Hutter, “Per-contact iteration method for solving contact dynamics,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 895–902, 2018.
  23. A. Ikkala and P. Hämäläinen, “Converting biomechanical models from opensim to mujoco,” in International Conference on NeuroRehabilitation.   Springer, 2020, pp. 277–281.
  24. H. Wang, V. Caggiano, G. Durandau, M. Sartori, and V. Kumar, “Myosim: Fast and physiologically realistic mujoco models for musculoskeletal and exoskeletal studies,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 8104–8111.
  25. V. Caggiano, H. Wang, G. Durandau, M. Sartori, and V. Kumar, “Myosuite – a contact-rich simulation suite for musculoskeletal motor control,” https://github.com/MyoHub/myosuite, 2022. [Online]. Available: https://sites.google.com/view/myosuite
  26. J. M. Wang, S. R. Hamner, S. L. Delp, and V. Koltun, “Optimizing locomotion controllers using biologically-based actuators and objectives,” ACM Transactions on Graphics (TOG), vol. 31, no. 4, pp. 1–11, 2012.
  27. Y. Lee, M. S. Park, T. Kwon, and J. Lee, “Locomotion control for many-muscle humanoids,” ACM Trans. Graph., vol. 33, no. 6, nov 2014. [Online]. Available: https://doi.org/10.1145/2661229.2661233
  28. J. Xu, V. Makoviychuk, Y. Narang, F. Ramos, W. Matusik, A. Garg, and M. Macklin, “Accelerated policy learning with parallel differentiable simulation,” 2022. [Online]. Available: https://arxiv.org/abs/2204.07137
  29. S. Song, Ł. Kidziński, X. B. Peng, C. Ong, J. Hicks, S. Levine, C. G. Atkeson, and S. L. Delp, “Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation,” Journal of neuroengineering and rehabilitation, vol. 18, pp. 1–17, 2021.
  30. V. La Barbera, F. Pardo, Y. Tassa, M. Daley, C. Richards, P. Kormushev, and J. Hutchinson, “Ostrichrl: A musculoskeletal ostrich simulation to study bio-mechanical locomotion,” 2021. [Online]. Available: https://arxiv.org/abs/2112.06061
  31. P. Schumacher, D. Häufle, D. Büchler, S. Schmitt, and G. Martius, “Dep-rl: Embodied exploration for reinforcement learning in overactuated and musculoskeletal systems,” 2022. [Online]. Available: https://arxiv.org/abs/2206.00484
  32. J. Weng, E. Hashemi, and A. Arami, “Natural Walking With Musculoskeletal Models Using Deep Reinforcement Learning,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 4156–4162, Apr. 2021, conference Name: IEEE Robotics and Automation Letters.
  33. V. Caggiano, S. Dasari, and V. Kumar, “Myodex: A generalizable prior for dexterous manipulation,” 2023.
  34. R. Cheng, Y. Sui, D. Sayenko, and J. W. Burdick, “Motor control after human sci through activation of muscle synergies under spinal cord stimulation,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 6, pp. 1331–1340, 2019.
  35. C. Berg, V. Caggiano, and V. Kumar, “Sar: Generalization of physiological agility and dexterity via synergistic action representation,” arXiv preprint arXiv:2307.03716, 2023.
  36. P. Walker, J. Rovick, and D. Robertson, “The effects of knee brace hinge design and placement on joint mechanics,” Journal of biomechanics, vol. 21, no. 11, pp. 965–974, 1988.
  37. M. Millard, T. Uchida, A. Seth, and S. L. Delp, “Flexing computational muscle: modeling and simulation of musculotendon dynamics,” Journal of biomechanical engineering, vol. 135, no. 2, p. 021005, 2013.
  38. T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel et al., “Soft actor-critic algorithms and applications,” arXiv preprint arXiv:1812.05905, 2018.
  39. S. A. Chvatal and L. H. Ting, “Common muscle synergies for balance and walking,” Frontiers in computational neuroscience, vol. 7, p. 48, 2013.
  40. L. H. Ting and J. L. McKay, “Neuromechanics of muscle synergies for posture and movement,” Current opinion in neurobiology, vol. 17, no. 6, pp. 622–628, 2007.
  41. A. d’Avella, P. Saltiel, and E. Bizzi, “Combinations of muscle synergies in the construction of a natural motor behavior,” Nature neuroscience, vol. 6, no. 3, pp. 300–308, 2003.
  42. A. J. Izenman, “Introduction to manifold learning,” Wiley Interdisciplinary Reviews: Computational Statistics, vol. 4, no. 5, pp. 439–446, 2012.
  43. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
  44. M. Ringnér, “What is principal component analysis?” Nature biotechnology, vol. 26, no. 3, pp. 303–304, 2008.
  45. J. V. Stone, “Independent component analysis: a tutorial introduction,” 2004.
  46. A. Raffin, A. Hill, M. Ernestus, A. Gleave, A. Kanervisto, and N. Dormann, “Stable baselines3,” 2019.
  47. Y. Jin, X. Liu, Y. Shao, H. Wang, and W. Yang, “High-speed quadrupedal locomotion by imitation-relaxation reinforcement learning,” Nature Machine Intelligence, vol. 4, no. 12, pp. 1198–1208, Dec. 2022, number: 12 Publisher: Nature Publishing Group. [Online]. Available: https://www.nature.com/articles/s42256-022-00576-3
  48. V. Klamroth-Marganska, J. Blanco, K. Campen, A. Curt, V. Dietz, T. Ettlin, M. Felder, B. Fellinghauer, M. Guidali, A. Kollmar, A. Luft, T. Nef, C. Schuster-Amft, W. Stahel, and R. Riener, “Three-dimensional, task-specific robot therapy of the arm after stroke: a multicentre, parallel-group randomised trial,” The Lancet Neurology, vol. 13, no. 2, pp. 159–166, Feb. 2014. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S1474442213703053
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now