Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors (2203.17138v1)

Abstract: We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots. Our approach builds upon previous work on imitating human or dog Motion Capture (MoCap) data to learn a movement skill module. Once learned, this skill module can be reused for complex downstream tasks. Importantly, due to the prior imposed by the MoCap data, our approach does not require extensive reward engineering to produce sensible and natural looking behavior at the time of reuse. This makes it easy to create well-regularized, task-oriented controllers that are suitable for deployment on real robots. We demonstrate how our skill module can be used for imitation, and train controllable walking and ball dribbling policies for both the ANYmal quadruped and OP3 humanoid. These policies are then deployed on hardware via zero-shot simulation-to-reality transfer. Accompanying videos are available at https://bit.ly/robot-npmp.

• The paper presents a novel two-tier control architecture that uses MoCap data to train reusable locomotion skills for robots.
• It demonstrates successful zero-shot imitation and effective real-time control on ANYmal quadruped and OP3 humanoid robots.
• The approach minimizes reward engineering by leveraging neural probabilistic motor primitives derived from human and animal movements.

Overview of the Paper

The paper "Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors" presents a novel methodology for developing locomotion skills in legged robots through the utilization of motion capture (MoCap) data. The research emphasizes the derivation of reusable motor skills from recorded human and animal movement patterns, aiming to reduce the requirement for significant reward engineering, which is often necessary in reinforcement learning (RL) approaches to achieve natural and effective robot movement behaviors.

Methodology

The core of the proposed method involves creating a two-tier control architecture consisting of a high-level encoder and a low-level decoder. These components are trained using neural probabilistic motor primitives (NPMP) techniques alongside MoCap data to form a robust skill module. This module learns to imitate a comprehensive range of pre-recorded movement patterns from either humans or animals, serving as a generic and reusable skill set applicable to various downstream tasks.

The methodological stages include:

1. MoCap Retargeting: MoCap data from humans or animals are retargeted to the target robot models, such as the ANYmal quadruped and OP3 humanoid, using techniques like point-cloud retargeting.
2. Skill Module Training: A goal-conditioned policy is trained using a combination of imitation learning and regularization techniques, focusing on capturing general features of natural movement without being task-specific.
3. Reuse Phase for Task Learning: The pre-trained skill module is then employed as a low-level controller for new RL tasks, such as controllable walking or ball dribbling, thus enhancing the exploration process and reducing the complexity of reward design.

Empirical Evaluation

The proposed approach is validated on two distinct robotic platforms, the ANYmal quadruped and the OP3 humanoid. The experiments focus on both zero-shot imitation and the performance of novel tasks learned using the skill module. The transfer of the learned skills from simulation to real-world deployment is also successfully demonstrated, underscoring the practical applicability of the method.

Key results include:

• Successful zero-shot transfer of imitation policies to real-world environments with minimal performance loss.
• Effective real-time control in tasks requiring nuanced interaction, such as ball dribbling and adaptive locomotion over varied terrains.
• Demonstration of a significant reduction in the need for domain-specific reward shaping or engineering due to the inherent regularization provided by the MoCap-based skills.

Implications and Future Work

The results hold substantial implications for the development of agile and versatile robots across diverse domains. By facilitating the transfer of complex motion skills derived from biological analogs, this work provides a path towards more intuitive and naturally adaptive robotic systems. The framework lays the groundwork for future research in integrating broader MoCap datasets and exploring more complex interaction tasks, possibly extending to bi-manual manipulation or dynamic obstacle navigation.

Speculatively, future developments may include optimizing the simulation-to-reality transfer pipeline further, leveraging more sophisticated domain randomization techniques, and enhancing the robustness of learned behaviors under varied conditions. It will also be intriguing to explore the synergy between this approach and other learning paradigms such as learning from demonstrations, where the skill modules could serve as a basis for further fine-tuning.

In essence, this research paves the way towards more autonomous, versatile, and adaptive robots, defining a significant step forward in robotics and artificial intelligence.