EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation (2310.06751v1)

Abstract: In this paper, we explore the dynamic grasping of moving objects through active pose tracking and reinforcement learning for hand-eye coordination systems. Most existing vision-based robotic grasping methods implicitly assume target objects are stationary or moving predictably. Performing grasping of unpredictably moving objects presents a unique set of challenges. For example, a pre-computed robust grasp can become unreachable or unstable as the target object moves, and motion planning must also be adaptive. In this work, we present a new approach, Eye-on-hAnd Reinforcement Learner (EARL), for enabling coupled Eye-on-Hand (EoH) robotic manipulation systems to perform real-time active pose tracking and dynamic grasping of novel objects without explicit motion prediction. EARL readily addresses many thorny issues in automated hand-eye coordination, including fast-tracking of 6D object pose from vision, learning control policy for a robotic arm to track a moving object while keeping the object in the camera's field of view, and performing dynamic grasping. We demonstrate the effectiveness of our approach in extensive experiments validated on multiple commercial robotic arms in both simulations and complex real-world tasks.

• The paper introduces EARL, a reinforcement learning framework enabling 6-DoF dynamic grasping via active pose tracking without relying on precomputed motion models.
• It integrates a wrist-mounted RGB-D camera and curriculum-trained policy for continuous adaptation, validated through extensive simulated and real-world experiments.
• Sim-to-real transfer is achieved with less than a 5% performance drop, underscoring the framework's efficiency in dynamic robotic manipulation.

Evaluation of EARL: Eye-on-Hand Reinforcement Learner for Adaptive Dynamic Grasping

The paper "EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation" presents a novel approach for dynamic grasping of moving objects by leveraging reinforcement learning (RL) techniques and real-time active pose tracking in robotic manipulation, particularly in Eye-on-Hand (EoH) robotic systems. This research addresses the challenges associated with grasping objects that move unpredictably, introducing a framework that facilitates the integration of sensory perception with actuation to perform grasping tasks in dynamic environments.

The primary contribution of this work is the introduction of the Eye-on-Hand Reinforcement Learner (EARL) framework, designed to enable EoH systems to perform six degrees of freedom (6-DoF) dynamic grasping without relying on motion prediction models. EARL achieves this by continually adjusting the camera's field of view and actively tracking the pose of the target object, thus optimizing the grasp pose in real-time. This approach stands in contrast to the conventional model-based or model-free grasping methods that either employ pre-defined object models or rely solely on data-driven grasp synthesis.

Key Contributions and Methodology

1. Active Pose Tracking and Reinforcement Learning: The paper presents a model-free RL-based methodology for pose tracking wherein EARL leverages visual observations through a wrist-mounted RGB-D camera to map pose differentials to dynamic grasping actions. Active object tracking is incorporated through a curriculum-trained RL policy, ensuring the robot continuously adapts its grasp pose as the object moves.
2. Experimental Validation: The robust performance of EARL is substantiated through extensive experiments in both simulated environments and real-world setups. Utilizing two robotic arm systems, Universal UR-5e and Kinova Gen3, EARL is demonstrated to handle novel objects featuring a range of motion types with high success rates. The framework's ability to seamlessly transition between simulation and real-world scenarios underscores its practical applicability.
3. Sim-to-Real Transfer and Performance: EARL successfully narrows the sim-to-real gap, exhibiting less than a 5% performance decline when transitioning from simulations to real-world tests. The paper credits this compact gap to the decoupling of sim-to-real training, facilitated by intrinsic adjustments to damping parameters in real torque-controlled robotic arms.

Implications and Future Directions

The implications of EARL are twofold: practical and theoretical. Practically, EARL enhances the adaptive capabilities of robotic systems, paving the way for applications in unstructured environments, such as human-robot object handovers, occlusion management, and complex industrial tasks that necessitate responsive manipulation. Theoretically, EARL's integration of continuous visual feedback and pose differential mapping contributes to ongoing research in robotic learning, emphasizing real-time adaptability without precomputed models.

Looking forward, future developments could focus on incorporating explicit collision modeling for further robustness and expanding object tracking capabilities in cluttered settings. Additionally, integrating advances in pose estimation and leveraging self-supervised learning paradigms for improved grasp detection could further refine EARL's object retrieval capabilities. This ongoing research trajectory aligns with the broader objectives of robotics in dynamically shifting environments, encouraging adaptive learning models and more autonomous manipulation systems.

Overall, the EARL framework introduces a significant step toward enhancing the dexterity and adaptability of robotic grasping solutions, offering a scalable and responsive method to tackle the intricate dynamics of object manipulation tasks in real-world scenarios. This work exemplifies the collaborative potential between reinforcement learning and robotic vision, indicating promising orientations for future research avenues.