Dexterous Functional Grasping
(2312.02975)Abstract
While there have been significant strides in dexterous manipulation, most of it is limited to benchmark tasks like in-hand reorientation which are of limited utility in the real world. The main benefit of dexterous hands over two-fingered ones is their ability to pickup tools and other objects (including thin ones) and grasp them firmly to apply force. However, this task requires both a complex understanding of functional affordances as well as precise low-level control. While prior work obtains affordances from human data this approach doesn't scale to low-level control. Similarly, simulation training cannot give the robot an understanding of real-world semantics. In this paper, we aim to combine the best of both worlds to accomplish functional grasping for in-the-wild objects. We use a modular approach. First, affordances are obtained by matching corresponding regions of different objects and then a low-level policy trained in sim is run to grasp it. We propose a novel application of eigengrasps to reduce the search space of RL using a small amount of human data and find that it leads to more stable and physically realistic motion. We find that eigengrasp action space beats baselines in simulation and outperforms hardcoded grasping in real and matches or outperforms a trained human teleoperator. Results visualizations and videos at https://dexfunc.github.io/
Overview
-
The paper explores dexterous robotic manipulation for functional grasping of everyday objects like tools.
-
It highlights the limitation of conventional two-fingered grippers and introduces a method to enable robots to use tools effectively.
-
The research utilizes an affordance model along with reinforcement learning and 'eigengrasps' to predict and execute object grasping.
-
Simulations show the method surpassing existing baselines and the performance of human teleoperators for specific tasks.
-
Successful real-world experiments indicate progress toward practical robotic utility in daily activities and tool manipulation.
Abstract Overview
The paper addresses the field of dexterous robotic manipulation, particularly focusing on picking up and functionally grasping tools and objects of daily life such as hammers, drills, and saucepans. The aim is to enable robots to grasp objects in a manner that allows for their effective use afterward, which represents a much higher level of utility and complexity compared to basic in-hand object reorientation tasks that have been the focus of most research in robotic manipulation.
Introduction and Challenges
Dexterous manipulation, the skill humans regularly employ to use tools, has been a challenging arena for robotics. Machines generally use simple two-fingered grippers, which significantly limits the range of objects they can manipulate and the actions they can perform. To use tools effectively, robots need to perform functional grasping, where an object is picked up such that it can be utilized for its designed purpose. This process involves not only a complex series of movements but also the intersection of perception, reasoning, and control.
Methodology
The approach taken is modular, dealing with grasp prediction, execution, and post-grasp trajectory. For predicting pre-grasp poses, the research leverages an affordance model which is informed by large internet datasets to identify plausible contact points on various objects. A novel application of 'eigengrasps', which are essentially principal components of hand poses, is introduced to constrain the action space in reinforcement learning (RL) simulations, reducing complexity while ensuring realistic motion. This combination of internet data for affordance generalization and specialized simulation for policy training bridges the gap between virtual learning environments and real-world application.
Experimental Results
Experiments demonstrated that the proposed method outperformed baselines in simulation and could successfully translate to real-world object manipulation. Notably, it managed to match or even exceed the performance of a trained human teleoperator for some tasks, despite the robotic policy only being trained with hammers. The use of eigengrasps significantly stabilized RL training and improved the physical plausibility of learned grasps. Real-world experiments confirmed the effectiveness of the method on a broad spectrum of objects with varied shapes, sizes, and weights.
This investigation signifies a substantial step toward enabling robots to conduct functional grasping tasks, advancing the practical utility of robotic systems in everyday scenarios, including tool manipulation.
Create an account to read this summary for free: