Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
9 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
40 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning to Grasp Without Seeing (1805.04201v1)

Published 10 May 2018 in cs.RO, cs.AI, and cs.SY

Abstract: Can a robot grasp an unknown object without seeing it? In this paper, we present a tactile-sensing based approach to this challenging problem of grasping novel objects without prior knowledge of their location or physical properties. Our key idea is to combine touch based object localization with tactile based re-grasping. To train our learning models, we created a large-scale grasping dataset, including more than 30 RGB frames and over 2.8 million tactile samples from 7800 grasp interactions of 52 objects. To learn a representation of tactile signals, we propose an unsupervised auto-encoding scheme, which shows a significant improvement of 4-9% over prior methods on a variety of tactile perception tasks. Our system consists of two steps. First, our touch localization model sequentially 'touch-scans' the workspace and uses a particle filter to aggregate beliefs from multiple hits of the target. It outputs an estimate of the object's location, from which an initial grasp is established. Next, our re-grasping model learns to progressively improve grasps with tactile feedback based on the learned features. This network learns to estimate grasp stability and predict adjustment for the next grasp. Re-grasping thus is performed iteratively until our model identifies a stable grasp. Finally, we demonstrate extensive experimental results on grasping a large set of novel objects using tactile sensing alone. Furthermore, when applied on top of a vision-based policy, our re-grasping model significantly boosts the overall accuracy by 10.6%. We believe this is the first attempt at learning to grasp with only tactile sensing and without any prior object knowledge.

Citations (60)

Summary

  • The paper presents a novel tactile grasping approach that localizes objects via sequential touch and iteratively adjusts grasps without visual input.
  • The methodology employs particle filtering for localization and an unsupervised auto-encoder for re-grasping, improving tactile perception by 4%-9%.
  • Experimental results demonstrate a 40% grasping accuracy on unknown objects, with an additional 10.6% improvement when integrated with vision-based policies.

Overview of "Learning to Grasp without Seeing"

The paper "Learning to Grasp without Seeing" presents a novel approach to robotic grasping, focusing on the capability of a robot to grasp an unknown object using solely tactile sensing, without any prior knowledge of the object's properties or location. The authors, hailing from the Robotics Institute at Carnegie Mellon University, explore the intriguing possibility of robotic tactile-based manipulation, inspired by observations of human tactile abilities.

Methodology and Contributions

The research introduces a comprehensive system for tactile-based grasping that involves two significant components: tactile-based localization and re-grasping.

  1. Touch-based Localization: The system includes a touch localization model that incorporates particle filtering to estimate an object's position based solely on tactile feedback. This sequential touch-scan method collects data by touching various points in the workspace and aggregates this information to localize the object.
  2. Re-grasping Model: The re-grasping process iteratively adjusts the robot's grasp based on tactile feedback until a stable grasp is achieved. The re-grasping model is driven by a learning-based approach, employing an unsupervised auto-encoder to extract meaningful features from tactile signals. This approach shows a notable improvement of 4%-9% in tactile perception tasks compared to previous methods.

The authors have also created a substantial grasping dataset comprising over 30,000 RGB frames and 2.8 million tactile samples generated from interactions with 52 different objects. This dataset serves as a crucial training and validation resource for the proposed models.

Results and Implications

The research demonstrated the efficacy of their methods through extensive experimental results. The tactile-only system achieves a 40% grasping accuracy on a diverse set of novel objects. Moreover, when integrated with a vision-based policy, the re-grasping model enhances overall accuracy by 10.6%. These findings underscore the potential of tactile feedback in improving robotic grasping reliability, even when initial visual perceptions are leveraged.

Despite the promising outcomes, the system's performance is currently contingent on an initially random guess for the object location, which can be further optimized. The authors suggest future explorations in joint learning of localization and re-grasping using reinforcement learning techniques, as well as integrating novel tactile sensors for improved observability.

Future Directions

The paper opens new avenues in tactile manipulation, emphasizing the need for continued development in tactile sensor technology and integration with existing visual systems. The improvement of tactile sensing and its incorporation into a multi-modal framework could bring significant advancements in robotic manipulation tasks, including applications in environments where vision is obstructed or unavailable. Furthermore, the dataset provided could stimulate further research into tactile feature learning and its applications in various robotic tasks beyond grasping.

Overall, "Learning to Grasp without Seeing" sheds light on the untapped potential of haptic feedback in robotics, presenting a compelling case for the incorporation of tactile sensing in autonomous robot systems. This work serves as a foundational step towards more intuitive and adaptable robotic grasping, inspired by human tactile dexterity.

Youtube Logo Streamline Icon: https://streamlinehq.com