Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Novel Bioinspired Neuromorphic Vision-based Tactile Sensor for Fast Tactile Perception (2403.10120v1)

Published 15 Mar 2024 in cs.RO, cs.SY, and eess.SY

Abstract: Tactile sensing represents a crucial technique that can enhance the performance of robotic manipulators in various tasks. This work presents a novel bioinspired neuromorphic vision-based tactile sensor that uses an event-based camera to quickly capture and convey information about the interactions between robotic manipulators and their environment. The camera in the sensor observes the deformation of a flexible skin manufactured from a cheap and accessible 3D printed material, whereas a 3D printed rigid casing houses the components of the sensor together. The sensor is tested in a grasping stage classification task involving several objects using a data-driven learning-based approach. The results show that the proposed approach enables the sensor to detect pressing and slip incidents within a speed of 2 ms. The fast tactile perception properties of the proposed sensor makes it an ideal candidate for safe grasping of different objects in industries that involve high-speed pick-and-place operations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. W. Chen, H. Khamis, I. Birznieks, N. F. Lepora, and S. J. Redmond, “Tactile sensors for friction estimation and incipient slip detection—toward dexterous robotic manipulation: A review,” IEEE Sensors Journal, vol. 18, no. 22, pp. 9049–9064, 2018.
  2. H. Yousef, M. Boukallel, and K. Althoefer, “Tactile sensing for dexterous in-hand manipulation in robotics—a review,” Sensors and Actuators A: physical, vol. 167, no. 2, pp. 171–187, 2011.
  3. Q. Li, O. Kroemer, Z. Su, F. F. Veiga, M. Kaboli, and H. J. Ritter, “A review of tactile information: Perception and action through touch,” IEEE Transactions on Robotics, vol. 36, no. 6, pp. 1619–1634, 2020.
  4. Z. Kappassov, J.-A. Corrales, and V. Perdereau, “Tactile sensing in dexterous robot hands,” Robotics and Autonomous Systems, vol. 74, pp. 195–220, 2015.
  5. K. Shimonomura, “Tactile image sensors employing camera: A review,” Sensors, vol. 19, no. 18, p. 3933, 2019.
  6. A. Yamaguchi and C. G. Atkeson, “Recent progress in tactile sensing and sensors for robotic manipulation: can we turn tactile sensing into vision?” Advanced Robotics, vol. 33, no. 14, pp. 661–673, 2019.
  7. U. H. Shah, R. Muthusamy, D. Gan, Y. Zweiri, and L. Seneviratne, “On the design and development of vision-based tactile sensors,” Journal of Intelligent & Robotic Systems, vol. 102, pp. 1–27, 2021.
  8. S. Zhang, Z. Chen, Y. Gao, W. Wan, J. Shan, H. Xue, F. Sun, Y. Yang, and B. Fang, “Hardware technology of vision-based tactile sensor: A review,” IEEE Sensors Journal, 2022.
  9. M. Li, T. Li, and Y. Jiang, “Marker displacement method used in vision-based tactile sensors—from 2d to 3d-a review,” IEEE Sensors Journal, 2023.
  10. W. Yuan, S. Dong, and E. H. Adelson, “Gelsight: High-resolution robot tactile sensors for estimating geometry and force,” Sensors, vol. 17, no. 12, p. 2762, 2017.
  11. A. C. Abad and A. Ranasinghe, “Visuotactile sensors with emphasis on gelsight sensor: A review,” IEEE Sensors Journal, vol. 20, no. 14, pp. 7628–7638, 2020.
  12. C. Chorley, C. Melhuish, T. Pipe, and J. Rossiter, “Development of a tactile sensor based on biologically inspired edge encoding,” in 2009 International Conference on Advanced Robotics.   IEEE, 2009, pp. 1–6.
  13. N. F. Lepora, “Soft biomimetic optical tactile sensing with the tactip: A review,” IEEE Sensors Journal, vol. 21, no. 19, pp. 21 131–21 143, 2021.
  14. A. Yamaguchi and C. G. Atkeson, “Tactile behaviors with the vision-based tactile sensor fingervision,” International Journal of Humanoid Robotics, vol. 16, no. 03, p. 1940002, 2019.
  15. E. Donlon, S. Dong, M. Liu, J. Li, E. Adelson, and A. Rodriguez, “Gelslim: A high-resolution, compact, robust, and calibrated tactile-sensing finger,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2018, pp. 1927–1934.
  16. G. Zhang, Y. Du, H. Yu, and M. Y. Wang, “Deltact: A vision-based tactile sensor using a dense color pattern,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 778–10 785, 2022.
  17. C. Pang, K. Mak, Y. Zhang, Y. Yang, Y. A. Tse, and M. Y. Wang, “Viko: An adaptive gecko gripper with vision-based tactile sensor,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 736–742.
  18. J. W. James and N. F. Lepora, “Slip detection for grasp stabilization with a multifingered tactile robot hand,” IEEE Transactions on Robotics, 2020.
  19. A. Yamaguchi, “Fingervision with whiskers: Light touch detection with vision-based tactile sensors,” in 2021 Fifth IEEE International Conference on Robotic Computing (IRC).   IEEE, 2021, pp. 56–64.
  20. M. Lambeta, P.-W. Chou, S. Tian, B. Yang, B. Maloon, V. R. Most, D. Stroud, R. Santos, A. Byagowi, G. Kammerer et al., “Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation,” IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 3838–3845, 2020.
  21. Y. She, S. Wang, S. Dong, N. Sunil, A. Rodriguez, and E. Adelson, “Cable manipulation with a tactile-reactive gripper,” The International Journal of Robotics Research, vol. 40, no. 12-14, pp. 1385–1401, 2021.
  22. C. Wang, S. Wang, B. Romero, F. Veiga, and E. Adelson, “Swingbot: Learning physical features from in-hand tactile exploration for dynamic swing-up manipulation,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 5633–5640.
  23. B. Ward-Cherrier, N. Pestell, L. Cramphorn, B. Winstone, M. E. Giannaccini, J. Rossiter, and N. F. Lepora, “The tactip family: Soft optical tactile sensors with 3d-printed biomimetic morphologies,” Soft robotics, vol. 5, no. 2, pp. 216–227, 2018.
  24. R. Muthusamy, X. Huang, Y. Zweiri, L. Seneviratne, and D. Gan, “Neuromorphic event-based slip detection and suppression in robotic grasping and manipulation,” IEEE Access, vol. 8, pp. 153 364–153 384, 2020.
  25. G. Gallego, T. Delbrück, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis et al., “Event-based vision: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 154–180, 2020.
  26. R. Muthusamy, A. Ayyad, M. Halwani, D. Swart, D. Gan, L. Seneviratne, and Y. Zweiri, “Neuromorphic eye-in-hand visual servoing,” IEEE Access, vol. 9, pp. 55 853–55 870, 2021.
  27. B. Ward-Cherrier, N. Pestell, and N. F. Lepora, “Neurotac: A neuromorphic optical tactile sensor applied to texture recognition,” arXiv preprint arXiv:2003.00467, 2020.
  28. F. L. Macdonald, N. F. Lepora, J. Conradt, and B. Ward-Cherrier, “Neuromorphic tactile edge orientation classification in an unsupervised spiking neural network,” Sensors, vol. 22, no. 18, p. 6998, 2022.
  29. K. Kumagai and K. Shimonomura, “Event-based tactile image sensor for detecting spatio-temporal fast phenomena in contacts,” in 2019 IEEE World Haptics Conference (WHC).   IEEE, 2019, pp. 343–348.
  30. H. Sajwani, A. Ayyad, Y. Alkendi, M. Halwani, Y. Abdulrahman, A. Abusafieh, and Y. Zweiri, “Tactigraph: An asynchronous graph neural network for contact angle prediction using neuromorphic vision-based tactile sensing,” Sensors, vol. 23, no. 14, p. 6451, 2023.
  31. A. Rigi, F. Baghaei Naeini, D. Makris, and Y. Zweiri, “A novel event-based incipient slip detection using dynamic active-pixel vision sensor (davis),” Sensors, vol. 18, no. 2, p. 333, 2018.
  32. F. Baghaei Naeini, D. Makris, D. Gan, and Y. Zweiri, “Dynamic-vision-based force measurements using convolutional recurrent neural networks,” Sensors, vol. 20, no. 16, p. 4469, 2020.
  33. F. B. Naeini, A. Alali, R. Al-Husari, A. Rigi, M. K. AlSharman, D. Makris, and Y. Zweiri, “A novel dynamic-vision-based approach for tactile sensing applications,” IEEE Transactions on Instrumentation and Measurement, 2019.
  34. X. Huang, R. Muthusamy, E. Hassan, Z. Niu, L. Seneviratne, D. Gan, and Y. Zweiri, “Neuromorphic vision based contact-level classification in robotic grasping applications,” Sensors, vol. 20, no. 17, 2020. [Online]. Available: https://www.mdpi.com/1424-8220/20/17/4724
  35. N. Funk, E. Helmut, G. Chalvatzaki, R. Calandra, and J. Peters, “Evetac: An event-based optical tactile sensor for robotic manipulation,” arXiv preprint arXiv:2312.01236, 2023.
  36. E. Jarocka, J. A. Pruszynski, and R. S. Johansson, “Human touch receptors are sensitive to spatial details on the scale of single fingerprint ridges,” Journal of Neuroscience, vol. 41, no. 16, pp. 3622–3634, 2021.
  37. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Omar Faris (1 paper)
  2. Mohammad I. Awad (3 papers)
  3. Murana A. Awad (1 paper)
  4. Yahya Zweiri (32 papers)
  5. Kinda Khalaf (2 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now