Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Autonomous Marker-less Rapid Aerial Grasping (2211.13093v3)

Published 23 Nov 2022 in cs.RO and cs.CV

Abstract: In a future with autonomous robots, visual and spatial perception is of utmost importance for robotic systems. Particularly for aerial robotics, there are many applications where utilizing visual perception is necessary for any real-world scenarios. Robotic aerial grasping using drones promises fast pick-and-place solutions with a large increase in mobility over other robotic solutions. Utilizing Mask R-CNN scene segmentation (detectron2), we propose a vision-based system for autonomous rapid aerial grasping which does not rely on markers for object localization and does not require the appearance of the object to be previously known. Combining segmented images with spatial information from a depth camera, we generate a dense point cloud of the detected objects and perform geometry-based grasp planning to determine grasping points on the objects. In real-world experiments on a dynamically grasping aerial platform, we show that our system can replicate the performance of a motion capture system for object localization up to 94.5 % of the baseline grasping success rate. With our results, we show the first use of geometry-based grasping techniques with a flying platform and aim to increase the autonomy of existing aerial manipulation platforms, bringing them further towards real-world applications in warehouses and similar environments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez, “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognit., vol. 47, pp. 2280–2292, 2014.
  2. “Intel realsense d455,” accessed: 2022-09-15. [Online]. Available: https://web.archive.org/*/https://www.intelrealsense.com/depth-camera-d455
  3. K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980–2988.
  4. Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2,” https://github.com/facebookresearch/detectron2, 2019.
  5. Q.-Y. Zhou, J. Park, and V. Koltun, “Open3D: A modern library for 3D data processing,” arXiv:1801.09847, 2018.
  6. A. X. Appius, E. Bauer, M. Blöchlinger, A. Kalra, R. Oberson, A. Raayatsanati, P. Strauch, S. Suresh, M. von Salis, and R. K. Katzschmann, “Raptor: Rapid aerial pickup and transport of objects by robots,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 349–355.
  7. E. Olson, “Apriltag: A robust and flexible visual fiducial system,” 2011 IEEE International Conference on Robotics and Automation, pp. 3400–3407, 2011.
  8. V. Lippiello, J. Cacace, A. Santamaria-Navarro, J. Andrade-Cetto, M. A. Trujillo, Y. Rodríguez, and A. Viguria, “Hybrid visual servoing with hierarchical task composition for aerial manipulation,” IEEE Robotics and Automation Letters, vol. 1, pp. 259–266, 2016.
  9. L. R. Buonocore, J. Cacace, and V. Lippiello, “Hybrid visual servoing for aerial grasping with hierarchical task-priority control,” 2015 23rd Mediterranean Conference on Control and Automation (MED), pp. 617–623, 2015.
  10. J. Fishman and L. Carlone, “Control and trajectory optimization for soft aerial manipulation,” 2021 IEEE Aerospace Conference (50100), pp. 1–17, 2021.
  11. J. Fishman, S. Ubellacker, N. Hughes, and L. Carlone, “Dynamic grasping with a ”soft” drone: From theory to practice,” 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4214–4221, 2021.
  12. B. Luo, H. Chen, F. Quan, S. Zhang, and Y. hui Liu, “Natural feature-based visual servoing for grasping target with an aerial manipulator,” Journal of Bionic Engineering, vol. 17, pp. 215–228, 2020.
  13. L. Lin, Y. Yang, H. Cheng, and X. Chen, “Autonomous vision-based aerial grasping for rotorcraft unmanned aerial vehicles,” Sensors (Basel, Switzerland), vol. 19, 2019.
  14. P. Ramón-Soria, B. C. Arrue, and A. Ollero, “Grasp planning and visual servoing for an outdoors aerial dual manipulator,” Engineering, vol. 6, pp. 77–88, 2020.
  15. P. J. Besl and N. D. McKay, “A method for registration of 3-d shapes,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, pp. 239–256, 1992.
  16. P. R. Soria, B. C. Arrue, and A. Ollero, “Detection, location and grasping objects using a stereo sensor on uav in outdoor environments,” Sensors (Basel, Switzerland), vol. 17, 2017.
  17. G. LoweDavid, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, 2004.
  18. E. Rosten, R. B. Porter, and T. Drummond, “Faster and better: A machine learning approach to corner detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, pp. 105–119, 2008.
  19. J. R. Thomas, G. Loianno, K. Daniilidis, and V. R. Kumar, “Visual servoing of quadrotors for perching by hanging from cylindrical objects,” IEEE Robotics and Automation Letters, vol. 1, pp. 57–64, 2016.
  20. H. Seo, S. Kim, and H. J. Kim, “Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control,” 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 6362–6368, 2017.
  21. B. S. Zapata-Impata, P. Gil, J. Pomares, and F. Torres, “Fast geometry-based computation of grasping points on three-dimensional point clouds,” International Journal of Advanced Robotic Systems, vol. 16, 2019.
  22. R. Haschke, G. Walck, and H. J. Ritter, “Geometry-based grasping pipeline for bi-modal pick and place,” 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4002–4008, 2021.
  23. Y. Jiang, S. Moseson, and A. Saxena, “Efficient grasping from rgbd images: Learning using a new rectangle representation,” 2011 IEEE International Conference on Robotics and Automation, pp. 3304–3311, 2011.
  24. A. Mousavian, C. Eppner, and D. Fox, “6-dof graspnet: Variational grasp generation for object manipulation,” 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2901–2910, 2019.
  25. S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” ArXiv, vol. abs/1504.00702, 2016.
  26. K. He, G. Gkioxari, P. Dollár, and R. B. Girshick, “Mask r-cnn,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, pp. 386–397, 2017.
  27. D. Bolya, C. Zhou, F. Xiao, and Y. J. Lee, “Yolact: Real-time instance segmentation,” 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9156–9165, 2019.
  28. C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” ArXiv, vol. abs/2207.02696, 2022.
  29. G. Jocher, “YOLOv5 by Ultralytics,” 5 2020. [Online]. Available: https://github.com/ultralytics/yolov5
  30. P. Foehn, E. Kaufmann, A. Romero, R. Pěnička, S. Sun, L. Bauersfeld, T. M. Laengle, G. Cioffi, Y. Song, A. Loquercio, and D. Scaramuzza, “Agilicious: Open-source and open-hardware agile quadrotor for vision-based flight,” Science Robotics, vol. 7, 2022.
  31. “Nvidia jetson nano,” accessed: 2022-09-15. [Online]. Available: https://web.archive.org/*/https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-nano/
  32. “Pixhawk,” accessed: 2022-09-15. [Online]. Available: https://web.archive.org/*/https://pixhawk.org/
  33. “Mavlink,” accessed: 2022-09-15. [Online]. Available: https://web.archive.org/*/https://mavlink.io/
  34. S. Macenski, T. Foote, B. P. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” Science Robotics, vol. 7, 2022.
  35. T. Kronauer, J. Pohlmann, M. Matthé, T. Smejkal, and G. P. Fettweis, “Latency overhead of ros2 for modular time-critical systems,” ArXiv, vol. abs/2101.02074, 2021.
  36. “Zeromq,” accessed: 2022-09-15. [Online]. Available: https://github.com/zeromq/
  37. “Protobuf,” accessed: 2022-09-15. [Online]. Available: https://github.com/protocolbuffers/protobuf
  38. M. Coatsworth, J. Tran, and A. Ferworn, “A hybrid lossless and lossy compression scheme for streaming rgb-d data in real time,” 2014 IEEE International Symposium on Safety, Security, and Rescue Robotics (2014), pp. 1–6, 2014.
  39. G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools, 2000.
  40. C. Forster, Z. Zhang, M. Gassner, M. Werlberger, and D. Scaramuzza, “Svo: Semidirect visual odometry for monocular and multicamera systems,” IEEE Transactions on Robotics, vol. 33, no. 2, pp. 249–265, 2017.
  41. R. Mur-Artal, J. M. M. Montiel, and J. D. Tardós, “Orb-slam: A versatile and accurate monocular slam system,” IEEE Transactions on Robotics, vol. 31, pp. 1147–1163, 2015.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube