Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Sim-to-Real Surgical Robot Learning and Autonomous Planning for Internal Tissue Points Manipulation using Reinforcement Learning (2306.14085v1)

Published 25 Jun 2023 in cs.RO

Abstract: Indirect simultaneous positioning (ISP), where internal tissue points are placed at desired locations indirectly through the manipulation of boundary points, is a type of subtask frequently performed in robotic surgeries. Although challenging due to complex tissue dynamics, automating the task can potentially reduce the workload of surgeons. This paper presents a sim-to-real framework for learning to automate the task without interacting with a real environment, and for planning preoperatively to find the grasping points that minimize local tissue deformation. A control policy is learned using deep reinforcement learning (DRL) in the FEM-based simulation environment and transferred to real-world situation. Grasping points are planned in the simulator by utilizing the trained policy using Bayesian optimization (BO). Inconsistent simulation performance is overcome by formulating the problem as a state augmented Markov decision process (MDP). Experimental results show that the learned policy places the internal tissue points accurately, and that the planned grasping points yield small tissue deformation among the trials. The proposed learning and planning scheme is able to automate internal tissue point manipulation in surgeries and has the potential to be generalized to complex surgical scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (26)
  1. Z. Chen, A. Malpani, P. Chalasani, A. Deguet, S. S. Vedula, P. Kazanzides, and R. H. Taylor, “Virtual fixture assistance for needle passing and knot tying,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2016, pp. 2343–2350.
  2. Z.-Y. Chiu, F. Richter, E. K. Funk, R. K. Orosco, and M. C. Yip, “Bimanual regrasping for suture needles using reinforcement learning for rapid motion planning,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 7737–7743.
  3. F. Richter, S. Shen, F. Liu, J. Huang, E. K. Funk, R. K. Orosco, and M. C. Yip, “Autonomous robotic suction to clear the surgical field for hemostasis using image-based blood flow detection,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1383–1390, 2021.
  4. Y. Li, F. Richter, J. Lu, E. K. Funk, R. K. Orosco, J. Zhu, and M. C. Yip, “Super: A surgical perception framework for endoscopic tissue manipulation with surgical robotics,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2294–2301, 2020.
  5. J. Lu, A. Jayakumari, F. Richter, Y. Li, and M. C. Yip, “Super deep: A surgical perception framework for robotic tissue manipulation using deep learning for feature extraction,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 4783–4789.
  6. S. Lin, A. J. Miao, J. Lu, S. Yu, Z.-Y. Chiu, F. Richter, and M. C. Yip, “Semantic-super: A semantic-aware surgical perception framework for endoscopic tissue classification, reconstruction, and tracking,” arXiv preprint arXiv:2210.16674, 2022.
  7. F. Zhong, Y. Wang, Z. Wang, and Y.-H. Liu, “Dual-arm robotic needle insertion with active tissue deformation for autonomous suturing,” IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2669–2676, 2019.
  8. M. Afshar, J. Carriere, T. Meyer, R. Sloboda, S. Husain, N. Usmani, and M. Tavakoli, “A model-based multi-point tissue manipulation for enhancing breast brachytherapy,” IEEE Transactions on Medical Robotics and Bionics, 2022.
  9. F. Alambeigi, Z. Wang, Y.-h. Liu, R. H. Taylor, and M. Armand, “Toward semi-autonomous cryoablation of kidney tumors via model-independent deformable tissue manipulation technique,” Annals of biomedical engineering, vol. 46, no. 10, pp. 1650–1662, 2018.
  10. Y. Wu, W. Yan, T. Kurutach, L. Pinto, and P. Abbeel, “Learning to manipulate deformable objects without demonstrations,” arXiv preprint arXiv:1910.13439, 2019.
  11. C. Shin, P. W. Ferguson, S. A. Pedram, J. Ma, E. P. Dutson, and J. Rosen, “Autonomous tissue manipulation via surgical robot using learning based model predictive control,” in 2019 International Conference on Robotics and Automation.   IEEE, 2019, pp. 3875–3881.
  12. J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS).   IEEE, 2017, pp. 23–30.
  13. B. Thananjeyan, A. Garg, S. Krishnan, C. Chen, L. Miller, and K. Goldberg, “Multilateral surgical pattern cutting in 2d orthotropic gauze with deep reinforcement learning policies for tensioning,” in 2017 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2017, pp. 2371–2378.
  14. E. Tagliabue, A. Pore, D. Dall’Alba, E. Magnabosco, M. Piccinelli, and P. Fiorini, “Soft tissue simulation environment to learn manipulation tasks in autonomous robotic surgery,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 3261–3266.
  15. F. Liu, Z. Li, Y. Han, J. Lu, F. Richter, and M. C. Yip, “Real-to-sim registration of deformable soft tissue with position-based dynamics for surgical robot autonomy,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 12 328–12 334.
  16. T. Wada, S. Hirai, and S. Kawamura, “Indirect simultaneous positioning operations of extensionally deformable objects,” in Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No. 98CH36190), vol. 2.   IEEE, 1998, pp. 1333–1338.
  17. F. Faure, C. Duriez, H. Delingette, J. Allard, B. Gilles, S. Marchesseau, H. Talbot, H. Courtecuisse, G. Bousquet, I. Peterlik et al., “Sofa: A multi-model framework for interactive physical simulation,” in Soft tissue biomechanical modeling for computer assisted surgery.   Springer, 2012, pp. 283–321.
  18. E. Tagliabue, D. Dall’Alba, E. Magnabosco, I. Peterlik, and P. Fiorini, “Biomechanical modelling of probe to tissue interaction during ultrasound scanning,” International Journal of Computer Assisted Radiology and Surgery, vol. 15, no. 8, pp. 1379–1387, 2020.
  19. T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel et al., “Soft actor-critic algorithms and applications,” arXiv preprint arXiv:1812.05905, 2018.
  20. E. Altman and P. Nain, “Closed-loop control with delayed information,” ACM sigmetrics performance evaluation review, vol. 20, no. 1, pp. 193–204, 1992.
  21. B. C. Csáji and L. Monostori, “Value function based reinforcement learning in changing markovian environments.” Journal of Machine Learning Research, vol. 9, no. 8, 2008.
  22. A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,” Journal of Machine Learning Research, vol. 22, no. 268, pp. 1–8, 2021. [Online]. Available: http://jmlr.org/papers/v22/20-1364.html
  23. P. Kazanzides, Z. Chen, A. Deguet, G. S. Fischer, R. H. Taylor, and S. P. DiMaio, “An open-source research kit for the da vinci surgical system,” in IEEE Intl. Conf. on Robotics and Auto. (ICRA), Hong Kong, China, 2014, pp. 6434–6439.
  24. A. E. C. Santos, F. N. Linhares, M. C. A. M. Leite, V. A. Escócio, and R. C. R. Nunes, “A new device to simulate the performance of rubber dams for dental applications,” Polymer Testing, vol. 94, p. 107043, 2021.
  25. G. Singh and A. Chanda, “Mechanical properties of whole-body soft human tissues: a review,” Biomedical Materials, 2021.
  26. M. Afshar, J. Carriere, H. Rouhani, T. Meyer, R. S. Sloboda, S. Husain, N. Usmani, and M. Tavakoli, “Accurate tissue deformation modeling using a kalman filter and admm-based projective dynamics,” IEEE/ASME Transactions on Mechatronics, 2022.
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now