CholecTrack20: A Dataset for Multi-Class Multiple Tool Tracking in Laparoscopic Surgery
(2312.07352)Abstract
Tool tracking in surgical videos is vital in computer-assisted intervention for tasks like surgeon skill assessment, safety zone estimation, and human-machine collaboration during minimally invasive procedures. The lack of large-scale datasets hampers Artificial Intelligence implementation in this domain. Current datasets exhibit overly generic tracking formalization, often lacking surgical context: a deficiency that becomes evident when tools move out of the camera's scope, resulting in rigid trajectories that hinder realistic surgical representation. This paper addresses the need for a more precise and adaptable tracking formalization tailored to the intricacies of endoscopic procedures by introducing CholecTrack20, an extensive dataset meticulously annotated for multi-class multi-tool tracking across three perspectives representing the various ways of considering the temporal duration of a tool trajectory: (1) intraoperative, (2) intracorporeal, and (3) visibility within the camera's scope. The dataset comprises 20 laparoscopic videos with over 35,000 frames and 65,000 annotated tool instances with details on spatial location, category, identity, operator, phase, and surgical visual conditions. This detailed dataset caters to the evolving assistive requirements within a procedure.
Overview
-
CholecTrack20 dataset enhances tool tracking in laparoscopic surgery, providing data for complex surgical scenarios.
-
The dataset contains annotations from 20 laparoscopic cholecystectomy videos, with over 35,000 frames and 65,000 tool annotations.
-
Tools are annotated with categories, locations, phases, surgical conditions, and different perspectives of tool trajectories.
-
CholecTrack20 facilitates AI development in surgical tool tracking, phase recognition, and surgical skill assessment.
-
Researchers can access the dataset under a non-commercial license and integrate it with the TrackEval evaluation metric system.
Introduction to CholecTrack20 Dataset
The CholecTrack20 dataset represents a significant step forward in the field of surgical data science. It addresses the critical need for extensive datasets meticulously annotated for multi-class multi-tool tracking, specifically designed for the domain of laparoscopic surgery. This dataset will enhance analytic capabilities in computer-assisted interventions by providing data that reflects the complex reality of surgeries, including the diverse scenarios where instruments may be outside the camera's field of view.
Dataset Overview and Methodology
Laparoscopic surgery presents unique challenges for tool tracking due to the limited field of view and the number of tools used simultaneously. To overcome these challenges, the CholecTrack20 dataset was created with annotations from 20 laparoscopic cholecystectomy videos. The dataset stands out for including over 35,000 frames and over 65,000 tool annotations, with details on tool location, category, phase, and surgical conditions.
Annotation Process and Dataset Structure
The meticulous annotation process for CholecTrack20 follows a comprehensive tracking formalization protocol. Tools are classified by categories, spatial location (bounding boxes), operators at trocars, and track identities. Furthermore, the dataset encapsulates three distinct perspectives of tool trajectories: the intraoperative perspective, which encompasses the full duration of the tool's usage within a patient's body; the intracorporeal perspective, which focuses on the period the tool is inside the body; and the visibility perspective, which is restricted to the duration a tool is visible within the camera’s field of view.
Potential for AI and Research Applications
With its rich annotations and multi-perspective approach, the CholecTrack20 dataset is an invaluable resource for developing AI models aimed at tool tracking and complementary surgical research such as phase recognition, adverse event prediction, and skill assessment. The existence of such a comprehensive dataset opens the door to a myriad of possibilities for improving and understanding the intricacies of laparoscopic surgery through advanced machine learning applications.
Availability for Researchers
The dataset, along with the associated code for its usage, is made available under the CC BY-NC-SA license for non-commercial use. Researchers can access visualization and conversion scripts, as well as support for integrating with the TrackEval evaluation metric system. This facilitates the wider adoption of the dataset and contributes to the development of innovative tracking algorithms and the emergence of new research findings in surgical tool tracking and analytics.
Create an account to read this summary for free:
GitHub - CAMMA-public/cholectrack20 (7 stars)