TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos

Enhancing MOT in Sports through TeamTrack: A Comprehensive Dataset and Evaluation

Introduction to TeamTrack Dataset

The challenges of Multi-Object Tracking (MOT) in team sports are complex due to rapid player movements, occlusions, and similar player apparences. To address the need for advanced MOT in sports, the paper introduces the TeamTrack dataset, a large-scale benchmark featuring over 279,900 frames and 4,374,900 bounding boxes across soccer, basketball, and handball from multiple perspectives. This dataset not only fills the gap left by incomplete pitch coverage and single-sport focus in existing datasets but provides a unique setup to challenge and enhance current MOT solutions in a well-rounded and extensive way.

Contributions of the Study

The paper outlines three major contributions:

1. Development of TeamTrack, a large-scale dataset for MOT in sports featuring extensive variety and scope.
2. In-depth analysis and characterization of this novel dataset highlighting its potential to advance sports analytics and tracking technologies.
3. Comprehensive benchmarks using TeamTrack for object detection, trajectory forecasting, and multi-object tracking, setting a new standard for future research and applications in sports MOT.

Methodology and Dataset Features

• Data Collection and Annotation: Utilizing fisheye and drone cameras, videos were captured offering both side and top views. An extensive annotation process was described, applying tools like CVAT and Labelbox, and involved dealing with high volumes of data which required around 600 person-hours.
• Appearance Similarity and Dynamics: TeamTrack was assessed against other MOT datasets showing higher similarity in player appearances and more complex motion patterns, which are quantified by new metrics for evaluating cosine similarities of re-ID features and average IoU scores between adjacent frames.

Experimental Evaluations

• Object Detection: This involved fine-tuning a YOLOv8 model on the TeamTrack dataset, with the evaluation highlighting differences in performance based on the view types and the necessity for fine-tuning when dealing with sports footage.
• Trajectory Forecasting: Employing models like LSTM showed variance in performance across sports within the TeamTrack dataset, indicating complex player movement.
• Multi-object Tracking: Applying ByteTrack and BoT-SORT, the study revealed how the diversity and complexity of the dataset provided unique challenges, reflected in the variability of scores across sports and view perspectives.

Dataset Implications and Potential for Future Research

The TeamTrack dataset with its detailed annotations and dual perspectives (top and side) ensures a comprehensive set of data that can help in understanding the nuanced requirements of sports MOT. The dual perspectives particularly allow for the exploration of multi-view tracking systems which could significantly improve precision and robustly handle occlusions and player interactions.

This dataset’s introduction prompts several potential areas for development:

• Algorithmic Improvements: Given the complexity and scale of the dataset, there is significant room for the development of specialized MOT algorithms that can handle high dynamics, occlusions, and appearance similarities competently.
• Cross-disciplinary Applications: Insights from TeamTrack could extend into other areas such as crowd analysis and robotics, where similar challenges are prevalent.
• Enhancement through Augmentation: Including more sports, broader geographic diversity, and environmental conditions in the dataset could further validate and refine the robustness of MOT systems.

In conclusion, the TeamTrack dataset represents a significant step toward addressing the complexities associated with tracking in team sports. By offering a detailed analysis and comprehensive benchmarks, it sets the foundation for future innovations in the field of sports analytics and beyond, promising enhanced capabilities in MOT across various challenging scenarios.