Cascade-DETR: Delving into High-Quality Universal Object Detection

Abstract: Object localization in general environments is a fundamental part of vision systems. While dominating on the COCO benchmark, recent Transformer-based detection methods are not competitive in diverse domains. Moreover, these methods still struggle to very accurately estimate the object bounding boxes in complex environments. We introduce Cascade-DETR for high-quality universal object detection. We jointly tackle the generalization to diverse domains and localization accuracy by proposing the Cascade Attention layer, which explicitly integrates object-centric information into the detection decoder by limiting the attention to the previous box prediction. To further enhance accuracy, we also revisit the scoring of queries. Instead of relying on classification scores, we predict the expected IoU of the query, leading to substantially more well-calibrated confidences. Lastly, we introduce a universal object detection benchmark, UDB10, that contains 10 datasets from diverse domains. While also advancing the state-of-the-art on COCO, Cascade-DETR substantially improves DETR-based detectors on all datasets in UDB10, even by over 10 mAP in some cases. The improvements under stringent quality requirements are even more pronounced. Our code and models will be released at https://github.com/SysCV/cascade-detr.

• The paper introduces a cascade attention mechanism that iteratively refines bounding box predictions to boost detection accuracy.
• It integrates an IoU-aware recalibration system to adjust confidence scores based on localization quality, improving final detection outcomes.
• Extensive experiments on the novel UDB10 benchmark demonstrate over 10 mAP improvement in diverse domains such as traffic and medical imaging.

Cascade-DETR: Delving into High-Quality Universal Object Detection

Introduction

Object detection is a pivotal task in computer vision, essential for applications such as autonomous driving and medical diagnostics. However, despite the advances brought by DETR-based models, they often falter when extended beyond the COCO benchmark into diverse real-world datasets. The paper "Cascade-DETR: Delving into High-Quality Universal Object Detection" introduces Cascade-DETR, a novel approach specifically designed to improve generalization across varied domains and enhance bounding box accuracy. This is achieved by integrating a cascade attention mechanism, which refines detection through iterative box predictions, and an IoU-aware scoring system to calibrate query confidence scores.

Figure 1: Detection results comparison between DN-DETR~\cite{dndetr} and Cascade-DN-DETR, illustrating improved performance across IoU thresholds.

Methodology

Cascade Attention

Cascade-DETR employs a cascade attention mechanism that confines the spatial scope of cross-attention layers within the predicted bounding box from the previous layer. This approach leverages object-centric priors to progressively refine box predictions, significantly enhancing detection accuracy. By iteratively narrowing the attention region, the cascade structure ensures that features vital for object recognition are prioritized, allowing for precise localization even under higher IoU thresholds.

Figure 2: The architecture of Cascade-DETR's transformer decoder featuring box-constrained cross-attention regions.

IoU-aware Query Recalibration

Query recalibration further augments prediction accuracy by integrating an IoU prediction branch that recalibrates classification scores based on localization quality. This branch predicts the expected IoU between the query and ground truth boxes, adjusting confidence scores to reflect bounding box precision rather than purely classification accuracy. This recalibration principle ensures that high-quality box predictions are consistently prioritized during inference.

Figure 3: Sparsification plot illustrating improved localization quality with IoU-aware query recalibration.

Universal Benchmark

To evaluate the proposed method, the authors constructed UDB10, a comprehensive universal object detection benchmark comprising 10 datasets from varied domains such as traffic, medical, and open-world scenarios. This allows for systematically assessing the generalization capabilities of DETR-based models beyond the COCO benchmark.

Figure 4: Detection results comparison, underscoring Cascade-DN-DETR's advances both on COCO and diverse datasets within UDB10.

Experimental Results

Through extensive experimentation across multiple benchmarks including COCO, UVO, and Cityscapes, Cascade-DETR demonstrated substantial improvements in performance. On UDB10, the method achieved significant performance gains of over 10 mAP in certain domains, markedly outperforming previous DETR-based architectures under stringent quality requirements. Additionally, Cascade-DETR exhibited superior convergence speed and model robustness, confirming its applicability to varied real-world detection tasks.

Conclusion

The innovation encapsulated in Cascade-DETR positions it as a formidable advancement for universal object detection. By explicitly embedding object-centric inductive bias and leveraging precise recalibration strategies, Cascade-DETR paves the way for developing vision systems capable of high-accuracy detection across heterogeneous environments. The introduction of UDB10 further fosters the exploration of DETR-based models' generalization potential, expanding their applicability in practical and diverse applications. The contributions of Cascade-DETR signify a crucial step towards bridging the gap between contemporary object detection capabilities and their deployment in comprehensive real-world scenarios.