OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline (2312.00343v8)

Abstract: Stereo matching aims to estimate the disparity between matching pixels in a stereo image pair, which is important to robotics, autonomous driving, and other computer vision tasks. Despite the development of numerous impressive methods in recent years, determining the most suitable architecture for practical application remains challenging. Addressing this gap, our paper introduces a comprehensive benchmark focusing on practical applicability rather than solely on individual models for optimized performance. Specifically, we develop a flexible and efficient stereo matching codebase, called OpenStereo. OpenStereo includes training and inference codes of more than 10 network models, making it, to our knowledge, the most complete stereo matching toolbox available. Based on OpenStereo, we conducted experiments and have achieved or surpassed the performance metrics reported in the original paper. Additionally, we conduct an exhaustive analysis and deconstruction of recent developments in stereo matching through comprehensive ablative experiments. These investigations inspired the creation of StereoBase, a strong baseline model. Our StereoBase ranks 1st on SceneFlow, KITTI 2015, 2012 (Reflective) among published methods and achieves the best performance across all metrics. In addition, StereoBase has strong cross-dataset generalization. Code is available at \url{https://github.com/XiandaGuo/OpenStereo}.

• The paper introduces OpenStereo, a comprehensive benchmark that integrates over ten stereo matching models for replicable research.
• It employs exhaustive ablative experiments and key insights on data augmentation, cost volume construction, and disparity refinement.
• StereoBase, the proposed baseline, achieves state-of-the-art performance on SceneFlow and KITTI datasets, demonstrating strong cross-domain robustness.

OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline

The paper, "OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline," introduces a robust benchmark and codebase designed for evaluating and advancing stereo matching methodologies. This work presents significant efforts to bridge the gap between performance enhancement and practical applicability in stereo matching tasks, which are crucial for areas such as robotics and autonomous driving.

Key Contributions

The foremost contribution of the paper is the development of OpenStereo, an extensive and flexible stereo matching codebase. This platform encompasses training and inference codes for over ten stereo matching models, positioning itself as the most complete toolbox in the field. OpenStereo facilitates replicating and surpassing the performance of originally reported models, thereby establishing a standard for consistency and comparability across various datasets like SceneFlow, KITTI 2012, and KITTI 2015.

Furthermore, through exhaustive ablative experiments, the paper introduces StereoBase, a robust baseline model that ranks first on the SceneFlow and KITTI leaderboards among published methods. StereoBase's innovative formulation and its exceptional cross-dataset generalization capability underscore its value as both a practical implementation and a stepping stone for future research.

Detailed Analysis

The paper performs a detailed examination of current stereo matching methodologies. The authors reevaluate previously established methods to determine the best configurations for practical implementation. These studies span multiple facets of stereo matching, including data augmentation techniques, feature extraction backbones, cost volume construction, and disparity refinement methods.

For instance, the analysis of data augmentation techniques revealed varied impacts on models, highlighting that combinations like RandomCrop with color augmentation yield the best results on the KITTI 2015 dataset. In terms of feature extraction, using pre-trained backbones like MobileNetV2 demonstrated substantial improvements, emphasizing the importance of leveraging existing large-scale image datasets for enhanced stereo matching tasks.

Numerical Results

The numerical results presented in this paper are compelling. StereoBase achieves an EPE of 0.34 on SceneFlow, outperforming existing methods. On the KITTI 2015 leaderboard, StereoBase surpasses state-of-the-art models with a D1-all metric of 1.44%. These competitive performances demonstrate the effectiveness of the proposed configurations and underline the necessity of a strong baseline for accurate algorithmic assessment.

Furthermore, in cross-domain evaluations, StereoBase achieves superior results compared to existing models, reinforcing its adaptability and robustness across varied datasets. This cross-domain performance is particularly essential for stereo matching applications in real-world scenarios where domain variation is frequent.

Implications and Future Directions

The introduction of OpenStereo and StereoBase has profound implications for the stereo matching community. OpenStereo provides a standardized evaluation protocol that can inspire consistent and reproducible research in stereo matching. Moreover, the strong performance of StereoBase sets a new benchmark, encouraging the development of more sophisticated and efficient models.

Future developments in AI and computer vision may leverage these tools to explore novel architectures, improved cost volume formulations, and innovative disparity refinement techniques. The extensibility of OpenStereo ensures its relevance as new datasets and stereo matching technologies emerge.

In conclusion, this paper makes substantial strides in addressing the challenges of stereo matching by offering a comprehensive benchmark and a strong baseline model. By enabling fair comparisons and pushing the envelope in stereo matching performance, it provides a pivotal platform for both academic research and practical implementations.