Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians (2405.16544v1)

Abstract: 3D Gaussian Splatting has emerged as a powerful representation of geometry and appearance for RGB-only dense Simultaneous Localization and Mapping (SLAM), as it provides a compact dense map representation while enabling efficient and high-quality map rendering. However, existing methods show significantly worse reconstruction quality than competing methods using other 3D representations, e.g. neural points clouds, since they either do not employ global map and pose optimization or make use of monocular depth. In response, we propose the first RGB-only SLAM system with a dense 3D Gaussian map representation that utilizes all benefits of globally optimized tracking by adapting dynamically to keyframe pose and depth updates by actively deforming the 3D Gaussian map. Moreover, we find that refining the depth updates in inaccurate areas with a monocular depth estimator further improves the accuracy of the 3D reconstruction. Our experiments on the Replica, TUM-RGBD, and ScanNet datasets indicate the effectiveness of globally optimized 3D Gaussians, as the approach achieves superior or on par performance with existing RGB-only SLAM methods methods in tracking, mapping and rendering accuracy while yielding small map sizes and fast runtimes. The source code is available at https://github.com/eriksandstroem/Splat-SLAM.

• The paper introduces a dynamic 3D Gaussian representation that optimizes RGB-only SLAM by achieving globally consistent tracking and mapping.
• It employs a deformable 3D Gaussian map with loop closure and bundle adjustment to refine camera poses and proxy depth in real time.
• Experiments on datasets like TUM-RGBD reveal improved depth accuracy and map size efficiency, underscoring significant practical advancements.

Overview of Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians

The paper "Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians" introduces an advancement in RGB-only Simultaneous Localization and Mapping (SLAM) systems, leveraging 3D Gaussian Splatting for scene representation. The fundamental premise is the integration of globally optimized tracking and mapping methodologies using only RGB input, addressing the limitations observed in prior approaches using similar methodologies.

3D Gaussian Splatting (3DGS) provides a compact yet powerful representation of the scene geometry and appearance. However, previous implementations using this framework typically struggled with lower reconstruction fidelity compared to those that utilized alternative 3D representations like neural point clouds, primarily due to a lack of global map optimization and reliance on monocular depth data. This research proposes a dynamic 3D Gaussian representation adapted to keyframe pose and depth updates, improving overall accuracy.

Core Contributions

This work integrates several notable components and innovations:

1. Globally Consistent Frame-to-Frame Tracking: By adopting a frame-to-frame RGB-only tracking system, the approach achieves greater global consistency. This system uses dense optical flow within a Disparity, Scale, and Pose Optimization (DSPO) framework to adjust camera poses and disparities iteratively.
2. Deformable 3D Gaussian Map: The proposed map representation allows real-time adjustments of the 3D map via loop closure and global bundle adjustment. The method involves dynamic deformations of Gaussians to integrate updated poses and proxy depths efficiently.
3. Proxy Depth Map Utilization: To enhance 3D reconstruction quality, the paper introduces a proxy depth that synergistically combines multi-view geometric depth estimates with monocular depth predictions. This combination provides a refined depth update mechanism for areas susceptible to inaccuracies.
4. Optimized Map Storage and Performance: The iterative refinement of the map focuses on maintaining small footprint sizes while enabling efficient rendering times. This aspect is crucial for real-world applications where resources may be limited.

Methodological Insights

The paper expands on the initialization, pruning, and densification strategies of Gaussian splats to ensure efficient mapping. By leveraging a systematically structured loss function that encapsulates photometric, geometric, and scale regularization losses, the algorithm promotes rendering quality alongside geometric accuracy.

The deformation technique applied post-optimization reflects the importance of adaptability in real-world scenes, where dynamic changes are frequent, contributing to the practical applicability of the system. Monitoring global and local window trajectory optimization substantiates the enhanced tracking fidelity achieved.

Experimental Validation

The effectiveness of this method was verified against several datasets, including Replica, TUM-RGBD, and ScanNet. Experiments demonstrated superior performance over existing SLAM solutions in terms of rendering fidelity and tracking accuracy. The research highlights improvements in depth rendering errors and map size efficiency metrics. Notable numerical results include a depth L1 error of 15.05 cm on the TUM-RGBD dataset, confirming significant depth refinement.

Practical and Theoretical Implications

The implications extend across multiple dimensions within the field of SLAM. From a practical standpoint, this research enables efficient and high-quality 3D scene reconstruction and camera localization without the need for depth sensors. Theoretically, it presents a novel avenue for simultaneously optimizing frame pose, disparities, and map completeness through a deformable and adaptive 3D Gaussian representation.

Future Directions

Anticipated developments in this domain could focus on enhancing monocular depth estimation models, which are critical to the performance of systems like Splat-SLAM. Other avenues may involve integrating frame-to-model tracking, or leveraging advanced machine learning techniques to further refine the proxy depth estimation and overall SLAM robustness.

In summary, this paper significantly advances RGB-only SLAM technology through the integration of dynamic map representations and global optimization techniques. Its contributions present a compelling case for the efficacy of 3D Gaussian Splatting in practical SLAM implementations.