Emergent Mind

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM

(2312.02126)

Published Dec 4, 2023 in cs.CV , cs.AI , and cs.RO

Abstract

Dense simultaneous localization and mapping (SLAM) is crucial for robotics and augmented reality applications. However, current methods are often hampered by the non-volumetric or implicit way they represent a scene. This work introduces SplaTAM, an approach that, for the first time, leverages explicit volumetric representations, i.e., 3D Gaussians, to enable high-fidelity reconstruction from a single unposed RGB-D camera, surpassing the capabilities of existing methods. SplaTAM employs a simple online tracking and mapping system tailored to the underlying Gaussian representation. It utilizes a silhouette mask to elegantly capture the presence of scene density. This combination enables several benefits over prior representations, including fast rendering and dense optimization, quickly determining if areas have been previously mapped, and structured map expansion by adding more Gaussians. Extensive experiments show that SplaTAM achieves up to 2x superior performance in camera pose estimation, map construction, and novel-view synthesis over existing methods, paving the way for more immersive high-fidelity SLAM applications.

Overview

The paper 'Splat-SLAM: Dense RGB-D SLAM via 3D Gaussian Splatting' introduces a novel method for Simultaneous Localization and Mapping, utilizing 3D Gaussian splatting to improve accuracy, memory efficiency, and real-time performance.
The authors present several key innovations, including the use of 3D Gaussian representations for scene geometry, visibility masks for enhanced mapping accuracy, and empirically chosen thresholds for optimal performance.
Empirical results highlight the method’s strengths in terms of accuracy versus efficiency, novel-view synthesis, and pose estimation, demonstrating potential practical applications for resource-constrained devices and inspiring future research directions in SLAM and computer vision.

Splat-SLAM: Dense RGB-D SLAM via 3D Gaussian Splatting

"Splat-SLAM: Dense RGB-D SLAM via 3D Gaussian Splatting" explores an innovative approach in the domain of Simultaneous Localization and Mapping (SLAM) by leveraging 3D Gaussian splatting for dense reconstruction. This paper proposes a unique method tailored to optimize the trade-offs between accuracy, memory efficiency, and real-time performance in SLAM applications.

The primary contribution of this work lies in utilizing 3D Gaussian splatting for RGB-D SLAM, an area that traditionally relies on more computationally intensive techniques like surfels or voxel grids. The authors advocate for the efficacy of their method through comprehensive experiments and evaluations conducted on standard RGB-D SLAM benchmarks.

Methodology and Experimental Details

The proposed approach involves several key innovations:

3D Gaussian Representation: The authors employ 3D Gaussian splats to represent the scene geometry. Each Gaussian is parameterized by its mean, covariance, and intensity. This representation facilitates continuous surface modeling and supports efficient rendering through alpha compositing.
Visibility Masks: To enhance mapping accuracy, visibility masks are incorporated, helping in the utilization of only the visible parts of the scene, thus refining pose estimation. This simple yet effective strategy contributes significantly to the robustness of the proposed method.
Empirical Thresholds: Specific thresholds, such as 50 times the median depth error, are chosen empirically to optimize performance under various conditions. The chosen thresholds are validated through visual and quantitative analysis, ensuring their applicability in real-world scenarios.
Baselines and Comparisons: The authors ensure a fair comparison by evaluating their method against established baselines using metrics such as Absolute Trajectory Error (ATE), while also addressing other relevant qualitative metrics like novel-view synthesis (NVS).

Key Results

The empirical results underscore the strengths of Splat-SLAM:

Accuracy vs. Efficiency: Ablation studies reveal that isotropic Gaussians offer competitive accuracy with notable improvements in speed and memory efficiency. Specifically, isotropic Gaussians achieve an ATE of 0.57 cm, slightly worse than anisotropic Gaussians (0.55 cm), but with substantial reductions in memory usage (57.5% of anisotropic) and computational time (83.3% of anisotropic).
Novel-View Synthesis: In the task of novel-view synthesis, the proposed method obtains improved PSNR scores when compared to its predecessors. For instance, on the ScanNet++ dataset, the Splat-SLAM method achieves an average PSNR of 24.41 dB, close to the 3D Gaussian Splatting (3DGS) with ground-truth poses (24.45 dB).
Pose Estimation: The incorporation of methods such as DROID-SLAM and ORB-SLAM3 for evaluation on the Replica dataset confirms the robustness of the proposed system across diverse environments.

Implications and Future Work

This research has both practical and theoretical implications for the field of SLAM:

Practical Application: The efficiency in computation and memory usage makes Splat-SLAM suitable for deployment in real-time applications on devices with constrained resources, such as mobile phones and drones. The inclusion of an online demo using RGB-D data from an iPhone exemplifies this practical applicability.
Theoretical Advances: The use of 3D Gaussians instead of traditional 2D surfels presents a shift in how scene geometry can be represented and optimized. This methodological pivot may inspire new directions in both SLAM and broader computer vision research.
Open Challenges and Speculation: Future developments may focus on extending the compatibility of Splat-SLAM with batched rasterization techniques, which could significantly enhance joint optimization capabilities. Additionally, exploring the integration with other neural rendering methods could further improve the fidelity and efficiency of SLAM systems.

In conclusion, "Splat-SLAM: Dense RGB-D SLAM via 3D Gaussian Splatting" represents a significant methodological contribution, showcasing how 3D Gaussian splatting can be effectively employed in SLAM. The findings and innovations presented are invaluable for researchers and practitioners aiming to enhance the performance and efficiency of SLAM systems.