How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey (2402.13255v3)

Abstract: Over the past two decades, research in the field of Simultaneous Localization and Mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evolution ranges from hand-crafted methods, through the era of deep learning, to more recent developments focused on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) representations. Recognizing the growing body of research and the absence of a comprehensive survey on the topic, this paper aims to provide the first comprehensive overview of SLAM progress through the lens of the latest advancements in radiance fields. It sheds light on the background, evolutionary path, inherent strengths and limitations, and serves as a fundamental reference to highlight the dynamic progress and specific challenges.

• The paper reviews the evolution of SLAM by integrating NeRF and 3DGS, highlighting the shift from implicit to explicit scene representations.
• It demonstrates how explicit 3D Gaussian Splatting delivers faster optimization and rendering while addressing challenges like memory demand and initialization sensitivity.
• The survey emphasizes issues such as catastrophic forgetting and standardization of benchmarks, calling for robust real-time processing and dynamic scene management.

Advances and Challenges in SLAM: Insights from NeRF and 3D Gaussian Splatting Techniques

Introduction to Recent SLAM Techniques

The landscape of Simultaneous Localization and Mapping (SLAM) has undergone substantial evolution with the advent of Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). These methodologies, diverging from traditional hand-crafted approaches, embody a transformative shift towards leveraging densely predicted scene representations for enhancing SLAM applications. This survey explores a broad spectrum of techniques developed in the past three years, shedding light on their inherent strengths, limitations, and the ongoing challenges faced by researchers in the domain.

Scene Representation Insights

A pivotal aspect of current SLAM solutions is the choice of scene representation, which exerts a significant influence on various facets of system performance such as mapping accuracy, rendering quality, and computational demand. Early ventures predominantly employed network-based implicit models, favoring compact and continuous scene modeling. However, such models have shown to struggle with real-time processing and tend to produce oversmoothed reconstructions. Conversely, explicit representations, particularly those based on 3DGS, have showcased faster optimization and rendering capabilities, albeit with challenges like increased memory consumption and sensitivity to initialization quality.

Confronting Catastrophic Forgetting and Real-time Constraints

Catastrophic forgetting remains a formidable challenge, especially prominent in large-scale mapping scenarios. Various strategies have been proposed to mitigate this issue, ranging from sparse sampling and replay-based keyframe buffering to the division of environments into submaps. However, these approaches introduce their own set of complexities, such as managing overlapping regions without inducing map fusion artifacts. Furthermore, achieving real-time SLAM processing confronts the computational intensity inherent to methods relying on per-pixel ray marching, presenting a considerable bottleneck for NeRF-style implementations.

Global Optimization and Dynamic Scene Management

Effective incorporation of loop closure (LC) and global bundle adjustment (BA) is paramount for ensuring trajectory accuracy. While frame-to-model methods offer compelling advancements, they often grapple with prohibitive computational overhead, reflective of the complexities in updating entire 3D models. Additionally, the dynamic nature of real-world scenes poses significant hurdles, with many systems underperforming due to the assumption of static environments, thereby necessitating advanced strategies to reliably manage dynamic objects and sensor noise.

Evaluation Protocols and Future Directions

The absence of standardized benchmarks generates evaluation inconsistencies, complicating the comparison between different SLAM systems. This underscores the need for well-defined evaluation protocols and benchmarks that could facilitate fair and consistent comparisons. Notably, the evaluation of rendering performance using training views invites concerns regarding overfitting, highlighting the urgency for exploring alternative methods for evaluation within the SLAM context.

Conclusion

This survey not only synthesizes the progress made in the field of SLAM, guided by the innovations in NeRF and 3DGS but also illuminates the gamut of challenges that persist. It underscores the critical aspects of scene representation, catastrophic forgetting, real-time processing capabilities, and the need for robust global optimization mechanisms. Furthermore, it identifies dynamic scene management, sensitivity to sensor noise, and the lack of standardized evaluation protocols as key areas warranting further exploration. As the field continues to evolve, this comprehensive survey aims to serve as a valuable resource, guiding future research towards overcoming existing limitations and unlocking new possibilities in SLAM technology.