We propose a method to allow precise and extremely fast mesh extraction from 3D Gaussian Splatting. Gaussian Splatting has recently become very popular as it yields realistic rendering while being significantly faster to train than NeRFs. It is however challenging to extract a mesh from the millions of tiny 3D gaussians as these gaussians tend to be unorganized after optimization and no method has been proposed so far. Our first key contribution is a regularization term that encourages the gaussians to align well with the surface of the scene. We then introduce a method that exploits this alignment to extract a mesh from the Gaussians using Poisson reconstruction, which is fast, scalable, and preserves details, in contrast to the Marching Cubes algorithm usually applied to extract meshes from Neural SDFs. Finally, we introduce an optional refinement strategy that binds gaussians to the surface of the mesh, and jointly optimizes these Gaussians and the mesh through Gaussian splatting rendering. This enables easy editing, sculpting, rigging, animating, compositing and relighting of the Gaussians using traditional softwares by manipulating the mesh instead of the gaussians themselves. Retrieving such an editable mesh for realistic rendering is done within minutes with our method, compared to hours with the state-of-the-art methods on neural SDFs, while providing a better rendering quality. Our project page is the following: https://anttwo.github.io/sugar/
Introduces an efficient mesh extraction method from 3D Gaussian Splatting with fast reconstruction and high-quality rendering capabilities.
Includes a regularization term that aligns Gaussian functions with the surface geometry, increasing the accuracy of surface representation.
Implement a process that combines level set sampling, Poisson reconstruction, and a refinement stage for mesh creation and enhancement.
Enables real-time editing and fast mesh retrieval using a single GPU, outperforming traditional methods such as Neural Radiance Fields (NeRFs).
Aims to transform computer graphics workflows by providing a tool that offers performance, detail, and efficiency in real-time rendering.
The development and enhancement of 3D mesh reconstruction and rendering processes are critical for a range of applications in computer graphics, from animation to virtual reality. A recent advancement in this area is the introduction of an efficient method for mesh extraction from 3D Gaussian Splatting representations. This technology leads not only to quick mesh reconstruction but also enables the refined meshes to be rendered with exceptional quality in a fraction of the time compared to the prior state-of-the-art.
The core of the method involves a regularization term that encourages the alignment of Gaussian functions—used to represent 3D scenes—with the actual surface geometry. This alignment is crucial as traditional techniques, such as Marching Cubes, commonly struggle to construct meaningful surfaces from the 3D Gaussian representations due to their unstructured nature after optimization. By promoting the arrangement of these Gaussians to closely align with the scene surface, the process substantially improves the accuracy of the geometry captured by the Gaussians.
The process of creating a mesh starts by efficiently sampling points on a level set of the density function derived from the Gaussians, using it to run the Poisson reconstruction algorithm. This approach, scalable and rapid, enables mesh creation within minutes on a single GPU. Following mesh extraction, the method integrates a further refinement stage. This stage binds new Gaussians to the mesh, optimizing both in concert to further improve the quality of the rendering. By rendering the mesh with the bound Gaussians rather than conventional texture rendering, superior results can be achieved.
One of the compelling benefits of this method is the possibility to use common mesh editing tools for manipulating the Gaussian Splatting scene representation, thus expanding creative possibilities in graphics applications. The method, referred to as SuGaR, allows for fast retrieval of meshes from Gaussian Splatting, vastly outperforming Neural Radiance Fields (NeRFs). SuGaR has not only shown impressive outcomes in terms of render quality but also in terms of efficiency, demonstrating the ability to operate significantly faster on a single GPU.
This approach is set to revolutionize workflows in computer graphics by providing creators with a tool that is highly performant, provides detailed and editable meshes, and is efficient enough to be used within real-time rendering contexts.