Gaussian splatting is a novel approach to rendering complex 3D scenes by representing them as a collection of anisotropic Gaussians in 3D space. This technique enables real-time rendering of photorealistic scenes learned from small sets of images, making it ideal for applications in gaming, virtual reality, and real-time professional visualization.
vk_gaussian_splatting is a new Vulkan-based sample that demonstrates real-time Gaussian splatting, a cutting-edge volume rendering technique that enables highly efficient representations of radiance fields. It is the latest addition to the NVIDIA DesignWorks Samples.
The NVIDIA DevTech team envisions this new sample project as a testbed to explore and compare various approaches to real-time visualization of 3D Gaussian splatting (3DGS). By evaluating various techniques and optimizations, the team aims to provide valuable insights into performance, quality, and implementation trade-offs when using the Vulkan API.
The initial implementation is based on rasterization and demonstrates two approaches for rendering splats, one leveraging mesh shaders and another using vertex shaders.
Because Gaussian splats require back-to-front sorting for correct alpha compositing, two alternative sorting methods are provided:
- A GPU-based Radix Sort implemented in a compute pipeline
- A CPU-based asynchronous sorting strategy that uses the multithreaded sort function from the C++ STL
The sample allows you to explore and experiment with multiple aspects of this rendering technique, including:
- Several visualization modes to inspect the different aspects of Gaussian splats (spherical harmonics, splats, point density, and more)
- A complete benchmarking system is available and enables profiling in real time
- More details about both the RAM and VRAM memory consumption, to understand the stream of data to render
- GPU timings for each stage of the different techniques investigated, to have an understanding of the workload and potential bottlenecks
- Graphical reports made generated with all of these numbers
This sample provides a starting point for developers looking to experiment with Gaussian splatting rendering techniques and Vulkan-based optimizations.
To start exploring real-time rendering of neural radiance fields, check out the nvpro-samples/vk_gaussian_splatting GitHub repo.
SAGECONNECT TECHNOLOGY GROUP CO., LIMITED
Sageconnect is a reliable provider of IT network system integration and solutions for global enterprise customers.
