Skip to content

3D Gaussian Splatting for Real-Time Radiance Field Rendering

Authors: Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis

Published: 2023 (Journal Paper)

Source: ACM Transactions on Graphics (TOG)

Algorithm: 3DGS

arXiv: 2308.04079

DOI: 10.1145/3592433

Summary

Seminal paper that introduced 3D Gaussian Splatting, a rendering technique that optimizes the pose, shape, transparency, and view-dependent optic properties (modeled with spherical harmonics) of a large collection of 3D Gaussians (ellipsoidal distributions) to reconstruct a ground truth represented by multiple 2D image views (typically collected by taking photos or video from multiple views around an object or scene). The trained Gaussians are rendered via differentiable tile-based rasterization, enabling high-quality novel view synthesis at real-time frame rates.

Abstract

Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.

Primary

Paper arxiv.org

Standard

arXiv Abstract 2308.04079 arXiv PDF 2308.04079 arXiv HTML 2308.04079 DOI 10.1145/3592433

Alternate

dl.acm.org dl.acm.org GitHub: graphdeco-inria/gaussian-splatting github.com en.wikipedia.org en.wikipedia.org youtube.com youtube.com

Tags

  • 3DGS

  • Gaussian splat

  • Gaussian splatting

  • 3D scene

  • Rendering

  • Real-time

  • GPU

  • Ellipsoid