Pixel-GS：用于3D高斯溅射的具有像素感知梯度的密度控制

Abstract 摘要         [2403.15530] Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) has demonstrated impressive novel view synthesis results while advancing real-time rendering performance. However, its efficacy heavily relies on the quality of the initial point cloud, leading to blurring and needle-like artifacts in regions with inadequate initializing points. This issue is mainly due to the point cloud growth condition, which only considers the average gradient magnitude of points from observable views, thereby failing to grow for large Gaussians that are observable for many viewpoints while many of them are only covered in the boundaries. To address this, we introduce Pixel-GS, a novel approach to take into account the number of pixels covered by the Gaussian in each view during the computation of the growth condition. We regard the covered pixel numbers as the weights to dynamically average the gradients from different views, such that the growth of large Gaussians can be prompted. As a result, points within the areas with insufficient initializing points can be grown more effectively, leading to a more accurate and detailed reconstruction. In addition, we propose a simple yet effective strategy to scale the gradient field according to the distance to the camera, to suppress the growth of floaters near the camera. Extensive qualitative and quantitative experiments confirm that our method achieves state-of-the-art rendering quality while maintaining real-time speeds, outperforming on challenging datasets such as Mip-NeRF 360 and Tanks & Temples. Code and demo are available at: https://pixelgs.github.io
3D高斯溅射（3DGS）已经展示了令人印象深刻的新颖的视图合成结果，同时提高了实时渲染性能。然而，它的有效性严重依赖于初始点云的质量，导致在初始化点不足的区域中出现模糊和针状伪影。这个问题主要是由于点云增长条件，它只考虑来自可观察视图的点的平均梯度幅度，从而无法增长对于许多视点可观察的大高斯，而其中许多仅覆盖在边界中。为了解决这个问题，我们引入了Pixel-GS，这是一种新的方法，可以在计算生长条件的过程中考虑每个视图中高斯覆盖的像素数量。我们将覆盖像素数作为权重，动态平均来自不同视图的梯度，从而可以促进大高斯的增长。 结果，可以更有效地生长初始化点不足的区域内的点，从而导致更准确和详细的重建。此外，我们提出了一个简单而有效的策略，根据到相机的距离来缩放梯度场，以抑制相机附近漂浮物的增长。大量的定性和定量实验证实，我们的方法实现了最先进的渲染质量，同时保持实时速度，在具有挑战性的数据集，如Mip-NeRF 360和坦克和寺庙。代码和演示可在：https://pixelgs.github。io

Keywords: 

Novel View Synthesis (NVS) is a fundamental problem in computer vision and graphics. Recently, 3D Gaussian Splatting (3DGS) [21] has drawn increasing attention for its explicit point-based representation of 3D scenes and real-time rendering performance.
新视图合成是计算机视觉和图形学中的一个基本问题。最近，3D高斯溅射（3DGS）[ 21]因其显式的基于点的3D场景表示和实时渲染性能而受到越来越多的关注。