解析溅射：通过解析积分进行抗混叠的3D高斯溅射_analytic-splatting: anti-aliased 3d gaussian splat

Abstract 摘要    Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration

The 3D Gaussian Splatting (3DGS) gained its popularity recently by combining the advantages of both primitive-based and volumetric 3D representations, resulting in improved quality and efficiency for 3D scene rendering. However, 3DGS is not alias-free, and its rendering at varying resolutions could produce severe blurring or jaggies. This is because 3DGS treats each pixel as an isolated, single point rather than as an area, causing insensitivity to changes in the footprints of pixels. Consequently, this discrete sampling scheme inevitably results in aliasing, owing to the restricted sampling bandwidth. In this paper, we derive an analytical solution to address this issue. More specifically, we use a conditioned logistic function as the analytic approximation of the cumulative distribution function (CDF) in a one-dimensional Gaussian signal and calculate the Gaussian integral by subtracting the CDFs. We then introduce this approximation in the two-dimensional pixel shading, and present Analytic-Splatting, which analytically approximates the Gaussian integral within the 2D-pixel window area to better capture the intensity response of each pixel. Moreover, we use the approximated response of the pixel window integral area to participate in the transmittance calculation of volume rendering, making Analytic-Splatting sensitive to the changes in pixel footprint at different resolutions. Experiments on various datasets validate that our approach has better anti-aliasing capability that gives more details and better fidelity.
3D高斯溅射（3DGS）结合了连续和体3D表示的优点，提高了3D场景绘制的质量和效率。然而，3DGS不是无锯齿的，它在不同分辨率下的渲染可能会产生严重的模糊或锯齿。这是因为3DGS将每个像素视为孤立的单个点而不是区域，从而导致对像素轮廓线的变化不敏感。因此，由于有限的采样带宽，这种离散采样方案不可避免地导致混叠。在本文中，我们推导出一个解析解来解决这个问题。更具体地说，我们使用条件逻辑函数作为一维高斯信号中的累积分布函数（CDF）的解析近似，并通过减去CDF来计算高斯积分。 然后，我们在二维像素着色中引入这种近似，并提出了解析溅射，它解析地近似2D像素窗口区域内的高斯积分，以更好地捕获每个像素的强度响应。此外，我们使用像素窗口积分面积的近似响应来参与体绘制的透射率计算，使得Analytic-Splatting对不同分辨率下像素足迹的变化敏感。在不同数据集上的实验表明，该方法具有更好的抗锯齿能力，提供了更多的细节和更好的保真度。

Keywords: 

Novel view synthesis of a scene captured from multiple images has achieved great progress due to the rapid advancements of neural rendering. As a prominent representative, Neural Radiance Field (NeRF) [23] models the scene using a neural volumetric representation, enabling photorealistic rendering of novel views via ray marching. Ray marching trades off rendering efficiency with quality, and subsequent works [33, 8, 24] are proposed to have a better quality-efficiency balance. More recently, 3D Gaussian Splatting (3DGS) [16] proposes a GPU-friendly differentiable rasterization pipeline that incorporates an explicit point-based representation, achieving high-quality and real-time renderings for novel view synthesis. In contrast to ray marching in NeRF, which renders a pixel by accumulating the radiance of samples along the ray that intersects the image plane at the pixel, 3DGS employs a forward-mapping technique that can be rasterized very efficiently. Specifically, 3DGS represents the scene as a set of anisotropic 3D Gaussians with scene properties; when rendering a pixel, 3DGS orders and projects these 3D Gaussians onto the image plane as 2D Gaussians, and then queries values and scene properties associated with the Gaussians that have overlaps with the pixel, and finally shades the pixel by cumulatively compositing these queried values and properties.
由于神经绘制技术的快速发展，从多幅图像中获取场景的视图合成已经取得了很大的进展。作为一个突出的代表，神经辐射场（NeRF）[ 23]使用神经体积表示对场景进行建模，通过光线行进实现新颖视图的照片级真实感渲染。光线行进在渲染效率与质量之间进行权衡，并且随后的作品[ 33，8，24]被提议具有更好的质量-效率平衡。最近，3D高斯溅射（3DGS）[ 16]提出了一种GPU友好的可微分光栅化流水线，该流水线包含显式的基于点的表示，实现了高质量和实时渲染，用于新颖的视图合成。与NeRF中的光线行进（其通过沿着在像素处与图像平面相交的光线沿着累积样本的辐射率来渲染像素）相比，3DGS采用可以非常有效地光栅化的前向映射技术。 具体地，3DGS将场景表示为具有场景属性的各向异性3D高斯的集合;当渲染像素时，3DGS将这些3D高斯排序并投影到图像平面上作为2D高斯，然后查询与像素重叠的高斯相关联的值和场景属性，最后通过累积合成这些查询的值和属性来对像素进行着色。

3DGS works for scene representation learning and novel view synthesis at constant resolutions; however, its performance degrades greatly either when the multi-view images are captured at varying distances, or when the novel view to be rendered has a resolution different from those of the captured images. The main reason is that the footprint
3DGS适用于场景表示学习和恒定分辨率下的新视图合成;然而，当在不同距离处捕获多视图图像时，或者当要渲染的新视图具有与捕获图像不同的分辨率时，其性能大大降低。主要原因是，11The footprint is defined as the ratio between the pixel window area in screen space and its covered Gaussian signals region in the world space.
覆盖区定义为屏幕空间中的像素窗口面积与其在世界空间中覆盖的高斯信号区域之间的比率。 of the pixel changes at different resolutions and 3DGS is insensitive to such changes since it treats each pixel as an isolated point (i.e. merely pixel center) when retrieving the corresponding Gaussian values; Fig. 1a gives an illustration. As a result, 3DGS could produce significant artifacts (e.g. blurry or jaggies) especially when pixel footprints change drastically (e.g. synthesizing novel views with zooming-in and zooming-out effects).
在不同分辨率下像素的变化，3DGS对这种变化不敏感，因为它在检索相应的高斯值时将每个像素视为孤立点（即仅仅是像素中心）;图1a给出了说明。因此，3DGS可能会产生明显的伪影（例如模糊或锯齿状），特别是当像素足迹急剧变化时（例如合成具有放大和缩小效果的新视图）。

By delving into the details, we know that 3DGS represents a continuous signal in the image space as a set of �-blended 2D Gaussians, and the pixel shading is a process of integrating the signal response within each pixel area; artifacts in 3DGS are caused by the limited sampling bandwidth for the Gaussians that retrieves erroneous responses, especially when the pixel footprint changes drastically. It is possible to increase sampling bandwidth (i.e. via super sampling) or use prefiltering techniques to alleviate this problem; for example, Mip-Splatting [36] employs the prefiltering technique and presents a hybrid filtering mechanism to regularize the high-frequency components of 2D and 3D Gaussians to achieve anti-aliasing. While Mip-Splatting overcomes most aliasing in 3DGS, it is limited in capturing details and synthesizes over-smoothing results. Consequently, solving the integral of Gaussian signals within the pixel window area as intensity responses is crucial for both anti-aliasing and capturing details.
通过深入研究细节，我们知道3DGS将图像空间中的连续信号表示为一组 � 混合的2D高斯，并且像素着色是在每个像素区域内整合信号响应的过程; 3DGS中的伪影是由高斯的有限采样带宽引起的，该高斯检索错误的响应，特别是当像素足迹急剧变化时。可以增加采样带宽（即通过超采样）或使用预滤波技术来缓解这个问题;例如，Mip-Splatting [ 36]采用预滤波技术并提出混合滤波机制来正则化2D和3D高斯的高频分量以实现抗混叠。虽然Mip-Splatting克服了3DGS中的大多数锯齿，但它在捕捉细节和合成过度平滑结果方面受到限制。 因此，求解像素窗口区域内高斯信号的积分作为强度响应对于抗混叠和捕获细节都是至关重要的。

In this paper, we revisit pixel shading in 3DGS and introduce an analytic approximation of the window integral response of Gaussian signals for anti-aliasing. Rather than discrete sampling in 3DGS and prefiltering in Mip-Splatting, we analytically approximate the integral within each pixel area as shown in Fig. 1b. We term our method as Analytic-Splatting. Compared with Mip-Splatting, which approximates the pixel window as a 2D Gaussian low-pass filter, our proposed method does not suppress the high-frequency components in Gaussian signals and can better preserve high-quality details. Experiments show that our method removes the aliasing existing in 3DGS and other methods while synthesizing more details with better fidelity. We summarize our contributions as follows.
在本文中，我们重新审视3DGS中的像素着色，并介绍了一个解析近似的窗口积分响应的高斯信号抗混叠。与3DGS中的离散采样和Mip—Splatting中的预滤波不同，我们解析地近似每个像素区域内的积分，如图1b所示。我们称我们的方法为分析飞溅。与将像素窗口近似为二维高斯低通滤波器的Mip—Splatting相比，该方法不抑制高斯信号中的高频成分，能更好地保留高质量的细节。实验结果表明，该方法在消除3DGS和其他方法中存在的混叠的同时，合成了更多的细节，具有更好的保真度。我们将我们的贡献总结如下。

• • 

  We revisit the causes of aliasing in 3D Gaussian Splatting from the perspective of signal window response and derive an analytic approximation of the window response for Gaussian signals;

  
  ·我们从信号窗口响应的角度重新审视了3D高斯溅射中混叠的原因，并推导出高斯信号的窗口响应的解析近似;
• • 

  Based on the derivation, we present Analytic-Splatting that improves the pixel shading in 3D Gaussian Splatting to achieve anti-aliasing and better detail fidelity.

  
  ·在此基础上，我们提出了一种改进3D高斯溅射像素着色的解析溅射算法，以实现抗锯齿和更好的细节保真度。
• • 

  Our experiments on challenging datasets demonstrate the superiority of our method to other approaches in terms of anti-aliasing and synthesizing results.

  
  ·我们在具有挑战性的数据集上的实验证明了我们的方法在抗锯齿和合成结果方面优于其他方法。

Neural Rendering. Recently, neural rendering techniques exemplified by Neural Radiance Field (NeRF) [23] have achieved impressive results in novel view synthesis, and further enhanced several advanced tasks [30, 34, 21, 14, 22, 25]. Nevertheless, the backward-mapping volume rendering used in NeRF hinders the real-time rendering performance, restricting the application prospects of NeRF. While several NeRF variants adopt efficient sampling strategies [35, 24, 19] or use explicit/hybrid representations [8, 29, 5, 9] with higher capacities, they still suffer from the tough sampling problem and struggle with real-time rendering. To overcome these limitations, 3DGS [16] employs forward mapping volume rendering technology and implements GPU-friendly tile-based rasterization to achieve real-time rendering and impressive rendering results. Due to its real-time rendering capability and impressive rendering performance, 3DGS has been widely used in advanced tasks such as Human/Avatar modeling [27, 12, 38, 40], surface reconstruction [10, 6], inverse rendering [20, 15, 28], physical simulation [32, 7], etc. Although rasterization makes 3DGS avoid tough sampling problems along rays and achieve promising results, it also introduces aliasing caused by restricted sampling bandwidth when shading pixels using 2D Gaussians. And the aliasing will be noticeable when the pixel footprint changes drastically (e.g. zooming in and out). In this paper, we study the errors introduced by the discrete sampling scheme used in 3DGS and introduce our advanced resolution.
神经渲染。最近，以神经辐射场（NeRF）[ 23]为例的神经渲染技术在新颖的视图合成中取得了令人印象深刻的结果，并进一步增强了几个高级任务[ 30，34，21，14，22，25]。然而，NeRF中采用的后向映射体绘制方法阻碍了实时绘制性能，限制了NeRF的应用前景。虽然几个NeRF变体采用了有效的采样策略[ 35，24，19]或使用具有更高容量的显式/混合表示[ 8，29，5，9]，但它们仍然存在坚韧的采样问题，并且难以实时渲染。为了克服这些限制，3DGS [ 16]采用前向映射体绘制技术，并实现GPU友好的基于瓦片的光栅化，以实现实时渲染和令人印象深刻的渲染结果。 由于其实时渲染能力和令人印象深刻的渲染性能，3DGS已被广泛用于高级任务，如人类/化身建模[ 27，12，38，40]，表面重建[ 10，6]，逆渲染[ 20，15，28]，物理模拟[ 32，7]，虽然光栅化使得3DGS避免了沿沿着射线的坚韧采样问题并获得了有希望的结果，但是当使用2D高斯对像素进行着色时，光栅化也引入了由有限的采样带宽引起的混叠。当像素占用空间急剧变化时（例如放大和缩小），混叠将是明显的。在本文中，我们研究了3DGS中使用的离散采样方案所引入的误差，并介绍了我们的先进的解决方案。

Anti-Aliasing. Aliasing is the phenomenon of overlapping frequency components when the discrete sampling rate is below the Nyquist rate. Anti-aliasing is critical for rendering high-fidelity results, which has been extensively explored in the computer graphics and vision community [1, 31, 18]. In the neural rendering context, MipNeRF [2] and Zip-NeRF [4] pioneer the use of prefiltering and multi-sampling to address the aliasing issue in neural radiance fields (NeRF). Recent works also explored the anti-aliased NeRF for unbounded scenes [3], efficient reconstruction [13], and surface reconstruction [39]. All these works are built upon the backward-mapping volume rendering to consider the pixel footprint, by replacing the original ray-casting with cone-casting. However, the backward-mapping volume rendering is too computationally expensive to achieve real-time rendering. On the other hand, 3DGS [16] introduced real-time forward-mapping volume rendering but suffers from aliasing artifacts due to the discrete sampling during shading pixels using projected Gaussians. To this end, Mip-Splatting [36] presents a hybrid filtering mechanism to restrict the high-frequency components of 2D and 3D Gaussians to achieve anti-aliasing. Nevertheless, this low-pass filtering strategy hinders the capability to preserve high-quality details. In contrast, our approach introduces an analytic approximation of the integral within the pixel area to better capture the intensity response of each pixel, harvesting both aliasing-free and detail-preserving rendering results.
抗锯齿。混叠是当离散采样率低于奈奎斯特速率时频率分量重叠的现象。抗锯齿对于渲染高保真结果至关重要，这在计算机图形和视觉社区中得到了广泛的探索[ 1，31，18]。在神经渲染上下文中，MipNeRF [ 2]和Zip-NeRF [ 4]率先使用预滤波和多采样来解决神经辐射场（NeRF）中的混叠问题。最近的工作还探索了无界场景的抗锯齿NeRF [ 3]，有效重建[ 13]和表面重建[ 39]。所有这些工作都是建立在向后映射体绘制考虑像素足迹，取代原来的光线投射与锥投射。然而，向后映射体绘制的计算量太大，无法实现实时绘制。 另一方面，3DGS [16]引入了实时前向映射体绘制，但由于使用投影高斯对像素进行着色期间的离散采样而遭受混叠伪影。为此，Mip—Splatting [36]提出了一种混合滤波机制，以限制2D和3D高斯的高频分量，从而实现抗混叠。然而，这种低通滤波策略阻碍了保持高质量细节的能力。相比之下，我们的方法引入了像素区域内积分的解析近似，以更好地捕获每个像素的强度响应，从而获得无混叠和细节保留的渲染结果。

In this section, we give the technical background necessary for presentation of our proposed method.
在本节中，我们给予介绍我们提出的方法所需的技术背景。

3D Gaussian Splatting (3DGS) explicitly represents 3D scene as a set of points {