3D gaussian splatting 代码阅读(四)：gaussian_model.py_3dgaussian mono

gaussian-splatting/scene/gaussian_model.py

这个文件主要涉及3d gaussian 模型初始化，以及论文提到dynamic density control相关。

1. init
   初始化变量为空，主要涉及到的变量。

self.active_sh_degree = 0 // 
self.max_sh_degree = sh_degree #球谐函数的阶数
self._xyz = torch.empty(0) #3d gaussian center
self._features_dc = torch.empty(0) #sh球谐参数直流分量
self._features_rest = torch.empty(0)#sh球谐函数高阶分量
self._scaling = torch.empty(0) #协方差eigen value
self._rotation = torch.empty(0) #协方差旋转四元数
self._opacity = torch.empty(0) #不透明度
self.max_radii2D = torch.empty(0) #2D投影半径
self.xyz_gradient_accum = torch.empty(0) 
self.denom = torch.empty(0)
self.optimizer = None
self.percent_dense = 0
self.spatial_lr_scale = 0
1
2
3
4
5
6
7
8
9
10
11
12
13
14

2. setup_functions
   在训练中像不透明度、旋转四元数等可能会出现优化后不合法（比如不透明度数值范围：0-1，参数优化后变成1.1；四元数优化后模长不为1），需要对这些参数加入激活函数

@property
def get_scaling(self):
    return self.scaling_activation(self._scaling)

@property
def get_rotation(self):
    return self.rotation_activation(self._rotation)

@property
def get_xyz(self):
    return self._xyz

@property
def get_features(self):
    features_dc = self._features_dc
    features_rest = self._features_rest
    return torch.cat((features_dc, features_rest), dim=1)

@property
def get_opacity(self):
    return self.opacity_activation(self._opacity)

def get_covariance(self, scaling_modifier = 1):
    return self.covariance_activation(self.get_scaling, scaling_modifier, self._rotation)

def oneupSHdegree(self):
    if self.active_sh_degree < self.max_sh_degree:
        self.active_sh_degree += 1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

可以看一下各种激活函数的设计

def setup_functions(self):
    def build_covariance_from_scaling_rotation(scaling, scaling_modifier, rotation):
        L = build_scaling_rotation(scaling_modifier * scaling, rotation)
        actual_covariance = L @ L.transpose(1, 2)
        symm = strip_symmetric(actual_covariance)
        return symm
    
    self.scaling_activation = torch.exp
    self.scaling_inverse_activation = torch.log

    self.covariance_activation = build_covariance_from_scaling_rotation

    self.opacity_activation = torch.sigmoid
    self.inverse_opacity_activation = inverse_sigmoid

    self.rotation_activation = torch.nn.functional.normalize
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

3. create_from_pcd
   从pcd中生成gaussian model
    a. 读取点云，生成xyz center
    b. 读取点的颜色，并转换为SH，这里实际上只初始化了直流的部分

features = torch.zeros((fused_color.shape[0], 3, (self.max_sh_degree + 1) ** 2)).float().cuda()
features[:, :3, 0 ] = fused_color
features[:, 3:, 1:] = 0.0
1
2
3

 c. 初始化协方差scales，这里是对每个点搜索最邻近的3个neighbor，用平均距离生成三个轴的长度，这里加了log，是因为scale的激活函数是exp

dist2 = torch.clamp_min(distCUDA2(torch.from_numpy(np.asarray(pcd.points)).float().cuda()), 0.0000001)
scales = torch.log(torch.sqrt(dist2))[...,None].repeat(1, 3)
1
2

 d. 初始化旋转四元数为单位四元数

rots = torch.zeros((fused_point_cloud.shape[0], 4), device="cuda")
rots[:, 0] = 1
1
2

 e. 初始化不透明度

opacities = inverse_sigmoid(0.1 * torch.ones((fused_point_cloud.shape[0], 1), dtype=torch.float, device="cuda"))
1

 f. 最后创建nn.Parameters，requires_grad_(True)表示这个变量是会被优化的

self._xyz = nn.Parameter(fused_point_cloud.requires_grad_(True))
self._features_dc = nn.Parameter(features[:,:,0:1].transpose(1, 2).contiguous().requires_grad_(True))
self._features_rest = nn.Parameter(features[:,:,1:].transpose(1, 2).contiguous().requires_grad_(True))
self._scaling = nn.Parameter(scales.requires_grad_(True))
self._rotation = nn.Parameter(rots.requires_grad_(True))
self._opacity = nn.Parameter(opacities.requires_grad_(True))
self.max_radii2D = torch.zeros((self.get_xyz.shape[0]), device="cuda")
1
2
3
4
5
6
7

优化的变量有：
  ● center点位置
  ● sh球谐参数
  ● scale
  ● rotation
  ● 不透明度

4. training_setup
   为每个优化变量设置不同的学习率
5. dynamic density control
   建议先阅读下面的文章
   3D Gaussian Splatting源码解读