devbox
2023面试高手
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

基于深度学习的三维重建（一）：三维重建简介、patchmatchNet环境部署、用colmap如何测试自己的数据集

作者：2023面试高手 | 2024-02-18 01:58:51

赞

踩

基于深度学习的三维重建

目录

1.什么是三维重建

2.MVS是什么

3.传统MVS的局限性和为什么基于深度学习的MVS性能好于传统三维重建

4.基础概念

5. patchmatchNet环境配置

6.如何测试自己的数据集（位姿计算）

6.1 colmap导出位姿

6.2 将colmap位姿转换成MVS读取的数据格式

1.什么是三维重建

        用相机拍摄真实世界的物体、场景，并通过计算机视觉技术进行处理，从而得到物体的三维模型。英文术语:3D Reconstruction。
        涉及的主要技术有：多视图立体几何、深度图估计、点云处理、网格重建和优化、纹理贴图、马尔科夫随机场、图割等。
        是增强现实(AR）、混合现实(MR）、机器人导航、自动驾驶等领域的核心技术之一。

2.MVS是什么

        用RGB-D信息重建3D几何的模型，输入的话是一系列的RGB-D照片，这些照片会有一些重合部分，我们计算这些照片的位姿信息（每个拍摄的相机角度的位置），我们就知道了每帧间的位置关系，然后对它进行一个3D模型的重建，最后将材质信息给它贴上去（纹理贴图，将RGB-D信息贴到几何模型上）。

        这里的位姿信息获得是通过SLAM或者SFM来做的，我们就可以得到它的深度图进行点云融合等操作。

3.传统MVS的局限性和为什么基于深度学习的MVS性能好于传统三维重建

        MVS重建我们是基于RGB-D信息做的，也就是说我们恢复3D信息是通过立体匹配等方法进行深度恢复，如果RGB信息出现大面积的单色情况（如下图）没有纹理信息的或者是透明的重复纹理的，那么我们很难进行特征点匹配，也就没有办法通过立体匹配进行深度恢复了，最终导致失败。

        在深度学习中，我们通过大量的数据学习一些信息规律和信息。

        但其也有一定的缺点，比如依赖显存和内存，依赖大数据，难以重建高分辨率的模型。

4.基础概念

        ①深度图(depth)/视差图(disparity)

        a.深度图：场景中每个点到相机的距离
        b.视差图：同一个场景在两个相机下成像的像素的位置偏差dis
        c.两者关系: depth = \frac{b_f}{dis}，是三维信息的一种表示方式。

        深度图存储的就是相机坐标系下的Z值

        ②三维点云：

        a.三维点云是某个坐标系下的点的数据集。
        b.包含了丰富的信息，包括三维坐标XYZ、颜色RGB等信息。

        ③三维网格：

        由物体的邻接点云构成的多边形组成的，通常由三角形、四边形或者其它的简单凸多边形组成。

       ④纹理贴图模型：

        带有颜色信息的三维网格模型。
        所有的颜色信息存储在一张纹理图上，显示时根据每个网格的纹理坐标和对应的纹理图进行渲染得到高分辨率的彩色模型。

        ⑤相机模型：

5. patchmatchNet环境配置

        利用Anaconda配置虚拟环境，python版本为3.8。如果有GPU的话cuda推荐版本为11.3，我是3080显卡，安装的cuda版本为11.3，pytorch版本为1.11.0，可顺利跑通！

         下载patchmatchNet源码：

PatchNet Git地址https://github.com/FangjinhuaWang/PatchmatchNet

        里面有个requirements.txt

        一键安装：pip install -r requirements.txt

        到这里我们就安装成功了，测试一下：

        先0下载数据集：

        下载DTU数据集 dtu.zip，在项目文件夹新建data文件夹，解压到data文件夹中。

        在代码路径下打开终端，激活conda环境，运行eval.sh，如果没有gpu，运行eval_cpu.sh。

        运行结果保存在当前目录下./outputs下。

        如果显卡性能不好或者CPU跑性能过慢的话，我们可以改变默认的分辨率大小，在eval.py中，main函数中，img_wh = (800,600)，可以改为原来的一半。同时也要改dataset/dtu_yao_eval.py文件，MvsDataset类中更改图像分辨率。

        最终重建的结果如下，效果还是不错滴！

6.如何测试自己的数据集（位姿计算）

6.1 colmap导出位姿

        比如我们用手机拍摄了一组照片，要对其进行三维重建，应该怎么做呢？

        我们上面说到，输入到patchmatchNet网络中的参数是一组照片以及它们的深度信息（位姿+深度范围或者稀疏点云），因此我们主要做的就是数据格式的转换。

        第一步就是位姿的计算，无论是sfm或者SLAM除了输出位姿（R,t）以外都会有稀疏点的输出，并且告诉哪些稀疏点会被哪些相机和关键帧看到，我们就是用这些信息完成重建。

        一、数据：

        数据要求：图的质量要好不能模糊遮挡、帧之间要有大量的重合区域。（弱纹理，高反光效果不好）

        二、位姿计算：使用colmap计算位姿，也可以使用别的比如OpenMVG，Slam这些需要自己写转换脚本参考colmap_input.py。colmap git链接https://github.com/colmap/colmap/releases        在windows下的使用，下载这两个其中一个，

        双击：

        新建一个工程：

        Database是新建的数据库名称，Images是包含图片的文件夹。

        可以看到桌面已经有这个工程了。

        下一步就是特征提取，

        点击提取

        特征匹配完成！

        重建，点击开始重建。

        红色的就是每帧相机的位置，黑色的就是稀疏的地图点。

        现在我们就得到了哪些相机可以观测到哪些点、相机的位姿信息、相机的内参等。

        我们就可以用地图点计算深度范围计算初始的深度图都是可以的，接下来我们导出信息。FIle/Import model as text：

        camera.txt里面是相机的参数：对应相机ID、相机属性、图像大小、相机内参

        images.txt：图像ID、平移四元数、旋转信息、哪个相机照的这个图像、图像的名称

         POINTS2D[] as (X, Y, POINT3D_ID)：投影坐标

        points3D.txt：3D点的ID、3D点坐标、3D点的RGB信息、错误信息、可以被哪张图片看到，该点在该张图片的索引是多少。

6.2 将colmap位姿转换成MVS读取的数据格式

        转换成MVS格式直接由DataLoader调用即可：

        我们看怎么实现：colmap_input.py 我在源码中进行了标注

  1. #!/usr/bin/env python
  2.  
  3.  
  4. from __future__ import print_function
  5.  
  6. import collections
  7. import struct
  8. import numpy as np
  9. import multiprocessing as mp
  10. import os
  11. import argparse
  12. import shutil
  13. import cv2
  14.  
  15.  
  16. #============================ read_model.py ============================#
  17. CameraModel = collections.namedtuple(
  18.     "CameraModel", ["model_id", "model_name", "num_params"])
  19. Camera = collections.namedtuple(
  20.     "Camera", ["id", "model", "width", "height", "params"])
  21. BaseImage = collections.namedtuple(
  22.     "Image", ["id", "qvec", "tvec", "camera_id", "name", "xys", "point3D_ids"])
  23. Point3D = collections.namedtuple(
  24.     "Point3D", ["id", "xyz", "rgb", "error", "image_ids", "point2D_idxs"])
  25.  
  26. class Image(BaseImage):
  27.     def qvec2rotmat(self):
  28.         return qvec2rotmat(self.qvec)
  29.  
  30.  
  31. CAMERA_MODELS = {
  32.     CameraModel(model_id=0, model_name="SIMPLE_PINHOLE", num_params=3),
  33.     CameraModel(model_id=1, model_name="PINHOLE", num_params=4),
  34.     CameraModel(model_id=2, model_name="SIMPLE_RADIAL", num_params=4),
  35.     CameraModel(model_id=3, model_name="RADIAL", num_params=5),
  36.     CameraModel(model_id=4, model_name="OPENCV", num_params=8),
  37.     CameraModel(model_id=5, model_name="OPENCV_FISHEYE", num_params=8),
  38.     CameraModel(model_id=6, model_name="FULL_OPENCV", num_params=12),
  39.     CameraModel(model_id=7, model_name="FOV", num_params=5),
  40.     CameraModel(model_id=8, model_name="SIMPLE_RADIAL_FISHEYE", num_params=4),
  41.     CameraModel(model_id=9, model_name="RADIAL_FISHEYE", num_params=5),
  42.     CameraModel(model_id=10, model_name="THIN_PRISM_FISHEYE", num_params=12)
  43. }
  44. CAMERA_MODEL_IDS = dict([(camera_model.model_id, camera_model) \
  45.                          for camera_model in CAMERA_MODELS])
  46.  
  47. def read_next_bytes(fid, num_bytes, format_char_sequence, endian_character="<"):
  48.     """Read and unpack the next bytes from a binary file.
  49.     :param fid:
  50.     :param num_bytes: Sum of combination of {2, 4, 8}, e.g. 2, 6, 16, 30, etc.
  51.     :param format_char_sequence: List of {c, e, f, d, h, H, i, I, l, L, q, Q}.
  52.     :param endian_character: Any of {@, =, <, >, !}
  53.     :return: Tuple of read and unpacked values.
  54.     """
  55.     data = fid.read(num_bytes)
  56.     return struct.unpack(endian_character + format_char_sequence, data)
  57.  
  58.  
  59. def read_cameras_text(path):
  60.     """
  61.     see: src/base/reconstruction.cc
  62.         void Reconstruction::WriteCamerasText(const std::string& path)
  63.         void Reconstruction::ReadCamerasText(const std::string& path)
  64.     """
  65.     cameras = {}
  66.     with open(path, "r") as fid:
  67.         while True:
  68.             line = fid.readline()
  69.             if not line:
  70.                 break
  71.             line = line.strip()
  72.             if len(line) > 0 and line[0] != "#":
  73.                 elems = line.split()
  74.                 camera_id = int(elems[0])
  75.                 # camera 类型
  76.                 model = elems[1]
  77.                 width = int(elems[2])
  78.                 height = int(elems[3])
  79.                 # 相机的参数
  80.                 params = np.array(tuple(map(float, elems[4:])))
  81.                 cameras[camera_id] = Camera(id=camera_id, model=model,
  82.                                             width=width, height=height,
  83.                                             params=params)
  84.     return cameras
  85.  
  86.  
  87. def read_cameras_binary(path_to_model_file):
  88.     """
  89.     see: src/base/reconstruction.cc
  90.         void Reconstruction::WriteCamerasBinary(const std::string& path)
  91.         void Reconstruction::ReadCamerasBinary(const std::string& path)
  92.     """
  93.     cameras = {}
  94.     with open(path_to_model_file, "rb") as fid:
  95.         num_cameras = read_next_bytes(fid, 8, "Q")[0]
  96.         print("num of cameras")
  97.         print(num_cameras)
  98.         for camera_line_index in range(num_cameras):
  99.             camera_properties = read_next_bytes(
  100.                 fid, num_bytes=24, format_char_sequence="iiQQ")
  101.             camera_id = camera_properties[0]
  102.             print("camera id")
  103.             print(camera_id)
  104.  
  105.             model_id = camera_properties[1]
  106.             # print("model id")
  107.             # print(model_id)
  108.             model_name = CAMERA_MODEL_IDS[camera_properties[1]].model_name
  109.             width = camera_properties[2]
  110.             height = camera_properties[3]
  111.             num_params = CAMERA_MODEL_IDS[model_id].num_params
  112.             params = read_next_bytes(fid, num_bytes=8*num_params,
  113.                                      format_char_sequence="d"*num_params)
  114.             cameras[camera_id] = Camera(id=camera_id,
  115.                                         model=model_name,
  116.                                         width=width,
  117.                                         height=height,
  118.                                         params=np.array(params))
  119.         assert len(cameras) == num_cameras
  120.     return cameras
  121.  
  122.  
  123. def read_images_text(path):
  124.     """
  125.     see: src/base/reconstruction.cc
  126.         void Reconstruction::ReadImagesText(const std::string& path)
  127.         void Reconstruction::WriteImagesText(const std::string& path)
  128.     """
  129.     images = {}
  130.     with open(path, "r") as fid:
  131.         while True:
  132.             # 一行一行读
  133.             line = fid.readline()
  134.             if not line:
  135.                 break
  136.             line = line.strip()
  137.             # 前面几行有带#的注释
  138.             if len(line) > 0 and line[0] != "#":
  139.                 elems = line.split()
  140.                 image_id = int(elems[0])
  141.                 # 四元数（旋转矩阵转换的四元数）
  142.                 qvec = np.array(tuple(map(float, elems[1:5])))
  143.                 # 内参t
  144.                 tvec = np.array(tuple(map(float, elems[5:8])))
  145.                 # camera ID
  146.                 camera_id = int(elems[8])
  147.                 # 图片名称
  148.                 image_name = elems[9]
  149.                 elems = fid.readline().split()
  150.                 # 特征点在图像中的2D坐标
  151.                 xys = np.column_stack([tuple(map(float, elems[0::3])),
  152.                                        tuple(map(float, elems[1::3]))])
  153.                 point3D_ids = np.array(tuple(map(int, elems[2::3])))
  154.                 images[image_id] = Image(
  155.                     id=image_id, qvec=qvec, tvec=tvec,
  156.                     camera_id=camera_id, name=image_name,
  157.                     xys=xys, point3D_ids=point3D_ids)
  158.     return images
  159.  
  160.  
  161. def read_images_binary(path_to_model_file):
  162.     """
  163.     see: src/base/reconstruction.cc
  164.         void Reconstruction::ReadImagesBinary(const std::string& path)
  165.         void Reconstruction::WriteImagesBinary(const std::string& path)
  166.     """
  167.     images = {}
  168.     with open(path_to_model_file, "rb") as fid:
  169.         num_reg_images = read_next_bytes(fid, 8, "Q")[0]
  170.         # print("img num")
  171.         # print(num_reg_images)
  172.         for image_index in range(num_reg_images):
  173.             binary_image_properties = read_next_bytes(
  174.                 fid, num_bytes=64, format_char_sequence="idddddddi")
  175.             image_id = binary_image_properties[0]
  176.             # print("image_id")
  177.             # print(image_id)
  178.             qvec = np.array(binary_image_properties[1:5])
  179.             tvec = np.array(binary_image_properties[5:8])
  180.             camera_id = binary_image_properties[8]
  181.             # print("image_id_camera")
  182.             # print(camera_id)
  183.             image_name = ""
  184.             current_char = read_next_bytes(fid, 1, "c")[0]
  185.             while current_char != b"\x00":   # look for the ASCII 0 entry
  186.                 image_name += current_char.decode("utf-8")
  187.                 current_char = read_next_bytes(fid, 1, "c")[0]
  188.             num_points2D = read_next_bytes(fid, num_bytes=8,
  189.                                            format_char_sequence="Q")[0]
  190.             x_y_id_s = read_next_bytes(fid, num_bytes=24*num_points2D,
  191.                                        format_char_sequence="ddq"*num_points2D)
  192.             xys = np.column_stack([tuple(map(float, x_y_id_s[0::3])),
  193.                                    tuple(map(float, x_y_id_s[1::3]))])
  194.             point3D_ids = np.array(tuple(map(int, x_y_id_s[2::3])))
  195.             # print("point_3d")
  196.             # print(point3D_ids)
  197.             images[image_id] = Image(
  198.                 id=image_id, qvec=qvec, tvec=tvec,
  199.                 camera_id=camera_id, name=image_name,
  200.                 xys=xys, point3D_ids=point3D_ids)
  201.     return images
  202.  
  203.  
  204. def read_points3D_text(path):
  205.     """
  206.     see: src/base/reconstruction.cc
  207.         void Reconstruction::ReadPoints3DText(const std::string& path)
  208.         void Reconstruction::WritePoints3DText(const std::string& path)
  209.     """
  210.     points3D = {}
  211.     with open(path, "r") as fid:
  212.         while True:
  213.             line = fid.readline()
  214.             if not line:
  215.                 break
  216.             line = line.strip()
  217.             if len(line) > 0 and line[0] != "#":
  218.                 elems = line.split()
  219.                 point3D_id = int(elems[0])
  220.                 xyz = np.array(tuple(map(float, elems[1:4])))
  221.                 rgb = np.array(tuple(map(int, elems[4:7])))
  222.                 error = float(elems[7])
  223.                 image_ids = np.array(tuple(map(int, elems[8::2])))
  224.                 point2D_idxs = np.array(tuple(map(int, elems[9::2])))
  225.                 points3D[point3D_id] = Point3D(id=point3D_id, xyz=xyz, rgb=rgb,
  226.                                                error=error, image_ids=image_ids,
  227.                                                point2D_idxs=point2D_idxs)
  228.     return points3D
  229.  
  230.  
  231. def read_points3d_binary(path_to_model_file):
  232.     """
  233.     see: src/base/reconstruction.cc
  234.         void Reconstruction::ReadPoints3DBinary(const std::string& path)
  235.         void Reconstruction::WritePoints3DBinary(const std::string& path)
  236.     """
  237.     points3D = {}
  238.     with open(path_to_model_file, "rb") as fid:
  239.         num_points = read_next_bytes(fid, 8, "Q")[0]
  240.         # print("num points")
  241.         # print(num_points)
  242.         for point_line_index in range(num_points):
  243.             binary_point_line_properties = read_next_bytes(
  244.                 fid, num_bytes=43, format_char_sequence="QdddBBBd")
  245.             point3D_id = binary_point_line_properties[0]
  246.             xyz = np.array(binary_point_line_properties[1:4])
  247.             rgb = np.array(binary_point_line_properties[4:7])
  248.             error = np.array(binary_point_line_properties[7])
  249.             track_length = read_next_bytes(
  250.                 fid, num_bytes=8, format_char_sequence="Q")[0]
  251.             track_elems = read_next_bytes(
  252.                 fid, num_bytes=8*track_length,
  253.                 format_char_sequence="ii"*track_length)
  254.             image_ids = np.array(tuple(map(int, track_elems[0::2])))
  255.             point2D_idxs = np.array(tuple(map(int, track_elems[1::2])))
  256.             points3D[point3D_id] = Point3D(
  257.                 id=point3D_id, xyz=xyz, rgb=rgb,
  258.                 error=error, image_ids=image_ids,
  259.                 point2D_idxs=point2D_idxs)
  260.     return points3D
  261.  
  262.  
  263.  
  264.  
  265. def read_model(path, ext):
  266.     if ext == ".txt":
  267.         # 读取相机参数、图像、3D点
  268.         cameras = read_cameras_text(os.path.join(path, "cameras" + ext))
  269.         images = read_images_text(os.path.join(path, "images" + ext))
  270.         points3D = read_points3D_text(os.path.join(path, "points3D") + ext)
  271.     else:
  272.         cameras = read_cameras_binary(os.path.join(path, "cameras" + ext))
  273.         images = read_images_binary(os.path.join(path, "images" + ext))
  274.         points3D = read_points3d_binary(os.path.join(path, "points3D") + ext)
  275.     return cameras, images, points3D
  276.  
  277.  
  278. def qvec2rotmat(qvec):
  279.     return np.array([
  280.         [1 - 2 * qvec[2]**2 - 2 * qvec[3]**2,
  281.          2 * qvec[1] * qvec[2] - 2 * qvec[0] * qvec[3],
  282.          2 * qvec[3] * qvec[1] + 2 * qvec[0] * qvec[2]],
  283.         [2 * qvec[1] * qvec[2] + 2 * qvec[0] * qvec[3],
  284.          1 - 2 * qvec[1]**2 - 2 * qvec[3]**2,
  285.          2 * qvec[2] * qvec[3] - 2 * qvec[0] * qvec[1]],
  286.         [2 * qvec[3] * qvec[1] - 2 * qvec[0] * qvec[2],
  287.          2 * qvec[2] * qvec[3] + 2 * qvec[0] * qvec[1],
  288.          1 - 2 * qvec[1]**2 - 2 * qvec[2]**2]])
  289.  
  290.  
  291. def rotmat2qvec(R):
  292.     Rxx, Ryx, Rzx, Rxy, Ryy, Rzy, Rxz, Ryz, Rzz = R.flat
  293.     K = np.array([
  294.         [Rxx - Ryy - Rzz, 0, 0, 0],
  295.         [Ryx + Rxy, Ryy - Rxx - Rzz, 0, 0],
  296.         [Rzx + Rxz, Rzy + Ryz, Rzz - Rxx - Ryy, 0],
  297.         [Ryz - Rzy, Rzx - Rxz, Rxy - Ryx, Rxx + Ryy + Rzz]]) / 3.0
  298.     eigvals, eigvecs = np.linalg.eigh(K)
  299.     qvec = eigvecs[[3, 0, 1, 2], np.argmax(eigvals)]
  300.     if qvec[0] < 0:
  301.         qvec *= -1
  302.     return qvec
  303.  
  304.  
  305.  
  306. if __name__ == '__main__':
  307.     parser = argparse.ArgumentParser(description='Convert colmap results into input for PatchmatchNet')
  308.  
  309.     parser.add_argument('--folder', default='./data',type=str, help='Project dir.')
  310.     parser.add_argument('--theta0', type=float, default=5)
  311.     parser.add_argument('--sigma1', type=float, default=1)
  312.     parser.add_argument('--sigma2', type=float, default=10)
  313.  
  314.     # 测试程序是否可用,如果置此位为true则不保存结果
  315.     parser.add_argument('--test', action='store_true', default=False, help='If set, do not write to file.')
  316.     # 转换格式,将图片转换成jpg格式
  317.     parser.add_argument('--convert_format', action='store_true', default=False, help='If set, convert image to jpg format.')
  318.  
  319.     args = parser.parse_args()
  320.  
  321.     # 图片路径,即这个路径下要有图像
  322.     image_dir = os.path.join(args.folder, 'images')
  323.     # 位姿路径
  324.     model_dir = os.path.join(args.folder, 'sparse')
  325.     
  326.     # 这两个文件夹是我们要保存的结果
  327.     cam_dir = os.path.join(args.folder, 'cams')
  328.     renamed_dir = os.path.join(args.folder, 'images')
  329.     
  330.     # 相机参数、图像、3D点读入到变量中
  331.     cameras, images, points3d = read_model(model_dir, '.txt')
  332.     #cameras, images, points3d = read_model(model_dir, '.bin')
  333.  
  334.     num_images = len(list(images.items()))
  335.  
  336.     param_type = {
  337.         'SIMPLE_PINHOLE': ['f', 'cx', 'cy'],
  338.         'PINHOLE': ['fx', 'fy', 'cx', 'cy'],
  339.         'SIMPLE_RADIAL': ['f', 'cx', 'cy', 'k'],
  340.         'SIMPLE_RADIAL_FISHEYE': ['f', 'cx', 'cy', 'k'],
  341.         'RADIAL': ['f', 'cx', 'cy', 'k1', 'k2'],
  342.         'RADIAL_FISHEYE': ['f', 'cx', 'cy', 'k1', 'k2'],
  343.         'OPENCV': ['fx', 'fy', 'cx', 'cy', 'k1', 'k2', 'p1', 'p2'],
  344.         'OPENCV_FISHEYE': ['fx', 'fy', 'cx', 'cy', 'k1', 'k2', 'k3', 'k4'],
  345.         'FULL_OPENCV': ['fx', 'fy', 'cx', 'cy', 'k1', 'k2', 'p1', 'p2', 'k3', 'k4', 'k5', 'k6'],
  346.         'FOV': ['fx', 'fy', 'cx', 'cy', 'omega'],
  347.         'THIN_PRISM_FISHEYE': ['fx', 'fy', 'cx', 'cy', 'k1', 'k2', 'p1', 'p2', 'k3', 'k4', 'sx1', 'sy1']
  348.     }
  349.  
  350.     # intrinsic
  351.     intrinsic = {}
  352.     for camera_id, cam in cameras.items():
  353.         params_dict = {key: value for key, value in zip(param_type[cam.model], cam.params)}
  354.         print(cam.model)
  355.         if 'f' in param_type[cam.model]:
  356.             params_dict['fx'] = params_dict['f']
  357.             params_dict['fy'] = params_dict['f']
  358.         i = np.array([
  359.             [params_dict['fx'], 0, params_dict['cx']],
  360.             [0, params_dict['fy'], params_dict['cy']],
  361.             [0, 0, 1]
  362.         ])
  363.         intrinsic[camera_id] = i
  364.     print('intrinsic[0]\n', intrinsic, end='\n\n')
  365.  
  366.     # extrinsic
  367.     extrinsic = {}
  368.     for image_id, image in images.items():
  369.         e = np.zeros((4, 4))
  370.         e[:3, :3] = qvec2rotmat(image.qvec)
  371.         e[:3, 3] = image.tvec
  372.         e[3, 3] = 1
  373.         extrinsic[image_id] = e
  374.     print('extrinsic[1]\n', extrinsic[1], end='\n\n')
  375.  
  376.     # depth range and interval
  377.     depth_ranges = {}
  378.     for i in range(num_images):
  379.         zs = []
  380.         for p3d_id in images[i+1].point3D_ids:
  381.             
  382.             if p3d_id == -1:
  383.                 continue
  384.             
  385.             transformed = np.matmul(extrinsic[i+1], [points3d[p3d_id].xyz[0], points3d[p3d_id].xyz[1], points3d[p3d_id].xyz[2], 1])
  386.             zs.append(np.asscalar(transformed[2]))
  387.         zs_sorted = sorted(zs)
  388.         # relaxed depth range
  389.         depth_min = zs_sorted[int(len(zs) * .01)]
  390.         depth_max = zs_sorted[int(len(zs) * .99)]
  391.         
  392.         
  393.         depth_ranges[i+1] = (depth_min, depth_max)
  394.     print('depth_ranges[1]\n', depth_ranges[1], end='\n\n')
  395.  
  396.     # view selection
  397.     score = np.zeros((len(images), len(images)))
  398.     queue = []
  399.     for i in range(len(images)):
  400.         for j in range(i + 1, len(images)):
  401.             queue.append((i, j))
  402.     # def calc_score(inputs):
  403.     #     i, j = inputs
  404.     #     id_i = images[i+1].point3D_ids
  405.     #     id_j = images[j+1].point3D_ids
  406.     #     id_intersect = [it for it in id_i if it in id_j]
  407.     #     cam_center_i = -np.matmul(extrinsic[i+1][:3, :3].transpose(), extrinsic[i+1][:3, 3:4])[:, 0]
  408.     #     cam_center_j = -np.matmul(extrinsic[j+1][:3, :3].transpose(), extrinsic[j+1][:3, 3:4])[:, 0]
  409.     #     score = 0
  410.     #     for pid in id_intersect:
  411.     #         if pid == -1:
  412.     #             continue
  413.     #         p = points3d[pid].xyz
  414.     #         theta = (180 / np.pi) * np.arccos(np.dot(cam_center_i - p, cam_center_j - p) / np.linalg.norm(cam_center_i - p) / np.linalg.norm(cam_center_j - p))
  415.     #         score += np.exp(-(theta - args.theta0) * (theta - args.theta0) / (2 * (args.sigma1 if theta <= args.theta0 else args.sigma2) ** 2))
  416.     #     return i, j, score
  417.     # p = mp.Pool(processes=mp.cpu_count())
  418.     # result = p.map(calc_score, queue)
  419.     for i, j in queue:
  420.         id_i = images[i+1].point3D_ids
  421.         id_j = images[j+1].point3D_ids
  422.         id_intersect = [it for it in id_i if it in id_j]
  423.         cam_center_i = -np.matmul(extrinsic[i+1][:3, :3].transpose(), extrinsic[i+1][:3, 3:4])[:, 0]
  424.         cam_center_j = -np.matmul(extrinsic[j+1][:3, :3].transpose(), extrinsic[j+1][:3, 3:4])[:, 0]
  425.         s = 0
  426.         for pid in id_intersect:
  427.             if pid == -1:
  428.                 continue
  429.             p = points3d[pid].xyz
  430.             theta = (180 / np.pi) * np.arccos(np.dot(cam_center_i - p, cam_center_j - p) / np.linalg.norm(cam_center_i - p) / np.linalg.norm(cam_center_j - p))
  431.             s += np.exp(-(theta - args.theta0) * (theta - args.theta0) / (2 * (args.sigma1 if theta <= args.theta0 else args.sigma2) ** 2))
  432.         score[i, j] = s
  433.         score[j, i] = s
  434.  
  435.     view_sel = []
  436.     for i in range(len(images)):
  437.         sorted_score = np.argsort(score[i])[::-1]
  438.         view_sel.append([(k, score[i, k]) for k in sorted_score[:10]])
  439.     print('view_sel[0]\n', view_sel[0], end='\n\n')
  440.  
  441.     # write
  442.     try:
  443.         os.makedirs(cam_dir)
  444.     except os.error:
  445.         print(cam_dir + ' already exist.')
  446.     for i in range(num_images):
  447.         with open(os.path.join(cam_dir, '%08d_cam.txt' % i), 'w') as f:
  448.             f.write('extrinsic\n')
  449.             for j in range(4):
  450.                 for k in range(4):
  451.                     f.write(str(extrinsic[i+1][j, k]) + ' ')
  452.                 f.write('\n')
  453.             f.write('\nintrinsic\n')
  454.             for j in range(3):
  455.                 for k in range(3):
  456.                     f.write(str(intrinsic[images[i+1].camera_id][j, k]) + ' ')
  457.                 f.write('\n')
  458.             f.write('\n%f %f \n' % (depth_ranges[i+1][0], depth_ranges[i+1][1]))
  459.     with open(os.path.join(args.folder, 'pair.txt'), 'w') as f:
  460.         f.write('%d\n' % len(images))
  461.         for i, sorted_score in enumerate(view_sel):
  462.             f.write('%d\n%d ' % (i, len(sorted_score)))
  463.             for image_id, s in sorted_score:
  464.                 f.write('%d %f ' % (image_id, s))
  465.             f.write('\n')
  466.     for i in range(num_images):
  467.         if args.convert_format:
  468.             img = cv2.imread(os.path.join(image_dir, images[i+1].name))
  469.             cv2.imwrite(os.path.join(renamed_dir, '%08d.jpg' % i), img)
  470.         else:
  471.             shutil.copyfile(os.path.join(image_dir, images[i+1].name), os.path.join(renamed_dir, '%08d.jpg' % i))

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/2023面试高手/article/detail/103525
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号