devbox
知新_RL
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

使用python-opencv对双目摄像头进行立体视觉图像矫正，去畸变_双目相机去畸变

作者：知新_RL | 2024-02-18 00:46:46

赞

踩

双目相机去畸变

准备

        1、一张棋盘图

        可以直接从opencv官方github下载，这是一个拥有10*7个格子的棋盘，共有9*6个角点，每个格子24mm，本文所使用的就是这一个棋盘。你需要将它打印在A4纸上用于后续使用。（也可以根据官方教程自行设置棋盘大小OpenCV: Create calibration pattern）

opencv/pattern.png at 4.x · opencv/opencv · GitHub

         2、一个双目摄像头

        随便在tb买的一个不知名摄像头，附赠了一个.exe的测试工具用于简单使用摄像头效果如下

 

 使用opencv简单测试一下，我用的笔记本，接上usb摄像头就是从1开始了，这个双目摄像头虽然有两个输入index=1和index=2但是其实只需要获取index=1的那个视频流就可以得到双目效果。

  1. import cv2
  2. cap = cv2.VideoCapture(1)
  3. cap.set(cv2.CAP_PROP_FRAME_WIDTH,1280)
  4. cap.set(cv2.CAP_PROP_FRAME_HEIGHT,480)
  5. while(1):
  6.     _, frame = cap.read()
  7.     assert _, "摄像头获取失败"
  8.     cv2.imshow('img', frame)
  9.     c = cv2.waitKey(1)
  10.     if c == 27:
  11.         cap.release()
  12.         break

开启前必须将分辨率设置为正确的宽度，我的相机是1280，如果设置宽度不正确会导致无法正确得到双目图像

可以通过下面代码获取相机分辨率,主要是获得width，双目图的width应该为两个相机的width之和

  1. import cv2
  2. cap0 = cv2.VideoCapture(1)
  3. cap1 = cv2.VideoCapture(2)
  4. res0 = [cap0.get(cv2.CAP_PROP_FRAME_WIDTH),cap0.get(cv2.CAP_PROP_FRAME_HEIGHT)]
  5. res1 = [cap1.get(cv2.CAP_PROP_FRAME_WIDTH),cap1.get(cv2.CAP_PROP_FRAME_HEIGHT)] 
  6. print(res0)
  7. print(res1)
  8. cap0.release()
  9. cap1.release()

分辨率正确的双目图(1280*480)

分辨率错误的双目图(2560*480)

开始操作

先给棋盘拍照

  1. import cv2,os
  2. cap = cv2.VideoCapture(1)
  3. cap.set(cv2.CAP_PROP_FRAME_WIDTH,1280)
  4. cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
  5. # 生成目录
  6. path = './calibration/'
  7. path_l = './calibration/left/'
  8. path_r = './calibration/right/'
  9. os.mkdir(path) if not os.path.exists(path) else None
  10. os.mkdir(path_l) if not os.path.exists(path_l) else None
  11. os.mkdir(path_r) if not os.path.exists(path_r) else None
  12. count = 0
  13. while cap.isOpened():
  14.     ret, frame = cap.read()
  15.     cv2.imshow('img',frame)
  16.     k = cv2.waitKey(1)
  17.     # 按下ESC退出
  18.     if k == 27:
  19.         break
  20.     # 按下空格键暂停
  21.     if k == 32:
  22.         cv2.imshow('img',frame)
  23.         # 再次按下空格保存
  24.         if cv2.waitKey() == 32:            
  25.             cv2.imwrite(path + "{}.jpg".format(count), frame)# 保存全图
  26.             cv2.imwrite(path_l + "{}.jpg".format(count), frame[:,0:640])# 保存左图
  27.             cv2.imwrite(path_r + "{}.jpg".format(count), frame[:,640:])# 保存右图
  28.             count += 1
  29. cv2.destroyAllWindows()
  30. cap.release()
  31.

按照下图至少拍摄12对左右图像，以获得最佳效果

 来源：Stereo Calibration for the Dual Camera Mezzanine - Blog - FPGA - element14 Community

 测试一下棋盘角点绘制

  1. import cv2
  2. img = cv2.imread('./calib/left/0.jpg')
  3. img1 = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
  4. ret, corner = cv2.findChessboardCorners(img1, (9,6))
  5. ret, corner = cv2.find4QuadCornerSubpix(img1, corner, (7,7))
  6. cv2.drawChessboardCorners(img, (9,6), corner, ret)
  7. cv2.imshow('corner', img)
  8. cv2.waitKey(0)
  9.

 接下来就获取矫正所需要的参数

  1. import cv2, glob
  2. import numpy as np
  3. '''
  4. 获得标定所需参数
  5. '''
  6. # 定义棋盘格的大小
  7. chessboard_size = (9, 6)
  8. # 定义图像分辨率，根据自己相机的分辨率修改
  9. imgsz = (640, 480)
  10. # 定义棋盘格中每个格子的物理大小，自己用尺子量，单位为毫米（mm）
  11. square_size = 24
  12.  
  13. # 定义棋盘格模板的点的坐标
  14. objp = np.zeros((chessboard_size[0]*chessboard_size[1], 3), np.float32) #生成每个角点三维坐标，共有chessboard_size[0]*chessboard_size[1]个坐标，z轴置0不影响
  15. objp[:, :2] = np.mgrid[0:chessboard_size[0], 0:chessboard_size[1]].T.reshape(-1, 2) * square_size #计算得到每个角点的x,y
  16.  
  17. # 读取所有棋盘格图像并提取角点
  18. imgpoints_left, imgpoints_right = [], []  # 存储图像中的角点
  19. objpoints = []  # 存储模板中的角点
  20. images = glob.glob('./calibration/right/*.jpg')  # 所有棋盘格图像所在的目录
  21. for fname in images:
  22.     img = cv2.imread(fname)
  23.     gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
  24.     ret, corners = cv2.findChessboardCorners(gray, chessboard_size, None) #计算corner
  25.     ret, corners = cv2.find4QuadCornerSubpix(gray, corners, (7,7)) #提高角点检测的准确性和稳定性
  26.     if ret == True:
  27.         imgpoints_right.append(corners)
  28.         objpoints.append(objp)
  29.  
  30. images = glob.glob('./calibration/left/*.jpg')  # 所有棋盘格图像所在的目录
  31. for fname in images:
  32.     img = cv2.imread(fname)
  33.     gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
  34.     ret, corners = cv2.findChessboardCorners(gray, chessboard_size, None) #计算corner
  35.     ret, corners = cv2.find4QuadCornerSubpix(gray, corners, (7,7)) #提高角点检测的准确性和稳定性
  36.     if ret == True:
  37.         imgpoints_left.append(corners)
  38. '''
  39. 开始标定，获得参数
  40. '''
  41. # 标定相机，获得内参和畸变参数
  42. ret, mtx_r, dist_r, rvecs_r, tvecs_r = cv2.calibrateCamera(objpoints, imgpoints_right, gray.shape[::-1], None, None)
  43. ret, mtx_l, dist_l, rvecs_l, tvecs_l = cv2.calibrateCamera(objpoints, imgpoints_left, gray.shape[::-1], None, None)
  44.  
  45. # 指定迭代次数最大30或者误差小于0.001
  46. term = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
  47.  
  48. # 进行双目相机标定,主要是获得R，T两个矩阵
  49. rotation_matrix, translation_matrix = cv2.stereoCalibrate(
  50.             objpoints, imgpoints_left, imgpoints_right,
  51.             mtx_l, dist_l,
  52.             mtx_r, dist_r,
  53.             imgsz, flags=cv2.CALIB_FIX_INTRINSIC, criteria=term)[5:7]
  54.  
  55. # 获得矫正矩阵和投影矩阵，用于后续进行图像校正
  56. rect_left, rect_right, \
  57. proj_left, proj_right, \
  58. dispartity, \
  59. ROI_left, ROI_right = cv2.stereoRectify(
  60.             mtx_l, dist_l,
  61.             mtx_r, dist_r,
  62.             imgsz, rotation_matrix, translation_matrix,
  63.             flags=cv2.CALIB_ZERO_DISPARITY, alpha=-1)
  64.  
  65.  
  66. '''
  67. 打印结果
  68. '''
  69. print('mtx_l = np.array({})'.format(np.array2string(mtx_l, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  70. print('mtx_r = np.array({})'.format(np.array2string(mtx_r, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  71. print('dist_l = np.array({})'.format(np.array2string(dist_l, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  72. print('dist_r = np.array({})'.format(np.array2string(dist_r, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  73. print('R = np.array({})'.format(np.array2string(rotation_matrix, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  74. print('T = np.array({})'.format(np.array2string(translation_matrix, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  75. print('rect_left = np.array({})'.format(np.array2string(rect_left, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  76. print('rect_right = np.array({})'.format(np.array2string(rect_right, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  77. print('proj_left = np.array({})'.format(np.array2string(proj_left, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  78. print('proj_right = np.array({})'.format(np.array2string(proj_right, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  79. print('dispartity = np.array({})'.format(np.array2string(dispartity, separator=', ', formatter={'int': lambda x: f'{x: 3d}'},prefix='[', suffix=']')))
  80. # print('mtx_l = np.array({})'.format(mtx_l))
  81. # print('mtx_r = np.array({})'.format(mtx_r))
  82. # print('dist_l = np.array({})'.format(dist_l))
  83. # print('dist_r = np.array({})'.format(dist_r))
  84. # print('R = np.array({})'.format(rotation_matrix))
  85. # print('T = np.array({})'.format(translation_matrix))
  86. # print('rect_left = np.array({})'.format(rect_left))
  87. # print('rect_right = np.array({})'.format(rect_right))
  88. # print('proj_left = np.array({})'.format(proj_left))
  89. # print('proj_right = np.array({})'.format(proj_right))
  90. # print('dispartity = np.array({})'.format(dispartity))
  91. print('ROI_left = np.array({})'.format(ROI_left))
  92. print('ROI_right = np.array({})'.format(ROI_right))
  93.

得到下面参数

可以直接复制用于图像矫正 

测试

  1. import cv2, glob, os
  2. import numpy as np
  3. def get_corners(imgs, corners):
  4.     for img in imgs:
  5.         # 9x12棋盘有8x11个角点
  6.         ret, c = cv2.findChessboardCorners(img, (9,6))
  7.         assert(ret)
  8.         ret, c = cv2.find4QuadCornerSubpix(img, c, (7,7))
  9.         assert(ret)
  10.         corners.append(c)
  11. mtx_l = np.array([[479.61836296,   0.,         339.91341613],
  12.  [  0.,         478.44413757, 240.61069496],
  13.  [  0.,           0.,           1.,        ]])
  14. mtx_r = np.array([[483.4989366,    0.,         306.98497259],
  15.  [  0.,         482.17064224, 228.91672333],
  16.  [  0.,           0.,           1.        ]])
  17. dist_l = np.array([[ 0.07539615, -0.51291496,  0.00405133, -0.00084347,  0.7514282 ]])
  18. dist_r = np.array([[-1.30834008e-01,  8.25592192e-01,  9.83305297e-04, -7.40611932e-06,  -1.67568022e+00]])
  19. R = np.array([[ 9.99947786e-01, -1.06501500e-03,  1.01632001e-02],
  20.  [ 8.52847758e-04,  9.99782093e-01,  2.08575744e-02],
  21.  [-1.01831991e-02, -2.08478176e-02,  9.99730799e-01]])
  22. T = np.array([[-62.0710667 ],
  23.  [  0.27233791],
  24.  [  0.49530174]])
  25. rect_left = np.array([[ 0.99998384, -0.005285,    0.00209416],
  26.  [ 0.00526285,  0.99993159,  0.01044553],
  27.  [-0.00214922, -0.01043434,  0.99994325]])
  28. rect_right = np.array([[ 0.99995854, -0.00438734, -0.00797926],
  29.  [ 0.00430379,  0.99993606, -0.01045726],
  30.  [ 0.00802463,  0.01042249,  0.99991348]])
  31. proj_left = np.array([[480.3073899,    0.,         322.84606934,   0.,        ],
  32.  [  0.,         480.3073899,  235.60386848,   0.,        ],
  33.  [  0.,           0.,           1.,           0.,        ]])
  34. proj_right = np.array([[ 4.80307390e+02,  0.00000000e+00,  3.22846069e+02, -2.98144281e+04],
  35.  [ 0.00000000e+00,  4.80307390e+02,  2.35603868e+02,  0.00000000e+00],
  36.  [ 0.00000000e+00,  0.00000000e+00,  1.00000000e+00,  0.00000000e+00]])
  37. dispartity = np.array([[ 1.00000000e+00,  0.00000000e+00,  0.00000000e+00, -3.22846069e+02],
  38.  [ 0.00000000e+00,  1.00000000e+00,  0.00000000e+00, -2.35603868e+02],
  39.  [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  4.80307390e+02],
  40.  [ 0.00000000e+00,  0.00000000e+00,  1.61098978e-02, -0.00000000e+00]])
  41. ROI_left = np.array((5, 10, 612, 456))
  42. ROI_right = np.array((14, 5, 626, 475))
  43. img_left = []
  44. img_right = []
  45. corners_left = []
  46. corners_right = []
  47. img_file = glob.glob('./calibration/*.jpg')
  48. imgsize = (640, 480)
  49.  
  50. for img in img_file:
  51.     frame = cv2.imread(img)
  52.     frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
  53.     l = frame[:,0:640]
  54.     r = frame[:,640:]
  55.     img_left.append(l)
  56.     img_right.append(r)
  57. print("获取角点", "left")
  58. get_corners(img_left, corners_left)
  59. print("获取角点", "right")
  60. get_corners(img_right, corners_right)
  61. for i in range(len(img_left)):
  62.     l = img_left[i]
  63.     r = img_right[i]
  64.     # 计算双目校正的矩阵
  65.     R1, R2, P1, P2, Q, validPixROI1, validPixROI2 = cv2.stereoRectify(mtx_l, dist_l, mtx_r, dist_r, imgsize, R, T)
  66.     # 计算校正后的映射关系
  67.     maplx , maply = cv2.initUndistortRectifyMap(mtx_l, dist_l, R1, P1, imgsize, cv2.CV_16SC2)
  68.     maprx , mapry = cv2.initUndistortRectifyMap(mtx_r, dist_r, R2, P2, imgsize, cv2.CV_16SC2)
  69.     # 映射新图像
  70.     lr = cv2.remap(l, maplx, maply, cv2.INTER_LINEAR)
  71.     rr = cv2.remap(r, maprx, mapry, cv2.INTER_LINEAR)
  72.     all = np.hstack((lr,rr))
  73.     # 变换之后和变换之前的角点坐标不一致，所以线不是正好经过角点，只是粗略估计，但偶尔能碰到离角点比较近的线，观察会比较明显
  74.     cv2.line(all, (-1, int(corners_left[i][0][0][1])), (all.shape[1], int(corners_left[i][0][0][1])), (255), 1)
  75.     # 可以看出左右图像y坐标对齐还是比较完美的，可以尝试着打印双目校正前的图片，很明显，左右y坐标是不对齐的
  76.     cv2.imshow('a', all)
  77.     c = cv2.waitKey()
  78.     cv2.destroyAllWindows()
  79.     if c == 27:
  80.         break
  81.  
  82.  
  83. print("end")
  84.

此段代码借鉴http://t.csdn.cn/uxwLA

 参考链接：

Depther project - part 2: calibrate dual camera, parameters rectification - edgenoon.ai

http://t.csdn.cn/uxwLA

OpenCV: Create calibration pattern

opencv/pattern.png at 4.x · opencv/opencv · GitHub

Stereo Calibration for the Dual Camera Mezzanine - Blog - FPGA - element14 Community

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/知新_RL/article/detail/102960
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号