How to Calibrate a Camera with OpenCV?

How to Calibrate a Camera with OpenCV: A Complete Guide

Welcome to our tutorial on camera calibration using OpenCV, a powerful tool in computer vision applications, particularly in the fields of robotics and augmented reality. In this blog, we will walk through the process of extracting frames from a video, detecting checkerboard patterns (used for calibration), and finally, calibrating the camera.

What is Camera Calibration?

Camera calibration is the process of estimating the parameters of the lens and the image sensor of a camera to improve the accuracy of capturing images. These parameters can be used to correct lens distortion, measure the size of an object in the world units, or determine the location of the camera in the scene.

The Code Breakdown

Our Python script uses OpenCV to perform camera calibration with the following steps:

1. Extract Frames from a Video
2. Find Checkerboard Corners
3. Calibrate the Camera

1. Extract Frames from a Video

The function extract_frames reads a video file and extracts frames at a specified interval (skip_frames).

def extract_frames(video_path, skip_frames=30):
    cap = cv2.VideoCapture(video_path)
    frames = []
    idx = 0
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
        if idx % skip_frames == 0:
            frames.append(frame)
        idx += 1
    
    cap.release()
    return frames
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

2. Find Checkerboard Corners

We use the find_checkerboard_corners function to detect the corners of a checkerboard pattern in each frame. This pattern is crucial for calibration as it provides a known geometry to compare against.

def find_checkerboard_corners(frames, checkerboard_size=(9, 13)):
    criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
    objpoints = []
    imgpoints = []
    
    objp = np.zeros((checkerboard_size[0] * checkerboard_size[1], 3), np.float32)
    objp[:, :2] = np.mgrid[0:checkerboard_size[0], 0:checkerboard_size[1]].T.reshape(-1, 2)

    for _, frame in enumerate(tqdm(frames)):
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        ret, corners = cv2.findChessboardCorners(gray, checkerboard_size, None)
        
        if ret:
            objpoints.append(objp)
            corners2 = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
            imgpoints.append(corners2)
    
    return objpoints, imgpoints, gray.shape[::-1]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

3. Calibrate the Camera

With the object and image points obtained from the checkerboard, the calibrate_camera function estimates the camera parameters.

def calibrate_camera(objpoints, imgpoints, frame_shape):
    ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, frame_shape, None, None)
    return mtx, dist
1
2
3

Conclusion

Once the calibration is done, the camera matrix and distortion coefficients are printed. These parameters allow you to correct images taken from this camera, enhance accuracy in measurement applications, and perform numerous other computer vision tasks.

Camera calibration is a fundamental step in any serious computer vision work. By accurately determining the camera’s intrinsic and extrinsic parameters, one can significantly improve the output and accuracy of their vision algorithms. Whether you’re developing a robot’s vision system or creating a 3D model from images, understanding how to calibrate a camera is essential.

Feel free to use this code as a starting point for your camera calibration needs and adapt it to different patterns or calibration methods as required.

Sample Code

import cv2
import numpy as np
from tqdm import tqdm

def extract_frames(video_path, skip_frames=30):
    """ Extract frames from a video file """
    cap = cv2.VideoCapture(video_path)
    frames = []
    idx = 0
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
        if idx % skip_frames == 0:
            frames.append(frame)
        idx += 1
    
    cap.release()
    return frames

def find_checkerboard_corners(frames, checkerboard_size=(9, 13)):
    """ Find and refine checkerboard corners in a list of frames """
    criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
    objpoints = []  # 3D points in real world space
    imgpoints = []  # 2D points in image plane
    
    objp = np.zeros((checkerboard_size[0] * checkerboard_size[1], 3), np.float32)
    objp[:, :2] = np.mgrid[0:checkerboard_size[0], 0:checkerboard_size[1]].T.reshape(-1, 2)

    for _,frame in enumerate(tqdm(frames)):
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        ret, corners = cv2.findChessboardCorners(gray, checkerboard_size, None)
        
        if ret:
            objpoints.append(objp)
            corners2 = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
            imgpoints.append(corners2)
    
    return objpoints, imgpoints, gray.shape[::-1]

def calibrate_camera(objpoints, imgpoints, frame_shape):
    """ Calibrate the camera given object points, image points, and the shape of the frames """
    ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, frame_shape, None, None)
    return mtx, dist

# Path to your video file
video_path = '20240509_155345.mp4'

# Extract frames from the video
frames = extract_frames(video_path, skip_frames=10)
print("frames: ",len(frames))

# Find checkerboard corners
objpoints, imgpoints, frame_shape = find_checkerboard_corners(frames)
print('valid frames: ', len(objpoints))

# Calibrate the camera
camera_matrix, dist_coeffs = calibrate_camera(objpoints, imgpoints, frame_shape)

# camera_matrix = np.round(camera_matrix,8)
# dist_coeffs = np.round(dist_coeffs, 8)
print("Camera matrix:")
print(camera_matrix)
print("Distortion coefficients:")
print(dist_coeffs)

cal_param=f'''
Camera1.fx: {camera_matrix[0,0]:.8f}
Camera1.fy: {camera_matrix[1,1]:.8f}
Camera1.cx: {camera_matrix[0,2]:.8f}
Camera1.cy: {camera_matrix[1,2]:.8f}

Camera1.k1: {dist_coeffs[0,0]:.8f}
Camera1.k2: {dist_coeffs[0,1]:.8f}
Camera1.p1: {dist_coeffs[0,2]:.8f}
Camera1.p2: {dist_coeffs[0,3]:.8f}
Camera1.k3: {dist_coeffs[0,4]:.8f}
'''
print(cal_param)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

Checkerboards Download

https://markhedleyjones.com/projects/calibration-checkerboard-collection