Mono/Stereo Camera Calibration using OpenCV

 

Calibrating a camera to compensate for lens distortion and positional offsets of stereo camera pairs is an important requirement for many applications such as pose reconstruction, depth-from-stereo and structure-from-motion.

I decided to put the required OpenCV code on github and provide a quick guide trough the calibration process for a single camera as well as for a pair of cameras for stereo tasks.

Single Camera calibration

The basic procedure for a single camera is as follows.

  • compile camera calibration code from github
  • print a camera calibration pattern (checkerboard 9×6 squares or asymmetric circle pattern) below in original scale on DIN A4 paper
  • fixate calibration pattern on flat surface (e.g. taped on a flat table)
  • perform calibration in a well-lit room with diffuse-as-possible illumination by capturing images or using the live-feed function of the program while moving the camera into different angles which show the calibration pattern from different angles and different scales
  • alternatively fixate the camera and move the flat pattern around (more difficult when you cannot see the camera image or amount of motion blur in the image)

Checkerboard (9x6 squares@25mm)

Asymmetric Circle Pattern 4x11

Method 1 : Single Camera calibration using live camera stream

The program uses the connected camera it finds first when using the live-feed option.

  • start the program with the following parameters (respectively for chessboard or circles pattern)
singlecamcalibration.exe -w 9 -h 6 -pt chessboard -n 15 -d 2000 -o mycam -op -oe
singlecamcalibration.exe -w 4 -h 11 -pt acircles -n 15 -d 2000 -o mycam -op -oe
  • increase the number of images (-n) to increase calibration quality
  • increase the duration between image capture (-d) in milliseconds to have more time repositioning the camera/checkerboard
  • change the output name of the calibration file (-o) if required
  • skip writing of the feature points (-op) or extrinsic camera parameters (-oe) for the captured images if not required

In case you are using the Sony Eye 3 camera using the dedicated driver for this camera may work better. This driver can be chosen by the flag -useSonyEye

singlecamcalibration.exe -useSonyEye -w 9 -h 6 -pt chessboard -n 15 -d 2000 -o mycam -op -oe
singlecamcalibration.exe -useSonyEye -w 4 -h 11 -pt acircles -n 15 -d 2000 -o mycam -op -oe

When starting the program you can adjust the camera angle to a central view before hitting ‘g’ to start the capture process.

 

Then move the camera from angle to angle. Stop movement while the program captures an image (image flashes) to reduce image motion blur and rolling shutter effects as much as possible.

 

After the defined number of images has been captures the program computes a camera calibration and writes the data into the defined file. The resulting average pixel error when – based on the derived camera matrix – projecting the detected features points back into the captured images is written into the console (mostly about 0.1 and 0.3 pixels for a successful calibration). In the calibrated state you can hit ‘u’ to toggle between the lens distortion corrected version and the original camera image. Distortion correction is performed using a build-in OpenCV function.

 

Method 2 : Single Camera calibration using image list

  • capture images for calibration manually
  • create xml containing image file names by using image list generator from OpenCV or by using the Matlab script provided below
  • run calibration using the following parameters
singlecamcalibration.exe files.xml -w 9 -h 6 -pt chessboard -o gopro3 -oe -su
singlecamcalibration.exe files.xml -w 4 -h 11 -pt acircles -o gopro3 -oe -su

Matlab image list generator script

files = dir ('C:\camcalibration\Release\frames');

fileID = fopen('files.xml', 'w');

fprintf(fileID, '<?xml version=\"1.0\"?>\n');
fprintf(fileID, '<opencv_storage>\n');
fprintf(fileID, '<images>\n');

for f = 3:size(files,1)
 fprintf(fileID, './frames/%s\n', files(f).name);
end

fprintf(fileID, '</images>\n');
fprintf(fileID, '</opencv_storage>\n');


fclose(fileID)

My generated image list (files.xml) used for calibrating the GoPro has the following content :

<?xml version="1.0"?>
<opencv_storage>
<images>
./frames/frame_00001.png
./frames/frame_00059.png
./frames/frame_00126.png
./frames/frame_00181.png
./frames/frame_00353.png
./frames/frame_00439.png
./frames/frame_00478.png
./frames/frame_00560.png
./frames/frame_00589.png
./frames/frame_00646.png
./frames/frame_00695.png
./frames/frame_00959.png
./frames/frame_01062.png
./frames/frame_01153.png
./frames/frame_01461.png
./frames/frame_01516.png
./frames/frame_01672.png
./frames/frame_01759.png
./frames/frame_01822.png
./frames/frame_01868.png
./frames/frame_01917.png
./frames/frame_02115.png
./frames/frame_02199.png
./frames/frame_02292.png
./frames/frame_02731.png
./frames/frame_02875.png
</images>
</opencv_storage>

 

Manual Calibration Optimization

The automatic calibration pattern reconstruction is not perfect for difficult lighting situations. I therefore added a guided calibration routine which can be compiled by deactivating the define CALIBRATION_DEFAULT and activating the define CALIBRATION_GUIDED.

Then run the program as normal using a camera or a file list. You can adjust the pattern detection now by the provided GUI. If you are happy with the quality of the respective camera angle and detection quality press the ‘C’ to proceed with the next image. If you want to skip an image use the ‘S’ key.

Short summary for manual calibration mode

  • use #define CALIBRATION_GUIDED when compiling the program
  • run the program as normal
  • use GUI to adjust pattern recognition quality
  • press C key to process the current frame (proceed in file list mode)
  • press S key to skip an image (only for file list mode)
  • use parameter -su to show calibration result (file list mode)
  • use U key to show undisortion result (camera mode)

 

Camera calibration GUI for pattern recognition optimization
Camera calibration GUI for pattern recognition optimization

 

Here an example for undistorting the lens distortion from a low-resolution thermal camera.

Therma camera Image with lens distortion
Thermal camera image with visible lens disortion
Undistorted thermal camera image
Undistorted thermal camera image

 

Calibration for GoPro 3 Black, wide angle (SuperView mode)

nframes: 50
image_width: 1920
image_height: 1080
board_width: 9
board_height: 6
square_size: 1.
flags: 0
camera_matrix: !!opencv-matrix
 rows: 3
 cols: 3
 dt: d
 data: [ 8.8884181712344071e+002, 0., 9.6605515331380832e+002, 0.,
 8.9280754125519923e+002, 5.0049450915014620e+002, 0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
 rows: 5
 cols: 1
 dt: d
 data: [ -2.7728908904180716e-001, 1.1386186914777004e-001,
 -1.7366503946725888e-003, -1.1410228753906762e-005,
 -2.8225149521799133e-002 ]
avg_reprojection_error: 5.7702254761830274e-001

Calibration result for Sony Eye 3 Camera (wide angle setting)

nframes: 15
image_width: 640
image_height: 480
board_width: 9
board_height: 6
square_size: 1.
camera_matrix: !!opencv-matrix
 rows: 3
 cols: 3
 dt: d
 data: [ 5.2897513438342014e+002, 0., 2.9905169082796260e+002, 0.,
 5.2733214339719768e+002, 2.3847755049709841e+002, 0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
 rows: 5
 cols: 1
 dt: d
 data: [ -1.2847268463448963e-001, 2.9677375751423829e-001,
 9.7908378847381115e-004, 1.1283989375252129e-003,
 -2.0191338479319310e-001 ]
avg_reprojection_error: 1.8037350912083178e-001

 

Calibration for Sony Eye 3 Camera (narrow angle setting)

nframes: 15
image_width: 640
image_height: 480
board_width: 9
board_height: 6
square_size: 1.
camera_matrix: !!opencv-matrix
 rows: 3
 cols: 3
 dt: d
 data: [ 7.7217360125212849e+002, 0., 2.9095881644331484e+002, 0.,
 7.6704869269702510e+002, 2.1998852370309382e+002, 0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
 rows: 5
 cols: 1
 dt: d
 data: [ -1.0660277961370355e-001, 2.9590384559265970e-001,
 -1.9368422837184270e-003, -1.5595766021217516e-003,
 -6.1360777313585912e-001 ]
avg_reprojection_error: 2.3414034726031402e-001

 

Calibration for hacked Sony Eye 3 Camera for infrared vision (removed IR filter)

nframes: 15
image_width: 640
image_height: 480
board_width: 9
board_height: 6
square_size: 1.
camera_matrix: !!opencv-matrix
 rows: 3
 cols: 3
 dt: d
 data: [ 8.7242522882168146e+002, 0., 3.0130692572297176e+002, 0.,
 8.7040258048779765e+002, 2.3606477763875225e+002, 0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
 rows: 5
 cols: 1
 dt: d
 data: [ -1.2801421281144118e-001, 3.1693104872855238e-001,
 -9.6554464185718766e-004, 2.0221714736715856e-003,
 -5.8185112899589153e-001 ]
avg_reprojection_error: 2.9232175512762920e-001

 

Stereo Camera calibration (Sony Eye Cam 3)

Coming soon….

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s