Tsai’s camera calibration method revisited Berthold K.P. Horn Copright © 2000 Introduction Basic camera calibration is the recovery of the principle distance f and the princi-ple point (x0,y0)T in the image plane — or, equivalently, recovery of the position of the center of projection (x0,y0,f)T in the image coordinate system. This is referred to as interior orientation in photogrammetry. A calibration target can be imaged to provide correspondences between points in the image and points in space. It is, however, generally impractical to position the calibration target accurately with respect to the camera coordinate system using only mechanical means. As a result, the relationship between the target coordinate system and the camera coordinate system typically also needs to be recovered from the correspondences. This is referred to as exterior orientation in photogrammetry. Since cameras often have appreciable geometric distortions, camera calibra-tion is often taken to include the recovery of power series coefficients of these distortions. Furthermore, an unknown scale factor in image sampling may also need to be recovered, because scan lines are typically resampled in the frame grabber, and so picture cells do not correspond discrete sensing elements. Note that in camera calibration we are trying to recover the transforma-tions, based on measurements of coordinates, where one more often uses known transformation to map coordinates from one coordinate system to another. Tsai’s method for camera calibration recovers the interior orientation, the exterior orientation, the power series coefficients for distortion, and an image scale factor that best fit the measured image coordinates corresponding to known target point coordinates. This is done in stages, starting off with closed form least-squares estimates of some parameters and ending with an iterative non-linear optimization of all parameters simultaneously using these estimates as starting values. Importantly, it is error in the image plane that is minimized. Details of the method are different for planar targets than for targets occu-pying some volume in space. Accurate planar targets are easier to make, but lead to some limitations in camera calibration, as pointed out below.2 Interior Orientation — Camera to Image Interior Orientation is the relationship between camera-centric coordinates and image coordinates. The camera coordinate system has its origin at the center of projection, its z axis along the optical axis, and its x and y axes parallel to the x and y axes of the image. Camera coordinates and image coordinates are related by the perspective projection equations: xI − x0 xC yI − y0 yC = and = f zC f zC where f is the principle distance (distance from the center of projection to the image plane), and (x0,y0) is the principle point (foot of the perpendicular from the center of projection to the image plane). That is, the center of projection is at (x0,y0,f)T , as measured in the image coordinate system. Interior orientation has three degrees of freedom. The problem of interior orientation is the recovery of x0, y0, and f . This is the basic task of camera calibration. However, as indicated above, in practice we also need to recover the position and attitude of the calibration target in the camera coordinate system. Exterior Orientation — Scene to Camera Exterior Orientation is the relationship between a scene-centered coordinate sys-tem and a camera-centered coordinate system. The transformation from scene to camera consists of a rotation and a translation. This transformation has six degrees of freedom, three for rotation and three for translation. The scene coordinate system can be any system convenient for the partic-ular design of the target. In the case of a planar target, the z axis is chosen perpendicular to the plane, and z = 0 in the target plane. If rS are the coordinates of a point measured in the scene coordinate system and rC coordinates measured in the camera coordinate system, then rC = R(rS ) + t where t is the translation and R(...) the rotation. If we chose for the moment to use an orthonormal matrix to represent rota-tion, then we can write this in component form: ⎛ ⎞ ⎛ ⎞⎛ ⎞ ⎛ ⎞ xC r11 r12 r13 xS tx ⎝ yC ⎠ ⎠⎝ yS ⎠ ⎝ ty ⎠ = ⎝ r21 r22 r23 + zC r31 r32 r33 zS tz where rC = (xC ,yC ,zC )T , rS = (xS ,yS ,zS )T , and t = (tx ,ty ,tz )T . The unknowns to be recovered in the problem of exterior orientation are the translation vector t and the rotation R(...).3 The Unknown Horizontal Scale Factor A complicating factor in the calibration of many modern electronic cameras is that the discrete nature of image sampling is not preserved in the signal. In typical CCD or CMOS cameras, the initially discrete (staircase) sensor signal in analog form is low pass filtered to produce a smooth video output signal in standard form that hides the transitions between cells of the sensor. This waveform is then digitized in the frame grabber. The sampling in the horizontal direction in the frame grabber is typically not equal to the spacing of sensor cells, and is not known accurately. The horizontal spacing between pixels in the sampled image do not in general correspond to the horizontal spacing between cells in the image sensor. This is in contrast with the vertical direction where sampling is controlled by the spacing of rows of sensor cells. Some digital cameras avoid the intermediate analog waveform and the low pass filtering, but many cameras — particularly cheaper ones intended for the consumer market — do not. In this case the ratio of picture cell size in the horizontal and in the vertical direction is not known a priori from the dimensions of the sensor cells and needs to be determined. This can be done separately using frequency domain methods exploiting limitations of the approximate low pass filter and resulting aliasing effects. Alternatively, the extra scaling parameter can be recovered as part of the camera calibration process. In this case we use a modified equation for xI : xI − x0 xC = s f zC where s is the unknown ratio of the pixel spacing in the x- and y-directions It is not possible to recover this extra parameter when using planar targets, as discussed below, and so it has to be estimated separately in that case. Combining Interior and Exterior Orientation If we combine the equations for interior and exterior