10/9/2002 CS 461, Copyright G.D. HagerComputer Vision, Lecture 11Professor Hagerhttp://www.cs.jhu.edu/~hagerJason Corsohttp://www.cs.jhu.edu/~jcorsoNotes from today’s lecture are downloadable there.10/9/2002 CS 461, Copyright G.D. HagerOutline for Today• Camera Calibration– Motivation– Review of Projective Camera Model– Methods for Calibration– The Multiplane Method in Detail10/9/2002 CS 461, Copyright G.D. HagerMotivation• Computer Vision is …– A conglomerate of aspects from many fields.– A science whose main goal is to compute properties of the 3-D world from one or more digital images.– So far in the course has been largely spent in “image space.”– Move beyond the images and infer properties of the metric world.– To do so, we need to know the parameters of projection; the camera parameters.10/9/2002 CS 461, Copyright G.D. HagerCamera parameters• Summary:– points expressed in external frame– points are converted to canonical camera coordinates– points are projected– points are converted to pixel units=TZYXWVUparameters extrinsicngrepresentitionTransformamodel projectionngrepresentitionTransformaparameters intrinsic ngrepresentitionTransformapoint in cam. coords.point in metricimage coords.point in pixelcoords.point in world coords.10/9/2002 CS 461, Copyright G.D. HagerExtrinsic ParametersUsing the idea of homogeneous coordinates,we can write:pTRp=1000'R and T both require 3 parameters. These correspondto the 6 extrinsic parameters needed for camera calibration10/9/2002 CS 461, Copyright G.D. HagerIntrinsic ParametersIntrinsic Parameters describe the conversion fromunit focal length metric to pixel coordinates (and the reverse)xmm= - (xpix–ox) sx ! -/sxxmm–ox= -xpixymm= - (ypix–oy) sy ! -/syymm–oy= -ypixpMwyxososwyxmmyyxxpixint100/100/1=−−=orIt is common to combine scale and focal length togetheras the are both scaling factors; note projection is unitless in this case!10/9/2002 CS 461, Copyright G.D. HagerThe problem:Compute the camera intrinsic (4 or 5) and extrinsic parameters (6) using only observed camera data.Calibration – Problem Statement10/9/2002 CS 461, Copyright G.D. HagerTypes of Calibration• Photogrammetric Calibration• Self Calibration• Multi-Plane Calibration• Others not discussed:– Pure Rotation– Vanishing Points10/9/2002 CS 461, Copyright G.D. HagerPhotogrammetric Calibration• Calibration is performed through imaging a pattern whose geometry in 3d is known with high precision.• PRO: Calibration can be performed very efficiently• CON: Expensive set-up apparatus is required; multiple orthogonal planes.• Approach 1: Direct Parameter Calibration• Approach 2: Projection Matrix Estimation10/9/2002 CS 461, Copyright G.D. HagerThe General Case• Affine is “easy” because it is linear and unconstrained (note orthographic is harder because of constraints)• Perspective case is also harder because it is both nonlinear andconstrained• Observation: optical center can be computed from the orthocenterof vanishing points of orthogonal sets of lines.10/9/2002 CS 461, Copyright G.D. HagerBasic Equations10/9/2002 CS 461, Copyright G.D. HagerBasic Equations10/9/2002 CS 461, Copyright G.D. HagerBasic Equationsone of these for each point10/9/2002 CS 461, Copyright G.D. HagerBasic Equations10/9/2002 CS 461, Copyright G.D. HagerProperties of SVD Again• Recall the singular values of a matrix are related to its rank.• Recall that Ax = 0 can have a nonzero x as solution only if A issingular.• Finally, note that the matrix V of the SVD is an orthogonal basis for the domain of A; in particular the zero singular values are the basis vectors for the null space.• Putting all this together, we see that A must have rank 7 (in this particular case) and thus x must be a vector in this subspace. • Clearly, x is defined only up to scale.10/9/2002 CS 461, Copyright G.D. HagerBasic EquationsWe now know Rxand Ryup to a sign and gamma.Rz= Rxx RyWe will probably use another SVD to orthogonalizethis system (R = U D V’; set D to I and multiply).10/9/2002 CS 461, Copyright G.D. HagerLast Details• We still need to compute the correct sign.– note that the denominator of the original equations must be positive (points must be in front of the cameras)– Thus, the numerator and the projection must disagree in sign.– We know everything in numerator and we know the projection, hence we can determine the sign.• We still need to compute Tzand fx– we can formulate this as a least squares problem on those two values using the first equation.10/9/2002 CS 461, Copyright G.D. HagerDirect Calibration: The Algorithm1. Compute image center from orthocenter2. Compute the A matrix (6.8)3. Compute solution with SVD4. Compute gamma and alpha5. Compute R (and normalize)6. Compute fxand and Tz10/9/2002 CS 461, Copyright G.D. HagerIndirect Calibration: The Basic Idea• We know that we can also just write– uh= M ph– x = (u/w) and y = (v/w), uh= (u,v,1)’– As before, we can multiply through (after plugging in for u,v, and w)• Once again, we can write–A m = 0• Once again, we use an SVD to compute m up to a scale factor.10/9/2002 CS 461, Copyright G.D. HagerGetting The Camera Parameters10/9/2002 CS 461, Copyright G.D. HagerGetting The Camera ParametersFIRST:|q3| is scale up to sign;divide by this valueM3,4is Tzup to sign, but Tzmust be positive; if not divide M by -1ox= q1. q3oy= q2. q3fx= (q1. q1–ox2)1/2fy= (q2. q2–oy2)1/2THEN:Ry= (q2–oyRz)/fyRx= Ryx RzTx= -(q4,1–oxTz)/fxTy= -(q4,2–oyTz)/fyFinally, use SVD to orthogonalize the rotation,10/9/2002 CS 461, Copyright G.D. HagerSelf-Calibration• Calculate the intrinsic parameters solely from point correspondences from multiple images.• Static scene and intrinsics are assumed.• No expensive apparatus.• Highly flexible but not well-established.• Projective Geometry – image of the absolute conic.10/9/2002 CS 461, Copyright G.D. HagerMulti-Plane Calibration• Hybrid method: Photogrammetric and Self-Calibration.• Uses a planar pattern imaged multiple times (inexpensive).• Used widely in practice and there are many implementations.• Based on a group of projective