Last 4 lecturesTodayPinhole cameraPinhole camera modelPinhole camera modelPinhole camera modelPinhole camera modelIntrinsic matrixIntrinsic matrixIntrinsic matrixDistortionBarrel DistortionPin Cushion DistortionModeling distortionCamera rotation and translationTwo kinds of parametersOther projection modelsOrthographic projectionOther types of projectionsFun with perspectivePerspective cuesPerspective cuesFun with perspectiveForced perspective in LOTRCamera calibrationCamera calibrationCamera calibration approachesLinear regressionLinear regressionLinear regressionNormal equation (Geometric Interpretation)Normal equation (Differential Interpretation)Normal equationNonlinear optimizationOptimal estimationNonlinear least square methodsLeast square fittingNonlinear least square fittingFunction minimizationFunction minimizationQuadratic functionsFunction minimizationComputing gradient and HessianComputing gradient and HessianComputing gradient and HessianComputing gradient and HessianComputing gradient and HessianSearching for update hSteepest descent methodSteepest descent methodSearching for update hRecap: CalibrationRecap: CalibrationRecap: CalibrationPractical IssueA popular calibration toolMulti-plane calibration Step 1: data acquisitionStep 2: specify corner orderStep 3: corner extractionStep 3: corner extractionStep 4: minimize projection errorStep 4: camera calibrationStep 4: camera calibrationStep 5: refinementLast 4 lecturesCamera StructureHDRImage FilteringImage TransformTodayCamera ProjectionCamera CalibrationPinhole cameraPinhole camera model(optical center)originprincipal pointP(X,Y,Z)p(x,y)• The coordinate system– We will use the pin-hole model as an approximation– Put the optical center (Center Of Projection) at the origin– Put the image plane (Projection Plane) in front of the COP (Why?)Pinhole camera modelZfYyZfXx==principal point⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛ZYXffZfYfXyx1000000~1Pinhole camera model(optical center)originprincipal pointP(X,Y,Z)p(x,y)⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛ZYXffZfYfXyx1000000~1xyPinhole camera model(optical center)originprincipal pointP(X,Y,Z)p(x,y)⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛ZYXyfxfZfYfXyx10000~100xyIntrinsic matrix• non-square pixels (digital video)⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=10000ccyfxfK⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=10000cycxyfxfKIs this form of K good enough?Intrinsic matrix• non-square pixels (digital video)• skew⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=10000ccyfxfKIs this form of K good enough?⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=1000cycxyfxsfKIntrinsic matrix• non-square pixels (digital video)• skew• radial distortion⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=10000ccyfxfKIs this form of K good enough?⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=1000cycxyfxsfKDistortion• Radial distortion of the image– Caused by imperfect lenses– Deviations are most noticeable for rays that pass through the edge of the lensBarrel DistortionNo distortionBarrelWide Angle LensPin Cushion DistortionPin cushionTelephoto lensNo distortionModeling distortion• To model lens distortion– Use above projection operation instead of standard projection matrix multiplication2. Apply radial distortion3. Apply focal length translate image center1. Project (X, Y, Z)to “normalized” image coordinatesZYyZXxnn==()()4221422122211rryyrrxxyxrndndnnκκκκ++=++=+=cdcdyfyyxfxx+=+=ZfYyZfXx==Distortion-Free: With Distortion:Camera rotation and translationtR +⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛=⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛×ZYXZYX33'''[]⎟⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛110000~1ZYXyfxfyxcctR[]XtRKx ~extrinsic matrixTwo kinds of parameters• internal or intrinsic parameters: focal length, optical center, skew• external or extrinsic (pose): rotation and translation:[]⎟⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=⎟⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛1110000~1ZYXlkjihgfedcbaZYXyfxfyxcctROther projection modelsOrthographic projection• Special case of perspective projection– Distance from the COP to the PP is infiniteImageWorld– Also called “parallel projection”: (x, y, z) → (x, y)Other types of projections• Scaled orthographic– Also called “weak perspective”• Affine projection– Also called “paraperspective”Fun with perspectivePerspective cuesPerspective cuesFun with perspectiveAmes roomForced perspective in LOTRElijah Wood: 5' 6" (1.68 m) Ian McKellen 5' 11" (1.80 m)Camera calibrationCamera calibration• Estimate both intrinsic and extrinsic parameters• Mainly, two categories:1. Using objects with known geometry as reference 2. Self calibration (structure from motion)[]⎟⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=⎟⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎜⎝⎛⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛1110000~1ZYXlkjihgfedcbaZYXyfxfyxcctRCamera calibration approaches• Directly estimate 11 unknowns in the M matrix using known 3D points (Xi,Yi,Zi) and measured feature positions (ui,vi)[]MXXtRKx =~Linear regressionLinear regression⎥⎦⎤⎢⎣⎡=⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦⎤⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣⎡⎥⎦⎤⎢⎣⎡−−−−−−iiiiiiiiiiiiiiiiiiiivummmmmmmmmmmYvYvXvZYXZuYuXuZYX22212013121110030201001000000001Linear regressionSolve for Projection Matrix M using least-square techniques⎥⎦⎤⎢⎣⎡=⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦⎤⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣⎡⎥⎥⎥⎥⎥⎦⎤⎢⎢⎢⎢⎢⎣⎡−−−−−−−−−−−−iiNNNNNNNNNNNNNNNNNNvummmmmmmmmmmYvYvXvZYXZuYuXuZYXYvYvXvZYXZuYuXuZYX222120131211100302010011111111111111111110000000011000000001#Normal equation (Geometric Interpretation)Given an overdetermined systembAx=0)(T=−⇔ bAxAthe normal equation is that which minimizes the sum of the square differences between left and right sides2min bAx −()bxT1TAAA−=⇔Normal equation (Differential Interpretation)2)( bAxx
View Full Document
Unlocking...