Camera Models class 8Multiple View Geometry course schedule (subject to change)Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Summary of Properties of a Projective CameraSlide 39Slide 40Next class: Camera calibrationCamera Modelsclass 8Multiple View GeometryComp 290-089Marc PollefeysMultiple View Geometry course schedule(subject to change)Jan. 7, 9 Intro & motivation Projective 2D GeometryJan. 14, 16 (no class) Projective 2D GeometryJan. 21, 23Projective 3D Geometry (no class)Jan. 28, 30Parameter Estimation Parameter EstimationFeb. 4, 6 Algorithm Evaluation Camera ModelsFeb. 11, 13Camera Calibration Single View GeometryFeb. 18, 20Epipolar Geometry 3D reconstructionFeb. 25, 27Fund. Matrix Comp. Structure Comp.Mar. 4, 6 Planes & Homographies Trifocal TensorMar. 18, 20Three View ReconstructionMultiple View GeometryMar. 25, 27MultipleView ReconstructionBundle adjustmentApr. 1, 3 Auto-Calibration PapersApr. 8, 10 Dynamic SfM PapersApr. 15, 17Cheirality PapersApr. 22, 24Duality Project DemosN measurements (independent Gaussian noise 2) model with d essential parameters(use s=d and s=(N-d))(i) RMS residual error for ML estimator(ii) RMS estimation error for ML estimator 2/12/12/1/XXˆNdNEeres 2/12/12//XXˆNdNEeestnXXXSMError in two images2/124nneres2/124nneestnNnd 4 and 28 Backward propagation of covarianceX f -1PX -11xTPJJOver-parameterizationA  JJ1xTPJfvForward propagation of covarianceJJPTXMonte-Carlo estimation of covariance=1 pixel  =0.5cm(Crimisi’97)Example:Single view geometryCamera modelCamera calibrationSingle view geom.Pinhole camera modelTTZfYZfXZYX )/,/(),,( 101001ZYXffZfYfXZYXPinhole camera model10100ZYXffZfYfX10101011ZYXffZfYfXPXx  0|I)1,,(diagP ffPrincipal point offsetTyxTpZfYpZfXZYX )/,/(),,( principal pointTyxpp ),(101001ZYXpfpfZZpfYZpfXZYXyxyxPrincipal point offset10100ZYXpfpfZZpfYZpfXyxxx camX0|IKx 1yxpfpfKcalibration matrixCamera rotation and translation C~-X~RX~camX10C~RR110C~RRXcamZYX camX0|IKx  XC~|IKRx  t|RKP C~Rt PXx CCD camera1yyxxppK11yxyxpfpfmmKFinite projective camera1yyxxppsK C~|IKRP non-singular11 dof (5+3+3)decompose P in K,R,C? 4p|MP 41pMC~  MRK, RQ{finite cameras}={P3x4 | det M≠0}If rank P=3, but rank M<3, then cam at infinityCamera anatomyCamera centerColumn pointsPrincipal planeAxis planePrincipal pointPrincipal rayCamera center0PC null-space camera projection matrixC is camera centerImage of camera center is (0,0,0)T, i.e. undefinedFinite cameras: 1pM41CInfinite cameras: 0Md,0dCColumn vectors   0010ppppp43212Image points corresponding to X,Y,Z directions and originRow vectors1ppp0321ZYXyxTTT1ppp0321ZYXwyTTTnote: p1,p2 dependent on image reparametrizationThe principal pointprincipal point 0,,,pˆ3332313ppp330MmpˆPx The principal axis vector3m camcamcamX0|IKXPx    T1,0,0mMdetv3camcamPP kvv4k  4p|MC~|IKRP kvector defining front side of camera(direction unaffected) vmMdetv43kkk camcamPP kAction of projective camera on pointPXx  MdDp|MPDx4Forward projectionBack-projectionxPX 1PPPPTTIPP (pseudo-inverse)0PC  λCxPλX   1p-μxM1pM-0xMμλX4-14-1-1xMd-1CDDepth of points  C~X~mCXPXPT3T3T3w(dot product)(PC=0)1m;0det3MIf , then m3 unit vector in positive direction 3m)sign(detMPX;depthTw TX X,Y,Z,TCamera matrix decompositionFinding the camera center0PC (use SVD to find null-space)  432p,p,pdetX  431p,p,pdetY  421p,p,pdetZ  321p,p,pdetTFinding the camera orientation and internal parametersKRM (use RQ ecomposition)When is skew non-zero?1yxxxppsK1arctan(1/s)for CCD/CMOS, always s=0Image from image, s≠0 possible(non coinciding principal axis)HPresulting camera:Euclidean vs. projective   homography 44010000100001homography 33P general projective interpretationMeaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean spaceCameras at infinityCamera center at infinity0Mdet Affine and non-affine camerasDefinition: affine camera has P3T=(0,0,0,1)Affine camerasAffine cameras C~rrC~rrC~rrKC~|IKRP3T3T2T2T1T1T0C~r3T0d  