Perspective Projection Describes Image Formation Berthold K.P. HornWheel Alignment:Camber, Caster, Toe-In, SAI, …Camber: angle between axle and horizontal plane.Toe: angle between projection of axle on horizontal plane and a per-pendicular to thrust line.Caster: angle in side elevation between steering axis and vertical.SAI: angle in frontal elevation between steering axis and vertical.Planar TargetMounted Rigidly on RimCCD Cameras and LED IlluminationDetermining Rolling AxisDetermining Steering AxisPhotogrammetric Problems:• Interior Orientation (2D ↔ 3D)• Exterior Orientation (3D ↔ 2D)• Absolute Orientation (3D ↔ 3D)• Relative Orientation (2D ↔ 2D)Interior Orientation:Principal Point + Principal DistanceCenter of Projection: (uo, vo, f)Exterior Orientation:Rotation + Translation of Camera: R and tRigid body transformation:rotation and translation —of object coordinate system into camera coordinate system.xcyczc= Rxtytzt+xoyozo(1)R isa3× 3 orthonormal matrix that represents rotation,while t = (xo,yo,zo)Trepresents translation.Constraints:Orthonormality: RTR = I(six independent second order constraints).Rotation rather than reflection: det(R) =+1(one third order constraint).Perspective projection:Camera coordinate system (3D) into image coordinate system (2D) (*) (**).u = f(xc/zc) + uov = f(yc/zc) + vo(2)f is principal distance — “effective focal length”while (uo,vo)Tis principal point — “image center”Interior orientation of the camera is given by (uo,vo,f)T.(*) Camera coordinate system origin is at center of projection.(**) Camera coordinate system is aligned with image axes.Planar object:Let zt= 0 in the plane of the object.xcyczc=r11r12r13r21r22r23r31r32r33xtyt0+xoyozo(3)Absorbing the translation (xo,yo,zo)Tinto the 3 × 3 matrix,and dividing the third component by f we get:xcyczc/f=r11r12xor21r22yor31/f r32/f zo/fxtyt1(4)Now perspective projection (eq. 2) gives (if we let k = zc/f ):k(u − uo)k(v − vo)k=r11r12xor21r22yor31/f r32/f zo/fxtyt1(5)(Odd form makes it easier to match result with projective geometry formulation).Homogeneous Representation:Homogeneous representation of point in plane uses three numbers [Wylie 70].(u, v, w)TActual planar coordinates are obtained by dividing first two elements by third:x = u/w and y = v/wRepresentation not unique since (ku, kv, kw)Tcorresponds to same point.A3× 3 matrix T can represent a homogeneous transformationfrom the object plane to the image plane.kukvk=t11t12t13t21t22t23t31t32t33xtyt1(6)Matching Homogeneous Transformation:We can matchk(u − uo)k(v − vo)k=r11r12xor21r22yor31/f r32/f zo/fxtyt1,(5)withkukvk=t11t12t13t21t22t23t31t32t33xtyt1(6)providedt11= r11,t12= r12,t13= xot21= r21,t22= r22,t23= yot31= r31/f , t32= r32/f , t33= zo/f(7)(In addition, measurements in the image must be made in a coordinate systemwith the origin at the principal point, so that uo, vo= 0.)Constraints on Transform:T same as kT (Scale factor ambiguity). Pick t33= 1 (leaves 8 DOF).T must satisfy two non-linear constraintst11t12+ t21t22+ f2t31t32= 0 (8)andt11t11+ t21t21+ f2t31t31= t12t12+ t22t22+ f2t32t32(9)(from the orthonormality of R).Two non-linear constraints reduce DOF from 8 to 6 —rotation has 3 DOF and translation has 3 DOF.Projective Transformation versus Perspective Projection:Given real perspective projection, one can always compute a matrix T .But it is not possible to go in the other direction —for most matrices T , there is no corresponding perspective projection.Only a subset of measure of zero of homogeneous transformations Tallow physical interpretation as rigid body motion and perspective projection.An arbitrary 3 × 3 matrix T will not satisfy the two non-linear constraints.Disallowed Mappings:Mappings allowed by projective geometry but not by perspective projection —skewing and anisotropic scalingDistort a normal perspective image by the additional operations1 s01or100 kresult is not an “image” —from any position with any camera orientation.Yet such distortions merely transform matrix T —thus are allowed in projective geometry.Disallowed Mappings:Picture of a Picture:Is there a physical imaging situation that corresponds tohomogeneous transformation by an arbitrary matrix T ?The transformations of perspective geometry correspond totaking a perspective image of a perspective image.In this case, the overall transformation need not satisfy the non-linear constraints.But, we are interested in “direct” images, not pictures taken of a picture!Finding T:Correspondence between (xi,yi, 1)Tand (kui,kvi,k)T(Horn 86):xit11+ yit12+ t13− xiuit31− yiuit32− uit33= 0xit21+ yit22+ t23− xivit31− yivit32− vit33= 0Four correspondences yield system of 8 homogeneous equations:x1y110 00−x1u1−y1u1−u1x2y210 00−x2u2−y2u2−u2x3y310 00−x3u3−y3u3−u3x4y410 00−x4u4−y4u4−u4000x1y11 −x1v1−y1v1−v1000x2y21 −x2v2−y2v2−v2000x3y31 −x3v3−y3v3−v3000x4y41 −x4v4−y4v4−v4t11t12t13t21t22t23t31t32t33=000000000T has 9 elements, but only 8 DOF, since any multiple of T describes same mapping.Arbitrarily pick value for one element of T , say t33= 1.Leaves 8 non-homogeneous equations in 8 unknown.Recovering Orientation using Vanishing PointsConsider(αa, αb, 1)Tas α →∞. That is, (a, b, 0)T.The vanishing point for line with direction (a, b)Tin object plane isu = (t11a + t12b)/(t31a + t32b)v = (t21a + t22b)/(t31a + t32b)(10)The line from the COP (0, 0, 0)T, to vanishing point in image plane (u, v, f )T,is parallel (in 3D) to line on object.Apply to x- and y-axes of object, and from the two vanishing pointsfind directions of the two axes in camera coordinate system.Recovering Orientation using Vanishing PointsVanishing point for x-axis is (1, 0, 0)Tin object coordinate system.T(1, 0, 0)T=