1CSE190A, Fall 06 BiometricsImage Formation and CamerasBiometricsCSE 190ALecture 5CSE190A, Fall 06 BiometricsImage Formation: Outline• Factors in producing images• Projection• Perspective• Vanishing points• Orthographic•Lenses•Sensors• Quantization/Resolution• Illumination• ReflectanceCSE190A, Fall 06 BiometricsEarliest Surviving Photograph• First photograph on record, “la table service” byNicephore Niepce in 1822.• Note: First photograph by Niepce was in 1816.CSE190A, Fall 06 BiometricsImages are two-dimensional patterns of brightness values.They are formed by the projection of 3D objects.Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.CSE190A, Fall 06 BiometricsEffect of Lighting: MonetCSE190A, Fall 06 BiometricsChange of Viewpoint: MonetHaystack at Chailly at Sunrise (1865)2CSE190A, Fall 06 BiometricsPinhole Camera: Perspective projection• Abstract camera model - box with a small hole in itForsyth&PonceCSE190A, Fall 06 BiometricsGeometric properties of projection• Points go to points• Lines go to lines• Planes go to whole image or half-plane• Polygons go to polygons• Angles & distances not preserved• Degenerate cases:– line through focal point yields point– plane through focal point yields lineCSE190A, Fall 06 BiometricsParallel lines meet in the image• vanishing pointImage planeCSE190A, Fall 06 BiometricsThe equation of projectionCartesian coordinates:• We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f)• Ignore the third coordinate, and get(x,y,z)→ ( fxz, fyz)CSE190A, Fall 06 BiometricsA DigressionHomogenous Coordinatesand Camera MatricesCSE190A, Fall 06 BiometricsHomogenous coordinates• Our usual coordinate system is called a Euclidean or affine coordinate system• Rotations, translations and projection in Homogenous coordinates can be expressed linearly as matrix multipliesEuclideanWorld3DHomogenousWorld3DHomogenousImage2DEuclideanWorld2DConvertConvertProjection3CSE190A, Fall 06 BiometricsHomogenous coordinatesA way to represent points in a projective space1. Add an extra coordinatee.g., (x,y) -> (x,y,1)=(u,v,w)2. Impose equivalence relation such that (λ not 0)(u,v,w) ≈ λ*(u,v,w) i.e., (x,y,1) ≈ (λx, λy, λ)3. “Point at infinity” – zero for last coordinatee.g., (x,y,0)• Why do this?– Possible to represent points “at infinity”• Where parallel lines intersect• Where parallel planes intersect– Possible to write the action of a perspective camera as a matrixCSE190A, Fall 06 BiometricsEuclidean -> Homogenous-> Euclidean In 2-D• Euclidean -> Homogenous: (x, y) -> k (x,y,1)• Homogenous -> Euclidean: (u,v,w) -> (u/w, v/w)In 3-D• Euclidean -> Homogenous: (x, y, z) -> k (x,y,z,1)• Homogenous -> Euclidean: (x, y, z, w) -> (x/w, y/w, z/w)CSE190A, Fall 06 BiometricsThe camera matrixTurn this expression into homogenous coordinates– HC’s for 3D point are (X,Y,Z,T)– HC’s for point in image are (U,V,W)UVW⎛ ⎝ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ =10 0 001 0 0001f0⎛ ⎝ ⎜ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ ⎟ XYZT⎛ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎞⎠⎟⎟⎟⎟⎟(x,y,z)→ ( fxz, fyz)PerspectiveCamera MatrixA 3x4 matrixCSE190A, Fall 06 BiometricsEnd of the DigressionCSE190A, Fall 06 BiometricsAffine Camera Model• Take Perspective projection equation, and perform Taylor Series Expansion about (some point (x0,y0,z0).• Drop terms of higher order than linear.• Resulting expression is affine camera model⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡000zyxAppropriate in NeighborhoodAbout (x0,y0,z0)CSE190A, Fall 06 Biometrics• Perspective• Assume that f=1, and perform a Taylor series expansion about (x0, y0, z0)• Dropping higher order terms and regrouping.⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡yxzfvu() ()() ()L+−⎥⎦⎤⎢⎣⎡+−⎥⎦⎤⎢⎣⎡+−⎥⎦⎤⎢⎣⎡+−⎥⎦⎤⎢⎣⎡−⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡20003000000002000022110101111zzyxzyyzxxzzzyxzyxzvubAp +=⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎥⎦⎤⎢⎣⎡−−+⎥⎦⎤⎢⎣⎡≈⎥⎦⎤⎢⎣⎡zyxzyzzxzyxzvu20002000000//10/0/114CSE190A, Fall 06 BiometricsbAp +=⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎥⎦⎤⎢⎣⎡−−+⎥⎦⎤⎢⎣⎡≈⎥⎦⎤⎢⎣⎡zyxzyzzxzyxzvu20002000000//10/0/11Rewrite Affine camera modelin terms of Homogenous Coordinates⎥⎥⎥⎥⎦⎤⎢⎢⎢⎢⎣⎡⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡−−≈⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡11000///10//0/1002000002000zyxzyzyzzxzxzwvuCSE190A, Fall 06 Biometrics⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡yxzvu01Orthographic projectionStarting with Affine camera modeTake Taylor series about (0, 0, z0) – a point on optical axis CSE190A, Fall 06 BiometricsThe projection matrix for orthographic projection⎟⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎜⎝⎛⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛=⎟⎟⎟⎠⎞⎜⎜⎜⎝⎛TZYXzzWVU100000/10000/100Parallel lines project to parallel linesRatios of distances are preserved under orthographic CSE190A, Fall 06 BiometricsOther camera models• Generalized camera – maps points lying on rays and maps them to points on the image plane.Omnicam (hemispherical) Light Probe (spherical)CSE190A, Fall 06 BiometricsSome Alternative “Cameras”CSE190A, Fall 06 BiometricsWhat if camera coordinate system differs from object coordinate system{c}P{W}5CSE190A, Fall 06 BiometricsEuclidean Coordinate Systems⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=⇔++=⇔⎪⎩⎪⎨⎧===zyxzyxOPOPzOPyOPxPkjikji...CSE190A, Fall 06 BiometricsCoordinate Changes: Rigid TransformationsABABABOPRP +=Rotation MatrixTranslation vectorCSE190A, Fall 06 BiometricsA rotation matrix R has the following properties:• Its inverse is equal to its transpose R-1= RT• its determinant is equal to 1: det(R)=1.Or equivalently:• Rows (or columns) of R form a right-handedorthonormal coordinate system.CSE190A, Fall 06 BiometricsRotation: Homogenous Coordinates• About z axisx'y'z'1=xyz1cos θsin θ00-sin θcos θ0000100001rot(z,θ)xyzpp'θCSE190A, Fall 06 BiometricsRoll-Pitch-Yaw),ˆ(),ˆ(),ˆ(ϕβαkrotjrotirotR =),ˆ(),'ˆ(),''ˆ(ϕβαkrotjrotkrotR =Euler AnglesCSE190A, Fall 06 BiometricsRotation• About (kx, ky, kz), a unit vector on an arbitrary axis(Rodrigues Formula)x'y'z'1=xyz1kxkx(1-c)+ckykx(1-c)+kzskzkx(1-c)-kys00001kzkx(1-c)-kzskzkx(1-c)+ckzkx(1-c)-kxs0kxkz(1-c)+kyskykz(1-c)-kxskzkz(1-c)+c0where c = cos θ & s = sin θRotate(k, θ)xyzθk6CSE190A,