1CSE152, Spr 06 Intro Computer VisionImage Formation and CamerasIntroduction to Computer VisionCSE 152Lecture 3CSE152, Spr 06 Intro Computer VisionAnnouncements• Assignment 0: Posted to web page, due on Thursday• Matlab will be discussed during discussion section, Wed 9:00 - 9:50 AM in WLH 2208• Read Trucco & Verri: pp. 15-40 CSE152, Spr 06 Intro Computer VisionImage Formation: Outline• Factors in producing images• Projection• Perspective• Vanishing points• Orthographic•Lenses•Sensors• Quantization/Resolution• Illumination• ReflectanceCSE152, Spr 06 Intro Computer VisionEarliest Surviving Photograph• First photograph on record, “la table service” byNicephore Niepce in 1822.• Note: First photograph by Niepce was in 1816.CSE152, Spr 06 Intro Computer VisionHow Cameras Produce Images• Basic process:– photons hit a detector– the detector becomes charged– the charge is read out as brightness• Sensor types:– CCD (charge-coupled device)• high sensitivity• high power• cannot be individually addressed• blooming–CMOS• most common• simple to fabricate (cheap)• lower sensitivity, lower power• can be individually addressedCSE152, Spr 06 Intro Computer VisionImages are two-dimensional patterns of brightness values.They are formed by the projection of 3D objects.Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.2CSE152, Spr 06 Intro Computer VisionEffect of Lighting: MonetCSE152, Spr 06 Intro Computer VisionChange of Viewpoint: MonetHaystack at Chailly at Sunrise (1865)CSE152, Spr 06 Intro Computer VisionPinhole Camera: Perspective projection• Abstract camera model - box with a small hole in itForsyth&PonceCSE152, Spr 06 Intro Computer Visionhttp://www.acmi.net.au/AIC/CAMERA_OBSCURA.html (Russell Naughton)Camera Obscura"When images of illuminated objects ... penetrate through a small hole into a very dark room ... you will see [on the opposite wall] these objects in their proper form and color, reduced in size ... in a reversed position, owing to the intersection of the rays".Da VinciCSE152, Spr 06 Intro Computer Vision• Used to observe eclipses (eg., Bacon, 1214-1294)• By artists (eg., Vermeer).CSE152, Spr 06 Intro Computer Visionhttp://brightbytes.com/cosite/collection2.html (Jack and Beverly Wilgus)Jetty at Margate England, 1898.3CSE152, Spr 06 Intro Computer VisionDistant objects are smaller(Forsyth & Ponce)CSE152, Spr 06 Intro Computer VisionGeometric properties of projection• Points go to points• Lines go to lines• Planes go to whole image or half-plane• Polygons go to polygons• Angles & distances not preserved• Degenerate cases:– line through focal point yields point– plane through focal point yields lineCSE152, Spr 06 Intro Computer VisionParallel lines meet in the image• vanishing pointImage planeCSE152, Spr 06 Intro Computer VisionCSE152, Spr 06 Intro Computer VisionVanishing pointsVPLVPRHVP1VP2VP3To different directions correspond different vanishing pointsCSE152, Spr 06 Intro Computer VisionVanishing Points4CSE152, Spr 06 Intro Computer VisionThe equation of projectionCartesian coordinates:• We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f)• Ignore the third coordinate, and get(x,y,z) → ( fxz, fyz)CSE152, Spr 06 Intro Computer VisionA DigressionHomogenous Coordinatesand Camera MatricesCSE152, Spr 06 Intro Computer VisionHomogenous coordinates• Our usual coordinate system is called a Euclidean or affine coordinate system• Rotations, translations and projection in Homogenous coordinates can be expressed linearly as matrix multipliesEuclideanWorld3DHomogenousWorld3DHomogenousImage2DEuclideanWorld2DConvertConvertProjectionCSE152, Spr 06 Intro Computer VisionHomogenous coordinatesA way to represent points in a projective space1. Add an extra coordinatee.g., (x,y) -> (x,y,1)=(u,v,w)2. Impose equivalence relation such that (λ not 0)(u,v,w) ≈ λ*(u,v,w) i.e., (x,y,1) ≈ (λx, λy, λ)3. “Point at infinity” – zero for last coordinatee.g., (x,y,0)• Why do this?– Possible to represent points “at infinity”• Where parallel lines intersect• Where parallel planes intersect– Possible to write the action of a perspective camera as a matrixCSE152, Spr 06 Intro Computer VisionEuclidean -> Homogenous-> Euclidean In 2-D• Euclidean -> Homogenous: (x, y) -> k (x,y,1)• Homogenous -> Euclidean: (u,v,w) -> (u/w, v/w)In 3-D• Euclidean -> Homogenous: (x, y, z) -> k (x,y,z,1)• Homogenous -> Euclidean: (x, y, z, w) -> (x/w, y/w, z/w)CSE152, Spr 06 Intro Computer VisionThe camera matrixTurn this expression into homogenous coordinates– HC’s for 3D point are (X,Y,Z,T)– HC’s for point in image are (U,V,W)UVW⎛ ⎝ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ =10 0 001 0 0001f0⎛ ⎝ ⎜ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ ⎟ XYZT⎛ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎞⎠⎟⎟⎟⎟⎟(x,y,z) → ( fxz, fyz)PerspectiveCamera MatrixA 3x4 matrix5CSE152, Spr 06 Intro Computer VisionEnd of the DigressionCSE152, Spr 06 Intro Computer VisionAffine Camera Model• Take Perspective projection equation, and perform Taylor Series Expansion about (some point (x0,y0,z0).• Drop terms of higher order than linear.• Resulting expression is affine camera model⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡000zyxAppropriate in NeighborhooAbout (x0,y0,z0CSE152, Spr 06 Intro Computer Vision• Perspective• Assume that f=1, and perform a Taylor series expansion about (x0, y0, z0)• Dropping higher order terms and regrouping.⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡yxzfvu() ()() ()L+−⎥⎦⎤⎢⎣⎡+−⎥⎦⎤⎢⎣⎡+−⎥⎦⎤⎢⎣⎡+−⎥⎦⎤⎢⎣⎡−⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡20003000000002000022110101111zzyxzyyzxxzzzyxzyxzvubAp +=⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎥⎦⎤⎢⎣⎡−−+⎥⎦⎤⎢⎣⎡≈⎥⎦⎤⎢⎣⎡zyxzyzzxzyxzvu20002000000//10/0/11CSE152, Spr 06 Intro Computer VisionbAp +=⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎥⎦⎤⎢⎣⎡−−+⎥⎦⎤⎢⎣⎡≈⎥⎦⎤⎢⎣⎡zyxzyzzxzyxzvu20002000000//10/0/11Rewrite Affine camera modelin terms of Homogenous Coordinates⎥⎥⎥⎥⎦⎤⎢⎢⎢⎢⎣⎡⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡−−≈⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡11000///10//0/1002000002000zyxzyzyzzxzxzwvuCSE152, Spr 06 Intro Computer Vision⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡yxzvu01Orthographic projectionStarting with Affine camera modeTake Taylor series