Image Formation• The twoparts of the image formation processThe geometry of image formation which determines where in the image plane theprojection of a point in the scene will be located.The physics of light which determines the brightness of a point in the image plane asafunction of illumination and surface properties.• Asimple model-The scene is illuminated by a single source.-The scene reflects radiation towards the camera.-The camera senses it via chemicals on film.-2-• Camera Geometry-The simplest device to form an image of a 3D scene on a 2D surface is the "pinhole"camera.-Rays of light pass through a "pinhole" and form an inverted image of the object onthe image plane.• Camera Optics-Inpractice, the aperture must be larger to admit more light.-Lens are placed in the aperture to focus the bundle of rays from each scene pointonto the corresponding point in the image plane.-3-• Diffraction and Pinhole Optics-Ifweuse a wide pinhole, light from the source spreads across the image (i.e., notproperly focused), making it blurry.-Ifwenarrowthe pinhole, only a small amount of light is let in.*the image sharpness is limited by diffraction.*when light passes through a small aperture, it does not travelinastraight line.*itisscattered in manydirection (this is a quantum effect).-Ingeneral, the aim of using lens is to duplicate the pinhole geometry without resort-ing to undesirable small apertures.• Human Vision-Athigh light levels, pupil (aperture) is small and blurring is due to diffraction.-Atlow light levels, pupil is open and blurring is due to lens imperfections.-4-• CCD Cameras-Anarray of tinysolid state cells convert light energy into electrical charge.-Manufactured on chips typically measuring about 1cm x 1cm (for a 512x512 array,each element has a real width of roughly 0.001 cm).-The output of a CCD array is a continuous electric signal (video signal)which isgenerated by scanning the photo-sensors in a givenorder (e.g., line by line) and read-ing out their voltages.• Frame grabber-The video signal is sent to an electronic device called the frame grabber.-The frame grabber digitizes the signal into a 2D, rectangular array N x M of integervalues, stored in the frame buffer.-5-CCD array and frame buffer-InaCCD camera, the physical image plane is the CCD array of nxm rectangulargrid of photo-sensors.-The pixel image plane (frame buffer) is an array of NxM integer values (pixels).-The position of the same point on the image plane will be different if measured inCCD elements (x, y)orimage pixels (xim, yim).-Ingeneral, n ≠ N and m ≠ M;assuming that the origin in both cases is the upperleft corner we have:xim=Nnxyim=Mmywhere (xim, yim)are the coordinates of the point in the pixel plane and (x, y)are thecoordinates of the point in the CCD plane.-Ingeneral, it is convenient to assume that the CCD elements are always in one-to-one correspondence with the image pixels.-Units in each case:(xim, yim)ismeasured in pixels(x, y)ismeasured, e.g., in millimeters.-6-Reference Frames-Fiv e reference frames are needed for general problems in 3D scene analysis.• Object Coordinate Frame-This is a 3D coordinate system: xb, yb, zb-Itisused to model ideal objects in both computer graphics and computer vision.-Itisneeded to inspect an object (e.g., to check if a particular hole is in proper posi-tion relative toother holes)-The coordinates of 3D point B,e.g., relative tothe object reference frame are(xb,0, zb)-Object coordinates do not change regardless howthe object is placed in the scene.Notation: (Xo,Yo, Zo)T-7-• World Coordinate Frame-This is a 3D coordinate system: xw, yw, zw-The scene consists of object models that have been placed (rotated and translated)into the scene, yielding object coordinates in the world coordinate system.-Itisneeded to relate objects in 3D (e.g., the image sensor tells the robot where to topick up ta bolt and in which hole to insert it).Notation: (Xw,Yw, Zw)T• Camera Coordinate Frame-This is a 3D coordinate system (xc, yc, zcaxes)-Its purpose is to represent objects with respect to the location of the camera.Notation: (Xc,Yc, Zc)T-8-• Image Plane Coordinate Frame (CCD plane)-This is a 2D coordinate system (xf, yfaxes)-Describes the coordinates of 3D points projected on the image plane.-The projection of A,e.g., is point a whose both coordinates are negative.Notation: (x, y)T• Pixel Coordinate Frame-This is a 2D coordinate system (r, c axes)-Each pixel in this frame has an integer pixel coordinates.-Point A,e.g., gets projected to image point (ar, ac)where arand acare integer rowand column.Notation: (xim, yim)T-9-• Transformations between framesObject Coordinates (3D)World Coordinates (3D)Camera Coordinates (3D)Image Plane Coordinates (2D)Pixel Coordinates (2D, int)extrinsic cameraintrinsic