Image FormationCarlo TomasiThe images we process in computer vision are formed by light bouncing off surfaces in the world and into thelens of the system. The light then hits a sensor inside the camera and produces electric charges that are read by anelectronic circuit and converted to voltages. These are in turn sampled by a device called a digitizer (or frame grabber)to produce the numbers that computers eventually process, called pixel values. Thus, the pixel values are a ratherindirect encoding of the physical properties of visible surfaces.In fact, it does not cease to amaze me that all those numbers in an image file carry information on how the propertiesof a packet of photons were changed by bouncing off a surface in the world. Even more amazing is that from thisinformation we can perceive shapes and colors. Although we are used to these notions nowadays, the discovery ofhow images form, say, on our retinas, is rather recent. In ancient Greece, Euclid, in 300 B.C., attributed sight to theaction of rectilinear rays issuing from the observer’s eye, a theory that remained prevalent until the sixteenth Centurywhen Johannes Kepler explained image formation as we understand it now. In Euclid’s view, then, the eye is an activeparticipant in the visual process. Not a receptor, but an agent that reaches out to apprehend its object. One of Euclid’spostulates on vision maintained that any given object can be removed to a distance from which it will no longer bevisible because it falls between adjacent visual rays. This is ray tracing in a very concrete, physical sense!Studying image formation amounts to formulating models of the process that encodes the properties of light offa surface into brightness values in the image array. We start from what happens once light leaves a visible surface.What happens thereafter is in fact a function only of the imaging device, if we assume that the medium in-between istransparent. In contrast, what happens at the visible surface, although definitely of great interest in computer vision,is so to speak out of our control, because it depends on the reflectance properties of the surface. In other words,reflectance is about the world, not about the imaging process.The study of image formation can be further divided into what happens up to the point when light hits the sensor,and what happens thereafter. The first part occurs in the realm of optics, the second is a matter of electronics. We willlook at the optics first and at what is called sensing (the electronic part) later.Any model is a simplified description of reality. In image formation, it is convenient to take the extreme approachof defining a very simple model, and call everything else an “error”. Calibration is the process whereby the errors aredetermined for a given camera so they can be undone. This is a very useful approach. In fact, as a result of it, all of thetheory of computer vision can assume a mathematically simple imaging model, and the cameras are made to conformto it through calibration.To summarize, we will now study the optics of image formation, some aspects of sensing, and a few simplecalibration techniques. The calibration methods we study are not accurate enough for photogrammetric applicationslike drawing geographic maps from aerial imagery. However, they are good enough for removing gross discrepanciesbetween ideal and real images.1 OpticsA camera projects light from surfaces onto a two-dimensional sensor. Two aspects of this projection are of interesthere: where light goes is the geometric aspect, how much of it lands on the sensor is the photometric, or radiometric,aspect.11.1 GeometryOur idealized model for the optics of a camera is the so-called pinhole camera model, for which we define the geometryof perspective projection. All rays in this model, as we will see, go through a small hole, and form therefore a star oflines.For ever more distant scenes, the rays of the star become more and more parallel to each other, and the perspectiveprojection transformation performed by a pinhole camera tends to a limit called orthographic projection, where allrays are exactly parallel. Because orthographic projection is mathematically simpler than perspective, it is often amore convenient and more reliable model to use. We will look at both the perspective projection of the pinhole cameraand the orthographic projection model.1.1.1 Perspective ProjectionA pinhole camera is a box with a pinhole on one, opaque face and a translucent screen on the opposite face. All otherfaces are opaque. A cardboard pinhole camera is easy and instructive to build. Figure 1 shows what happens in thebox. Only a thin beam from a narrow set of directions hits any given point on the screen. Thus, the pinhole acts asa selector of light rays: without the pinhole and the box, any point on the screen would be illuminated from a wholehemisphere of directions, yielding a uniform coloring. With the pinhole, on the other hand, an inverted image of thevisible world is formed on the screen. When the pinhole is reduced to a single point, this image is formed by thestar of rays through the pinhole, intersected by the plane of the screen. Of course, a pinhole reduced to a point is anidealization: no power would pass through such a pinhole, and the image would be infinitely dim (black).imagepointpinholeworld pointFigure 1: Model for a pinhole camera.The fact that the image on the screen is inverted is mathematically inconvenient. It is therefore customary toconsider instead the intersection of the star of rays through the pinhole with a plane parallel to the screen and in frontof the pinhole as shown in figure 2. This is of course an idealization, since a screen in this position would blockthe light rays. In this model, the pinhole is called more appropriately the center of projection. The new image isisomorphic to the old one. The new plane is often placed at unit distance from the center of projection to simplify theprojection equations.Mathematically, the easiest way to describe this situation is to select a spherical coordinate system with its originat the pinhole, as done in figure 3. The choice of reference directions is not critical, but it is natural to select one axisas the optical axis, defined as the line through the pinhole orthogonal to the screen, and the other axis as being parallelto the horizontal lines on the screen. Horizontal, here, is either an arbitrary direction or the direction on the
View Full Document
Unlocking...