13D Computer Visionand Video ComputingImage FormationImage FormationCSc I6716Spring 2011Topic 1 of Part IImage FormationpgZhigang Zhu, City College of New York [email protected] Computer Visionand Video ComputingAcknowledgementsAcknowledgementsTh lid i thi l t ki dl id d bThe slides in this lecture were kindly provided byProfessor Allen HansonUniversity of Massachusetts at Amherst23D Computer Visionand Video ComputingLecture OutlineLecture Outline Image Formation Basic Steps Geometryz Pinhole camera model & Thin lens modelz Perspective projection & Fundamental equationRadiometry Photometryz Color, human vision, & digital imaging DigitalizationSli titi&t lltizSampling, quantization & tessellations More on Digital Imagesz Neighbors, connectedness & distances3D Computer Visionand Video ComputingLecture OutlineLecture Outline Image Formation Basic Steps Geometryz Pinhole camera model & Thin lens modelz Perspective projection & Fundamental equationRadiometry Photometryz Color, human vision, & digital imaging DigitalizationSli titi&t lltizSampling, quantization & tessellations More on Digital Imagesz Neighbors, connectedness & distances33D Computer Visionand Video ComputingAbstract ImageAbstract Image An image can be represented by an image function whose general form is f(x,y).f(x y)is a vector-valued function whose argumentsf(x,y)is a vector-valued function whose arguments represent a pixel location. The value of f(x,y) can have different interpretations in different kinds of images.ExamplesIntensity Image - f(x,y) = intensity of the sceneRange Image - f(x,y) = depth of the scene from imaging systemColor Image - f(x,y) = {fr(x,y), fg(x,y), fb(x,y)}Video - f(x,y,t) = temporal image sequence3D Computer Visionand Video ComputingBasic RadiometryBasic Radiometry Radiometry is the part of image formation concerned with the relation among the amounts of light energy emitted from light sources, reflected from surfaces, and registeredfrom light sources, reflected from surfaces, and registered by sensors.OpticsCCD ArrayLight SourcenpSurfacePL(P,d)ie43D Computer Visionand Video ComputingLight and MatterLight and MatterThe interaction between light and matter can takeThe interaction between light and matter can take many forms:z Reflectionz Refractionz Diffractionz AbsorptionScatteringzScattering3D Computer Visionand Video ComputingLecture AssumptionsLecture Assumptions Typical imaging scenario:z visible lightz ideal lensesz standard sensor (e.g. TV camera)z opaque objects GoalTt'diitl'i hihbTo create 'digital' images which can be processed to recover some of the characteristics of the 3D world which was imaged.53D Computer Visionand Video ComputingImage FormationImage FormationLight (Energy) SourceSurfacePinhole LensImaging PlaneWorld Optics SensorSignalgB&W FilmColor FilmTV CameraSilver DensitySilver densityin three colorlayersElectrical3D Computer Visionand Video ComputingStepsStepsWorld Optics SensorWorld Optics SensorSignal Digitizer Digital RepresentationWorld realityOptics focus {light} from world on sensorp{g }Sensor converts {light} to {electrical energy}Signal representation of incident light as continuous electrical energyDigitizer converts continuous signal to discrete signalDigital Rep. final representation of reality in computer memory63D Computer Visionand Video ComputingFactors in Image FormationFactors in Image Formation Geometryz concerned with the relationship between points in the three-dimensional world and their images Radiometryz concerned with the relationship between the amount of light radiating from a surface and the amount incident at its imagePh t tPhotometryz concerned with ways of measuring the intensity of light Digitizationz concerned with ways of converting continuous signals (in both space and time) to digital approximations3D Computer Visionand Video ComputingLecture OutlineLecture Outline Image Formation Basic Steps Geometryz Pinhole camera model & Thin lens modelz Perspective projection & Fundamental equationRadiometry Photometryz Color, human vision, & digital imaging DigitalizationSli titi&t lltizSampling, quantization & tessellations More on Digital Imagesz Neighbors, connectedness & distances73D Computer Visionand Video ComputingGeometryGeometry Geometry describes the projection of:two-dimensional (2D) i lthree-dimensional (3D) ld(2D) image plane.(3D) world  Typical Assumptionsz Light travels in a straight line Optical Axis: the axis perpendicular to the image plane and passing through the pinhole (also called the central projection ray)Each point in the image corresponds to a particular direction defined byEach point in the image corresponds to a particular direction defined by a ray from that point through the pinhole. Various kinds of projections:z - perspective - obliquez - orthographic - isometricz - spherical3D Computer Visionand Video ComputingBasic OpticsBasic Optics Two models are commonly used:z Pin-hole cameraz Optical system composed of lenses Pin-hole is the basis for most graphics and visionz Derived from physical construction of early camerasz Mathematics is very straightforward Thin lens model is first of the lens modelszMathematical model for a physical lenszMathematical model for a physical lensz Lens gathers light over area and focuses on image plane.83D Computer Visionand Video ComputingPinhole Camera ModelPinhole Camera ModelImage Plane World projected to 2D Imagez Image invertedPinhole lensOptical Axisfgz Size reducedz Image is dimz No direct depth information f called the focal length of the lens Known as perspective projection3D Computer Visionand Video ComputingPinhole camera imagePinhole camera imageAmsterdamPhoto by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html93D Computer Visionand Video ComputingEquivalent GeometryEquivalent Geometry Consider case with object on the optical axis:ffz More convenient with upright image:- fzProjection plane z = 0 Equivalent mathematically3D Computer Visionand Video ComputingfThin Lens ModelThin Lens Model Rays entering parallel on one side converge at focal point. Rays diverging from the focal point become parallel.fioIMAGEPLANEOPTICAXISLENS1 1 1f i o=+‘THIN LENS LAW’103D Computer Visionand Video ComputingCoordinate SystemCoordinate System