Capturing Light… in man and machine15-463: Computational PhotographyAlexei Efros, CMU, Fall 2007Image FormationDigital CameraThe EyeFilmDigital cameraA digital camera replaces film with a sensor array• Each cell in the array is light-sensitive diode that converts photons to electrons• Two common types– Charge Coupled Device (CCD)–CMOS• http://electronics.howstuffworks.com/digital-camera.htmSlide by Steve SeitzSensor ArrayCMOS sensorSampling and QuantizationInterlace vs. progressive scanhttp://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve SeitzProgressive scanhttp://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve SeitzInterlacehttp://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve SeitzThe EyeThe human eye is a camera!• Iris - colored annulus with radial muscles• Pupil - the hole (aperture) whose size is controlled by the iris• What’s the “film”?– photoreceptor cells (rods and cones) in the retinaSlide by Steve SeitzThe RetinaCross-section of eyeGanglion cell layerBipolar cell layerReceptor layerPigmentedepitheliumGanglion axonsCross section of retinaRetina up-closeLight© Stephen E. Palmer, 2002Conescone-shaped less sensitiveoperate in high lightcolor visionTwo types of light-sensitive receptorsconerodRodsrod-shapedhighly sensitiveoperate at nightgray-scale visionRod / Cone sensitivityThe famous sock-matching problem…© Stephen E. Palmer, 2002Distribution of Rods and Cones0150,000100,00050,000020 40 60 8020406080Visual Angle (degrees from fovea)RodsConesConesRodsFoveaBlindSpot# Receptors/mm2Night Sky: why are there more stars off-center?Electromagnetic Spectrumhttp://www.yorku.ca/eye/photopik.htmHuman Luminance Sensitivity FunctionWhy do we see light of these wavelengths?© Stephen E. Palmer, 2002.0 1000 2000 3000EnergyWavelength (nm)400 700700 C2000 C5000 C10000 CVisibleRegion…because that’s where theSun radiates EM energyVisible LightThe Physics of LightAny patch of light can be completely describedphysically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm.400 500 600 700Wavelength (nm.)# Photons(per ms.)© Stephen E. Palmer, 2002The Physics of Light# PhotonsD. Normal DaylightWavelength (nm.)B. Gallium Phosphide Crystal400 500 600 700# PhotonsWavelength (nm.)A. Ruby Laser400 500 600 700400 500 600 700# PhotonsC. Tungsten Lightbulb400 500 600 700# PhotonsSome examples of the spectra of light sources© Stephen E. Palmer, 2002The Physics of LightSome examples of the reflectance spectra of surfacesWavelength (nm)% Photons ReflectedRed400 700Yellow400 700Blue400 700Purple400 700© Stephen E. Palmer, 2002The Psychophysical CorrespondenceThere is no simple functional description for the perceivedcolor of all lights under all viewing conditions, but …...A helpful constraint:Consider only physical spectra with normal distributionsareaWavelength (nm.)# Photons400 700500 600meanvariance© Stephen E. Palmer, 2002The Psychophysical CorrespondenceMeanHueyellowgreenblue# PhotonsWavelength© Stephen E. Palmer, 2002The Psychophysical CorrespondenceVariance SaturationWavelengthhighmediumlowhi.med.low# Photons© Stephen E. Palmer, 2002The Psychophysical CorrespondenceArea Brightness# PhotonsWavelengthB. Area Lightnessbrightdark© Stephen E. Palmer, 2002© Stephen E. Palmer, 2002400 450 500 550 600 650RELATIVE ABSORBANCE (%)WAVELENGTH (nm.)10050440S530 560 nm.MLThree kinds of cones:Physiology of Color Vision• Why are M and L cones so close?• Are are there 3?More SpectrametamersColor Sensing in Camera (RGB)3-chip vs. 1-chip: quality vs. costWhy more green?http://www.cooldihttp://www.cooldictionary.com/words/Bayertionary.com/words/Bayer--filter.wikipediafilter.wikipediaWhy 3 colors?Slide by Steve SeitzPractical Color Sensing: Bayer GridEstimate RGBat ‘G’ cels from neighboring valueshttp://www.cooldictionary.com/words/Bayer-filter.wikipediaSlide by Steve SeitzRGB color spaceRGB cube• Easy for devices• But not perceptual• Where do the grays live?• Where is hue and saturation?Slide by Steve SeitzHSVHue, Saturation, Value (Intensity)• RGB cube on its vertexDecouples the three components (a bit)Use rgb2hsv() and hsv2rgb() in MatlabSlide by Steve SeitzProgramming Project #1• How to compare R,G,B channels?• No right answer• Sum of Squared Differences (SSD):• Normalized Correlation
View Full Document