Capturing Light… in man and machineImage FormationDigital cameraSensor ArraySampling and QuantizationInterlace vs. progressive scanProgressive scanInterlaceThe EyeThe RetinaRetina up-closeSlide 12Rod / Cone sensitivitySlide 14Electromagnetic SpectrumSlide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24More SpectraColor Sensing in Camera (RGB)Practical Color Sensing: Bayer GridRGB color spaceHSVWhite BalanceProgramming Assignment #1Image Pyramids (preview)Slide 33Problem: Dynamic RangeIs Camera a photometer?Long ExposureShort ExposureImage Acquisition PipelineVarying ExposureWhat does the eye sees?Eye is not a photometer!Cornsweet IllusionSine waveMetamersCapturing Light… in man and machine15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005Some figures from Steve Seitz, Steve Palmer, Paul Debevec, and Gonzalez et al.Image FormationDigital CameraThe EyeFilmDigital cameraA digital camera replaces film with a sensor array•Each cell in the array is light-sensitive diode that converts photons to electrons•Two common types–Charge Coupled Device (CCD) –CMOS•http://electronics.howstuffworks.com/digital-camera.htmSensor ArrayCMOS sensorSampling and QuantizationInterlace vs. progressive scanhttp://www.axis.com/products/video/camera/progressive_scan.htmProgressive scanhttp://www.axis.com/products/video/camera/progressive_scan.htmInterlacehttp://www.axis.com/products/video/camera/progressive_scan.htmThe EyeThe human eye is a camera!•Iris - colored annulus with radial muscles•Pupil - the hole (aperture) whose size is controlled by the iris•What’s the “film”?–photoreceptor cells (rods and cones) in the retinaThe RetinaCross-section of eyeGanglion cell layerBipolar cell layerReceptor layerPigmentedepitheliumGanglion axonsCross section of retinaRetina up-closeLight© Stephen E. Palmer, 2002Cones cone-shaped less sensitive operate in high light color visionTwo types of light-sensitive receptorsconerodRods rod-shaped highly sensitive operate at night gray-scale visionRod / Cone sensitivityThe famous sock-matching problem…© Stephen E. Palmer, 2002Distribution of Rods and Cones.0150,000100,00050,000020 40 60 8020406080Visual Angle (degrees from fovea)RodsConesConesRodsFoveaBlindSpot# Receptors/mm2Night Sky: why are there more stars off-center?Electromagnetic Spectrumhttp://www.yorku.ca/eye/photopik.htmHuman Luminance Sensitivity FunctionWhy do we see light of these wavelengths?© Stephen E. Palmer, 2002.0 1000 2000 3000EnergyWavelength (nm)400 700700 C2000 C5000 C10000 CVisibleRegion…because that’s where theSun radiates EM energyVisible LightThe Physics of LightAny patch of light can be completely describedphysically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm.400 500 600 700Wavelength (nm.)# Photons(per ms.)© Stephen E. Palmer, 2002The Physics of Light.# PhotonsD. Normal DaylightWavelength (nm.)B. Gallium Phosphide Crystal400 500 600 700# PhotonsWavelength (nm.)A. Ruby Laser400 500 600 700400 500 600 700# PhotonsC. Tungsten Lightbulb400 500 600 700# PhotonsSome examples of the spectra of light sources© Stephen E. Palmer, 2002The Physics of LightSome examples of the reflectance spectra of surfacesWavelength (nm)% Photons ReflectedRed400 700Yellow400 700Blue400 700Purple400 700© Stephen E. Palmer, 2002The Psychophysical CorrespondenceThere is no simple functional description for the perceivedcolor of all lights under all viewing conditions, but …...A helpful constraint: Consider only physical spectra with normal distributionsareaWavelength (nm.)# Photons400 700500 600meanvariance© Stephen E. Palmer, 2002The Psychophysical CorrespondenceMean Hueyellowgreenblue# PhotonsWavelength© Stephen E. Palmer, 2002The Psychophysical CorrespondenceVariance SaturationWavelengthhighmediumlowhi.med.low# Photons© Stephen E. Palmer, 2002The Psychophysical CorrespondenceArea Brightness# PhotonsWavelengthB. Area Lightnessbrightdark© Stephen E. Palmer, 2002© Stephen E. Palmer, 2002.400 450 500 550 600 650RELATIVE ABSORBANCE (%)WAVELENGTH (nm.)10050440S530 560 nm.M LThree kinds of cones:Physiology of Color Vision• Why are M and L cones so close?• Are are there 3?More SpectrametamersColor Sensing in Camera (RGB)3-chip vs. 1-chip: quality vs. costWhy more green?http://www.cooldihttp://www.cooldictionary.com/words/Bayer-filter.wikipediationary.com/words/Bayer-filter.wikipediaWhy 3 colors?Practical Color Sensing: Bayer GridEstimate RGBat ‘G’ cels from neighboring valueshttp://www.cooldictionary.com/words/Bayer-filter.wikipediaRGB color spaceRGB cube•Easy for devices•But not perceptual•Where do the grays live?•Where is hue and saturation?HSVHue, Saturation, Value (Intensity)•RGB cube on its vertexDecouples the three components (a bit)Use rgb2hsv() and hsv2rgb() in MatlabWhite BalanceWhite World / Gray World assumptionsProgramming Assignment #1•How to compare R,G,B channels?•No right answer•Sum of Squared Differences (SSD):•Normalized Correlation (NCC):Image Pyramids (preview)Known as a Gaussian Pyramid [Burt and Adelson, 1983]•In computer graphics, a mip map [Williams, 1983]•A precursor to wavelet transformImage Formationf(x,y) = reflectance(x,y) * illumination(x,y)Reflectance in [0,1], illumination in [0,inf]Problem: Dynamic Range150015001125,00025,000400,000400,0002,000,000,0002,000,000,000The real world isHigh dynamic rangepixel (312, 284) = 42pixel (312, 284) = 42ImageImage42 photos?42 photos?Is Camera a photometer?Long Exposure10-610610-6106Real worldPicture0 to 255High dynamic rangeShort Exposure10-610610-6106Real worldPicture0 to 255High dynamic rangescenesceneradianceradiance(W/sr/m )(W/sr/m )scenesceneradianceradiance(W/sr/m )(W/sr/m )sensorsensorirradianceirradiancesensorsensorirradianceirradiancesensorsensorexposureexposuresensorsensorexposureexposureLensLensLensLensShutterShutterShutterShutter2222ttanalogvoltagesanalogvoltagesdigitalvaluesdigitalvaluespixelvaluespixelvaluesCCDCCDADCADCRemappingRemappingImage Acquisition PipelineCamera is NOT a photometer!Varying ExposureWhat does the eye sees?The eye has a huge dynamic rangeDo we see a true radiance map?Eye is not a photometer!"Every light is a shade, compared to the higher lights, till you come to the sun; and every shade is a light, compared to the deeper shades, till you come to the night."
View Full Document
Unlocking...