1High Dynamic Range Images15-463: Rendering and Image ProcessingAlexei EfrosThe Grandma ProblemThe Grandma Problem2Problem: Dynamic RangeProblem: Dynamic Range150015001125,00025,000400,000400,0002,000,000,0002,000,000,000The real world ishigh dynamic range.The real world ishigh dynamic range.pixel (312, 284) = 42pixel (312, 284) = 42ImageImage42 photos?42 photos?3Long ExposureLong Exposure10-610610-6106Real worldPicture0 to 255High dynamic rangeShort ExposureShort Exposure10-610610-6106Real worldPictureHigh dynamic range0 to 2554Camera CalibrationCamera Calibration• Geometric– How pixel coordinates relate to directions in the world• Photometric– How pixel values relate to radiance amounts in the world• Geometric– How pixel coordinates relate to directions in the world• Photometric– How pixel values relate to radiance amounts in the worldThe ImageAcquisition PipelineThe ImageAcquisition Pipelinesceneradiance(W/sr/m )sceneradiance(W/sr/m )∫∫sensorirradiancesensorirradiancesensorexposuresensorexposurelatentimagelatentimageLensLensShutterShutterFilmFilmElectronic CameraElectronic Camera22∆∆tt5filmdensityfilmdensityanalogvoltagesanalogvoltagesdigitalvaluesdigitalvaluespixelvaluespixelvaluesDevelopmentDevelopmentCCDCCDADCADCRemappingRemappingloglogExposure = Exposure = loglog(Radiance(Radiance* * ∆∆tt))Imaging system response functionImaging system response functionPixelPixelvaluevalue0255(CCD photon count)6Varying ExposureVarying ExposureCamera is not a photometer!Camera is not a photometer!• Limited dynamic range⇒ Perhaps use multiple exposures?• Unknown, nonlinear response⇒ Not possible to convert pixel values to radiance• Solution:– Recover response curve from multiple exposures, then reconstruct the radiance map• Limited dynamic range⇒ Perhaps use multiple exposures?• Unknown, nonlinear response⇒ Not possible to convert pixel values to radiance• Solution:– Recover response curve from multiple exposures, then reconstruct the radiance map7Recovering High Dynamic RangeRadiance Maps from PhotographsRecovering High Dynamic RangeRadiance Maps from PhotographsPaul DebevecJitendra MalikPaul DebevecJitendra MalikAugust 1997August 1997Computer Science DivisionUniversity of California at BerkeleyComputer Science DivisionUniversity of California at BerkeleyWays to vary exposureWays to vary exposure Shutter Speed (*) F/stop (aperture, iris) Neutral Density (ND) Filters Shutter Speed (*) F/stop (aperture, iris) Neutral Density (ND) Filters8Shutter SpeedShutter Speed• Ranges: Canon D30: 30 to 1/4,000 sec.• Sony VX2000: ¼ to 1/10,000 sec.• Pros:• Directly varies the exposure• Usually accurate and repeatable• Issues:• Noise in long exposures• Ranges: Canon D30: 30 to 1/4,000 sec.• Sony VX2000: ¼ to 1/10,000 sec.• Pros:• Directly varies the exposure• Usually accurate and repeatable• Issues:• Noise in long exposuresShutter SpeedShutter Speed• Note: shutter times usually obey a power series – each “stop” is a factor of 2• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec• Usually really is:• ¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec• Note: shutter times usually obey a power series – each “stop” is a factor of 2• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec• Usually really is:• ¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec9•3••33•1••11•2••22∆∆∆∆∆∆∆∆tt==11secsec•3••33•1••11•2••22∆∆∆∆∆∆∆∆tt==1/16 1/16 secsec•3••33• 1••11• 2••22∆∆∆∆∆∆∆∆tt==44secsec•3••33•1••11•2••22∆∆∆∆∆∆∆∆tt==1/64 1/64 secsecThe AlgorithmThe AlgorithmImage seriesImage seriesImage series•3••33•1••11•2••22∆∆∆∆∆∆∆∆tt==1/4 1/4 secsecExposure = Radiance × ∆tExposure = Radiance × ∆tlog Exposure = log Radiance + log ∆tlog Exposure = log Radiance + log ∆tPixel Value Z = f(Exposure)Pixel Value Z = f(Exposure)Response CurveResponse Curveln Exposureln ExposureAssumingunit radiancefor each pixelAssumingunit radiancefor each pixelAfteradjusting radiances to obtain a smooth response curveAfteradjusting radiances to obtain a smooth response curvePixel valuePixel value333111222ln Exposureln ExposurePixel valuePixel value10The MathThe Math• Let g(z) be the discrete inverse response function• For each pixel site i in each image j, want:• Solve the overdetermined linear system:• Let g(z) be the discrete inverse response function• For each pixel site i in each image j, want:• Solve the overdetermined linear system:fitting term smoothness termln Radiancei+ ln∆tj− g(Zij)[ ]2j=1P∑i =1N∑+λ′ ′ g (z)2z =ZminZmax∑lnRadiancei+ln∆tj=g(Zij)MatlabCodeMatlabCodefunction [g,lE]=gsolve(Z,B,l,w)n = 256;A = zeros(size(Z,1)*size(Z,2)+n+1,n+size(Z,1));b = zeros(size(A,1),1);k = 1; %% Include the data-fitting equationsfor i=1:size(Z,1)for j=1:size(Z,2)wij = w(Z(i,j)+1);A(k,Z(i,j)+1) = wij; A(k,n+i) = -wij; b(k,1) = wij * B(i,j);k=k+1;endendA(k,129) = 1; %% Fix the curve by setting its middle value to 0k=k+1;for i=1:n-2 %% Include the smoothness equationsA(k,i)=l*w(i+1); A(k,i+1)=-2*l*w(i+1); A(k,i+2)=l*w(i+1);k=k+1;endx = A\b; %% Solve the system using SVDg = x(1:n);lE = x(n+1:size(x,1));11Results: Digital CameraResults: Digital CameraRecovered response Recovered response curvecurvelog Exposurelog ExposurePixel valuePixel valueKodak DCS4601/30 to 30 secReconstructed radiance mapReconstructed radiance map12Results: Color FilmResults: Color Film• Kodak Gold ASA 100, PhotoCD• Kodak Gold ASA 100, PhotoCDRecovered Response CurvesRecovered Response CurvesRed GreenRGBBlue13The Radiance MapThe Radiance MapTheRadianceMapTheRadianceMapLinearly scaled toLinearly scaled todisplay devicedisplay device14Portable FloatMap (.pfm)Portable FloatMap (.pfm)• 12 bytes per pixel, 4 for each channelsign exponent mantissaPF768 5121<binary image data>Floating Point TIFF similarFloating Point TIFF similarText header similar to Jeff Poskanzer’s .ppmimage format:(145, 215, 87, 149) =(145, 215, 87) * 2^(149-128) =(1190000, 1760000, 713000)(145, 215, 87, 149) =(145, 215, 87) * 2^(149-128) =(1190000, 1760000, 713000)Red Green Blue ExponentRed Green Blue Exponent32 bits / pixel32 bits / pixel(145, 215, 87,
View Full Document
Unlocking...