High Dynamic Range ImagesThe Grandma ProblemProblem: Dynamic RangeSlide 4Long ExposureShort ExposureCamera CalibrationThe Image Acquisition PipelineSlide 9Slide 10Varying ExposureCamera is not a photometer!Recovering High Dynamic Range Radiance Maps from PhotographsWays to vary exposureShutter SpeedSlide 16The AlgorithmSlide 18The MathMatlab CodeResults: Digital CameraSlide 22Results: Color FilmRecovered Response CurvesThe Radiance MapSlide 26Portable FloatMap (.pfm)Radiance Format (.pic, .hdr)ILM’s OpenEXR (.exr)Now What?Tone MappingSimple Global OperatorGlobal Operator (Reinhart et al)Global Operator ResultsSlide 35What do we see?High Dynamic Range Images15-463: Computational PhotographyAlexei Efros, CMU, Fall 2006…with a lot of slides stolen from Paul Debevec and Yuanzhen Li, © Alyosha EfrosThe Grandma ProblemThe Grandma ProblemProblem: Dynamic RangeProblem: Dynamic Range150015001125,00025,000400,000400,0002,000,000,0002,000,000,000The real world ishigh dynamic range.The real world ishigh dynamic range.pixel (312, 284) = 42pixel (312, 284) = 42ImageImage42 photos?42 photos?Long ExposureLong Exposure10-610610-6106Real worldPicture0 to 255High dynamic rangeShort ExposureShort Exposure10-610610-6106Real worldPictureHigh dynamic range0 to 255Camera CalibrationCamera Calibration•Geometric–How pixel coordinates relate to directions in the world•Photometric–How pixel values relate to radiance amounts in the world•Geometric–How pixel coordinates relate to directions in the world•Photometric–How pixel values relate to radiance amounts in the worldThe ImageAcquisition PipelineThe ImageAcquisition Pipelinesceneradiance(W/sr/m )sceneradiance(W/sr/m )sensorirradiancesensorirradiancesensorexposuresensorexposurelatentimagelatentimageLensLensShutterShutterFilmFilmElectronic CameraElectronic Camera22ttfilmdensityfilmdensityanalogvoltagesanalogvoltagesdigitalvaluesdigitalvaluespixelvaluespixelvaluesDevelopmentDevelopmentCCDCCDADCADCRemappingRemappingloglog Exposure = Exposure = loglog (Radiance (Radiance * * t)t)Imaging system response functionImaging system response functionPixelPixelvaluevalue0255(CCD photon count)Varying ExposureVarying ExposureCamera is not a photometer!Camera is not a photometer!•Limited dynamic range Perhaps use multiple exposures?•Unknown, nonlinear response Not possible to convert pixel values to radiance•Solution:–Recover response curve from multiple exposures, then reconstruct the radiance map•Limited dynamic range Perhaps use multiple exposures?•Unknown, nonlinear response Not possible to convert pixel values to radiance•Solution:–Recover response curve from multiple exposures, then reconstruct the radiance mapRecovering High Dynamic RangeRadiance Maps from PhotographsRecovering High Dynamic RangeRadiance Maps from PhotographsPaul DebevecJitendra MalikPaul DebevecJitendra MalikAugust 1997August 1997Computer Science DivisionUniversity of California at BerkeleyComputer Science DivisionUniversity of California at BerkeleyWays to vary exposureWays to vary exposureShutter Speed (*)F/stop (aperture, iris)Neutral Density (ND) FiltersShutter Speed (*)F/stop (aperture, iris)Neutral Density (ND) FiltersShutter SpeedShutter Speed•Ranges: Canon D30: 30 to 1/4,000 sec.•Sony VX2000: ¼ to 1/10,000 sec.•Pros:•Directly varies the exposure•Usually accurate and repeatable•Issues:•Noise in long exposures•Ranges: Canon D30: 30 to 1/4,000 sec.•Sony VX2000: ¼ to 1/10,000 sec.•Pros:•Directly varies the exposure•Usually accurate and repeatable•Issues:•Noise in long exposuresShutter SpeedShutter Speed•Note: shutter times usually obey a power series – each “stop” is a factor of 2•¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec•Usually really is:•¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec•Note: shutter times usually obey a power series – each “stop” is a factor of 2•¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec•Usually really is:•¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec• • 33• • 33• • 11• • 11• • 22• • 22t =t =11 sec sec• • 33• • 33• • 11• • 11• • 22• • 22t =t =1/16 1/16 secsec• • 33• • 33• • 11• • 11• • 22• • 22t =t =44 sec sec• • 33• • 33• • 11• • 11• • 22• • 22t =t =1/64 1/64 secsecThe AlgorithmThe AlgorithmImage seriesImage seriesImage seriesImage series• • 33• • 33• • 11• • 11• • 22• • 22t =t =1/4 1/4 secsecExposure = Radiance tExposure = Radiance tlog Exposure = log Radiance log tlog Exposure = log Radiance log tPixel Value Z = f(Exposure)Pixel Value Z = f(Exposure)Response CurveResponse Curveln Exposureln ExposureAssuming unit radiancefor each pixelAssuming unit radiancefor each pixelAfter adjusting radiances to obtain a smooth response curveAfter adjusting radiances to obtain a smooth response curvePixel valuePixel value333311112222ln Exposureln ExposurePixel valuePixel valueThe MathThe Math•Let g(z) be the discrete inverse response function•For each pixel site i in each image j, want:•Solve the overdetermined linear system:•Let g(z) be the discrete inverse response function•For each pixel site i in each image j, want:•Solve the overdetermined linear system:fitting term smoothness termln Radiancei ln tj g(Zij) 2j 1Pi 1Ng (z)2z ZminZmaxln Radianceiln tjg(Zij)MatlabCodeMatlabCodefunction [g,lE]=gsolve(Z,B,l,w)n = 256;A = zeros(size(Z,1)*size(Z,2)+n+1,n+size(Z,1));b = zeros(size(A,1),1);k = 1; %% Include the data-fitting equationsfor i=1:size(Z,1) for j=1:size(Z,2) wij = w(Z(i,j)+1); A(k,Z(i,j)+1) = wij; A(k,n+i) = -wij; b(k,1) = wij * B(i,j); k=k+1; endendA(k,129) = 1; %% Fix the curve by setting its middle value to 0k=k+1;for i=1:n-2 %% Include the smoothness equations A(k,i)=l*w(i+1); A(k,i+1)=-2*l*w(i+1); A(k,i+2)=l*w(i+1); k=k+1;endx = A\b; %% Solve the system using SVDg = x(1:n);lE = x(n+1:size(x,1));Results: Digital CameraResults: Digital CameraRecovered response Recovered response curvecurvelog Exposurelog ExposurePixel valuePixel valueKodak DCS4601/30 to 30 secReconstructed radiance mapReconstructed radiance mapResults: Color FilmResults: Color Film•Kodak
View Full Document