High Dynamic Range ImagesThe Grandma ProblemProblem: Dynamic RangeLong ExposureShort ExposureCamera CalibrationThe Image Acquisition PipelineVarying ExposureCamera is not a photometer!Recovering High Dynamic Range Radiance Maps from PhotographsWays to vary exposureShutter SpeedShutter SpeedThe AlgorithmThe MathMatlab CodeResults: Digital CameraResults: Color FilmRecovered Response CurvesThe Radiance MapThe Radiance MapPortable FloatMap (.pfm)Radiance Format (.pic, .hdr)ILM’s OpenEXR (.exr)Now What?Tone MappingSimple Global OperatorGlobal Operator (Reinhart et al)Global Operator ResultsWhat do we see?What does the eye sees?MetamoresCompressing Dynamic RangeCompressing and Companding High Dynamic Range Images with Subband ArchitecturesDynamic Range ProblemRange CompressionMultiscale Subband DecompositionPoint Nonlinearity on SubbandsSmooth Gain ControlSmooth Gain Control Reduces DistortionSmooth Gain Control on SubbandsHigh Dynamic Range Images15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005…with a lot of slides stolen from Paul Debevec and Yuanzhen Li, © Alyosha EfrosThe Grandma ProblemThe Grandma ProblemProblem: Dynamic RangeProblem: Dynamic Range150015001125,00025,000400,000400,0002,000,000,0002,000,000,000The real world ishigh dynamic range.The real world ishigh dynamic range.pixel (312, 284) = 42pixel (312, 284) = 42ImageImage42 photos?42 photos?Long ExposureLong Exposure10-610610-6106Real worldPicture0 to 255High dynamic rangeShort ExposureShort Exposure10-610610-6106Real worldPictureHigh dynamic range0 to 255Camera CalibrationCamera Calibration• Geometric– How pixel coordinates relate to directions in the world• Photometric– How pixel values relate to radiance amounts in the world• Geometric– How pixel coordinates relate to directions in the world• Photometric– How pixel values relate to radiance amounts in the worldThe ImageAcquisition PipelineThe ImageAcquisition Pipelinesceneradiance(W/sr/m )sceneradiance(W/sr/m )∫∫sensorirradiancesensorirradiancesensorexposuresensorexposurelatentimagelatentimageLensLensShutterShutterFilmFilmElectronic CameraElectronic Camera22ΔΔttfilmdensityfilmdensityanalogvoltagesanalogvoltagesdigitalvaluesdigitalvaluespixelvaluespixelvaluesDevelopmentDevelopmentCCDCCDADCADCRemappingRemappingloglogExposure = Exposure = loglog(Radiance(Radiance* * ΔΔtt))Imaging system response functionImaging system response functionPixelPixelvaluevalue0255(CCD photon count)Varying ExposureVarying ExposureCamera is not a photometer!Camera is not a photometer!• Limited dynamic range⇒ Perhaps use multiple exposures?• Unknown, nonlinear response⇒ Not possible to convert pixel values to radiance• Solution:– Recover response curve from multiple exposures, then reconstruct the radiance map• Limited dynamic range⇒ Perhaps use multiple exposures?• Unknown, nonlinear response⇒ Not possible to convert pixel values to radiance• Solution:– Recover response curve from multiple exposures, then reconstruct the radiance mapRecovering High Dynamic RangeRadiance Maps from PhotographsRecovering High Dynamic RangeRadiance Maps from PhotographsPaul DebevecJitendra MalikPaul DebevecJitendra MalikAugust 1997August 1997Computer Science DivisionUniversity of California at BerkeleyComputer Science DivisionUniversity of California at BerkeleyWays to vary exposureWays to vary exposure Shutter Speed (*) F/stop (aperture, iris) Neutral Density (ND) Filters Shutter Speed (*) F/stop (aperture, iris) Neutral Density (ND) FiltersShutter SpeedShutter Speed• Ranges: Canon D30: 30 to 1/4,000 sec.• Sony VX2000: ¼ to 1/10,000 sec.• Pros:• Directly varies the exposure• Usually accurate and repeatable• Issues:• Noise in long exposures• Ranges: Canon D30: 30 to 1/4,000 sec.• Sony VX2000: ¼ to 1/10,000 sec.• Pros:• Directly varies the exposure• Usually accurate and repeatable• Issues:• Noise in long exposuresShutter SpeedShutter Speed• Note: shutter times usually obey a power series – each “stop” is a factor of 2• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec• Usually really is:• ¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec• Note: shutter times usually obey a power series – each “stop” is a factor of 2• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec• Usually really is:• ¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec•3••33•1••11•2••22ΔΔtt==11secsec•3••33•1••11•2••22ΔΔtt==1/16 1/16 secsec•3••33•1••11•2••22ΔΔtt==44secsec•3••33•1••11•2••22ΔΔtt==1/64 1/64 secsecThe AlgorithmThe AlgorithmImage seriesImage seriesImage series•3••33•1••11•2••22ΔΔtt==1/4 1/4 secsecExposure = Radiance ×ΔtExposure = Radiance × Δtlog Exposure = log Radiance + log Δtlog Exposure = log Radiance + log ΔtPixel Value Z = f(Exposure)Pixel Value Z = f(Exposure)Response CurveResponse Curveln Exposureln ExposureAssuming unit radiancefor each pixelAssuming unit radiancefor each pixelPixel valuePixel valueAfter adjusting radiances to obtain a smooth response After adjusting radiances to obtain a smooth response curvecurve333111222ln Exposureln ExposurePixel valuePixel valueThe MathThe Math• Let g(z) be the discrete inverse response function• For each pixel site i in each image j, want:• Solve the overdetermined linear system:• Let g(z) be the discrete inverse response function• For each pixel site i in each image j, want:• Solve the overdetermined linear system:fitting term smoothness termlnRadiancei+lnΔtj−g(Zij)[]2j=1P∑i=1N∑+λ′ ′ g (z)2z=ZminZmax∑lnRadiancei+lnΔtj= g(Zij)MatlabCodeMatlabCodefunction [g,lE]=gsolve(Z,B,l,w)n = 256;A = zeros(size(Z,1)*size(Z,2)+n+1,n+size(Z,1));b = zeros(size(A,1),1);k = 1; %% Include the data-fitting equationsfor i=1:size(Z,1)for j=1:size(Z,2)wij = w(Z(i,j)+1);A(k,Z(i,j)+1) = wij; A(k,n+i) = -wij; b(k,1) = wij * B(i,j);k=k+1;endendA(k,129) = 1; %% Fix the curve by setting its middle value tok=k+1;for i=1:n-2 %% Include the smoothness equationsA(k,i)=l*w(i+1); A(k,i+1)=-2*l*w(i+1); A(k,i+2)=l*w(i+1);k=k+1;endx = A\b; %% Solve the system using SVDg = x(1:n);lE = x(n+1:size(x,1));Results: Digital CameraResults: Digital CameraRecovered response Recovered response curvecurvelog Exposurelog ExposurePixel valuePixel valueKodak DCS4601/30 to 30
View Full Document