6 098 Digital and Computational Photography 6 882 Advanced Computational Photography HDR imaging and the Bilateral Filter Bill Freeman Fr do Durand MIT EECS Announcement Why Matting Matters Rick Szeliski Monday at 2pm in Kiva Patil Image matting e g blue screen matting has been a mainstay of Hollywood and the visual effects industry for decades but its relevance to computer vision is not yet fully appreciated In this talk I argue that the mixing of pixel color values at the boundaries of objects or even albedo changes if a fundamental process that must be correctly modeled to make meaningful signal level inferences about the visual world as well as to support high quality imaging transformations such as denoising and de blurring Starting with Ted Adelson et al s seminal work on layered motion models I review early stereo matching algorithms with transparency and matting with Polina Golland work on layered representations with matting with Simon Baker and Anandan through Larry Zitnick s 2 layer representation for 3D video I then present our recent work with Ce Liu et al on image de noising using a segmented description of the image and Eric Bennett s et al s work on multiimage de mosaicing again using a local two color model References Refs http www hdrsoft com resources dri html http www clarkvision com imagedetail dynamicrange2 http www debevec org HDRI2004 http www luminous landscape com tutorials hdr shtml http www anyhere com gward hdrenc http www debevec org IBL2001 NOTES 42 gward cic98 pdf http www openexr com http gl ict usc edu HDRShop http www dpreview com learn Glossary Digital Imaging Dynamic Range 01 htm http www normankoren com digital tonality html http www anyhere com http www cybergrain com tech hdr Contrast reduction Match limited contrast of the medium Preserve details 10 6 Real world High dynamic range 10 6 106 106 Picture Low contrast Histogram See http www luminous landscape com tutorials understandingseries understanding histograms shtml http www luminous landscape com tutorials expose right shtml Horizontal axis is pixel value Vertical axis is number of pixels Highlights Clipped pixels value 255 Pro and semi pro digital cameras allow you to make them blink Questions Multiple exposure photography Sequentially measure all segments of the range 10 6 Real world High dynamic range 10 6 106 106 Picture Low contrast Multiple exposure photography Sequentially measure all segments of the range 10 6 Real world High dynamic range 10 6 106 106 Picture Low contrast Multiple exposure photography Sequentially measure all segments of the range 10 6 Real world High dynamic range 10 6 106 106 Picture Low contrast Multiple exposure photography Sequentially measure all segments of the range 10 6 Real world High dynamic range 10 6 106 106 Picture Low contrast Multiple exposure photography Sequentially measure all segments of the range 10 6 Real world High dynamic range 10 6 106 106 Picture Low contrast Multiple exposure photography Sequentially measure all segments of the range 10 6 Real world High dynamic range 10 6 106 106 Picture Low contrast How do we vary exposure Options Shutter speed Aperture ISO Neutral density filter Slide inspired by Siggraph 2005 course on HDR Tradeoffs Shutter speed Range 30 sec to 1 4000sec 6 orders of magnitude Pros reliable linear Cons sometimes noise for long exposure Aperture Range f 1 4 to f 22 2 5 orders of magnitude Cons changes depth of field Useful when desperate ISO Range 100 to 1600 1 5 orders of magnitude Cons noise Useful when desperate Neutral density filter Range up to 4 densities 4 orders of magnitude can be stacked Cons not perfectly neutral color shift not very precise need to touch camera shake Pros works with strobe flash good complement when desperate Slide after Siggraph 2005 course on HDR Questions HDR image using multiple exposure Given N photos at different exposure Recover a HDR color for each pixel If we know the response curve Just look up the inverse of the response curve But how do we get the curve Pixel value scene value Calibrating the response curve Two basic solutions Vary scene luminance and see pixel values Assumes we control and know scene luminance Vary exposure and see pixel value for one scene luminance But note that we can usually not vary exposure more finely than by 1 3 stop Best of both Vary exposure Exploit the large number of pixels The Algorithm Image series 1 2 1 2 1 2 1 2 3 t 10 sec 3 t 1 sec 3 3 t 1 10 sec t 1 100 sec 1 2 3 t 1 1000 sec Pixel Value Z f Exposure Exposure Radiance t log Exposure log Radiance log t Slide adapted from Alyosha Efros who borrowed it from Paul Debevec t don t really correspond to pictures Oh well Response curve Exposure is unknown fit to find a smooth curve Assuming unit radiance After adjusting radiances to 3 2 obtain a smooth response curve Pixel value Pixel value for each pixel 1 log Exposure log Exposure Slide stolen from Alyosha Efros who stole it from Paul Debevec The Math Let g z be the discrete inverse response function For each pixel site i in each image j want log Radiancei log t j g Zij Solve the overdetermined linear system N P Zmax 2 logRadiancei log t j g Zij g z 2 i 1 j 1 z Zmin fitting term smoothness term Slide stolen from Alyosha Efros who stole it from Paul Debevec Matlab code function g lE gsolve Z B l w n 256 A zeros size Z 1 size Z 2 n 1 n size Z 1 b zeros size A 1 1 k 1 Include the data fitting equations for i 1 size Z 1 for j 1 size Z 2 wij w Z i j 1 A k Z i j 1 wij A k n i wij b k 1 wij B i j k k 1 end end A k 129 1 k k 1 Fix the curve by setting its middle value to 0 for i 1 n 2 Include the smoothness equations A k i l w i 1 A k i 1 2 l w i 1 A k i 2 l w i 1 k k 1 end x A b Solve the system using SVD g x 1 n lE x n 1 size x 1 Slide stolen from Alyosha Efros who stole it from Paul Debevec Result digital camera Kodak DCS460 1 30 to 30 sec Pixel value Recovered response curve log Exposure Slide stolen from Alyosha Efros who stole it from Paul Debevec Reconstructed radiance map Slide stolen from Alyosha Efros who stole it from Paul Debevec Result color film Kodak Gold ASA 100 PhotoCD Slide stolen from Alyosha Efros who stole it from Paul Debevec Recovered response curves Red Green Blue RGB Slide stolen from Alyosha Efros who stole it from Paul Debevec The Radiance map Slide stolen from Alyosha Efros who stole it from Paul Debevec The Radiance map Linearly scaled to display device Slide stolen from Alyosha Efros who stole it from Paul …
View Full Document
Unlocking...