Computational photography MIT 6 098 6 882 Bill Freeman Fredo Durand Finish digital forensics Analyzing multiple images Shapetime photography Image stacks Analysing and synthesizing motion sequences Motion without movement Motion magnification April 20 2006 Analyzing multiple images Bill Freeman Fredo Durand MIT Computational Photography 6 882 April 20 2006 Multiple exposure images by Marey Strobe photograph by Edgerton Other photographs by Doc Edgerton What hardware was needed to make these photographs Strobe light capacitor thyristor Computational photography Surely we can update those photographic techniques adding the generality and flexibility of digital methods Analyze and rerender the images Computational photography Fredo and Bill describing computational photography Fredo using computation to make better quality photographs to enhance Bill using computation to reveal things about the world that we otherwise couldn t see to reveal How display a single frame summary of multiple frames Typical frame Average over 50 frames Median filter over time Vector median filter 20x20 patchs 2x2 vector median median 2x2 vector least Shapetime photography Joint work with Hao Zhang U C Berkeley 2002 Video frames LayerBy Time Multipleexposure ShapeTime Frame index of each displayed pixel Resulting composite image Frame index of each displayed pixel With edge preserving regularization Resulting composite image how to sew Input sequence Shape Time composite inside out Z cam made by 3DV http www 3dvsystems com 3DV camera operation http www 3dvsystems com 3DV camera operation http www 3dvsystems comv 3DV camera operation http www 3dvsystems com RGB image Z image shapetime video image Zitnick et al Siggraph 2004 Show Michael Cohen slides a selection from http research microsoft com cohen FindingMa gicInAnImageStack pdf Demonstrate MSR group shot program downloadable from http research microsoft com cohen or http research microsoft com projects GroupShot Analyzing and synthesizing motion Bill Freeman Fredo Durand MIT Computational Photography 6 882 April 20 2006 Aperture Problem and Normal Flow Aperture Problem and Normal Flow Aperture Problem and Normal Flow Aperture Problem and Normal Flow Aperture Problem and Normal Flow Aperture Problem and Normal Flow Optical flow constraint equation Brightness should stay constant as you track motion I x u t y v t t t I x y t 1st order Taylor series valid for small t I x y t u tI x v tI y tI t I x y t Constraint equation uI x vI y I t 0 BCCE Brightness Change Constraint Equation Aperture Problem and Normal Flow The gradient constraint I x u I y v I t 0 I U 0 Defines a line in the u v space v Normal Flow I t I u I I u Combining Local Constraints v I 1 U I t1 I 2 U I t2 3 I U I u etc 3 t Lucas Kanade a good generic motion analysis method Integrate gradients over a patch Assume a single velocity u v for all pixels within an image patch Find the u v that minimizes the BCCE squared residual over the patch E u v I x y x x y u I y x y v I t 2 Setting derivative w r t u v equal to zero gives I x2 I x I y I I I x y 2 y u v I x It I I y t Note similarity of LHS matrix to Harris corner detector When full rank corner like specifies a unique u v Motion without movement Joint work with Ted Adelson and David Heeger MIT 1991 A linear combination of quadrature phase filters can advance the local phase Convolved with an image the image data now modulates the local amplitude People misattribute the phase advance to translation Steerable filters allow synthesizing motion in arbitrary directions Motion without movement video http www cs yorku ca kosta Motion Without Movement Motion Without Movement html http www cs yorku ca kosta Motion Without Movement Motion Without Movement html Konstantinos G Derpanis Motion Magnification go to other slides
View Full Document
Unlocking...