Unformatted text preview:

Computational photography MIT 6.098, 6.882 Bill Freeman, Fredo DurandAnalyzing multiple imagesMultiple-exposure images by MareySlide 4Other photographs by Doc EdgertonWhat hardware was needed to make these photographs?Computational photographySlide 8Slide 9Slide 10Typical frameAverage over 50 framesMedian filter over timeVector median filter (20x20 patchs)Slide 15Shapetime photographySlide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Z-cam, made by 3DV3DV camera operationSlide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Slide 38Slide 39Slide 40Slide 41Slide 42Slide 43Slide 44Analyzing and synthesizing motionAperture Problem and Normal FlowSlide 47Slide 48Slide 49Slide 50Slide 51Optical flow constraint equationSlide 53Combining Local ConstraintsLucas-Kanade (a good, generic motion analysis method): Integrate gradients over a patchMotion without movementA linear combination of quadrature-phase filters can advance the local phaseConvolved with an image, the image data now modulates the local amplitude. People mis-attribute the phase advance to translation.Motion without movement videoSlide 60Slide 61Motion MagnificationComputational photography MIT 6.098, 6.882 Bill Freeman, Fredo Durand •Finish digital forensics•Analyzing multiple images–Shapetime photography–Image stacks•Analysing and synthesizing motion sequences–Motion without movement–Motion magnificationApril 20, 2006Analyzing multiple imagesBill FreemanFredo DurandMIT Computational Photography, 6.882April 20, 2006Multiple-exposure images by MareyStrobe photograph by EdgertonOther photographs by Doc EdgertonWhat hardware was needed to make these photographs?Strobe light, capacitor, thyristor…Computational photographySurely we can update those photographic techniques, adding the generality and flexibility of digital methods. Analyze and re-render the images.Computational photographyFredo and Bill describing computational photography:•Fredo: using computation to make better quality photographs—to enhance.•Bill: using computation to reveal things about the world that we otherwise couldn’t see—to reveal.How display a single-frame summary of multiple frames?Typical frameAverage over 50 framesMedian filter over timeVector median filter (20x20 patchs)2x2 vector median 2x2 vector least medianShapetime photographyJoint work with Hao Zhang, U.C. Berkeley2002Video framesMultiple-exposureLayer-By-TimeShape-TimeFrame index of each displayed pixelResulting composite imageFrame index of each displayed pixelResulting composite imageWith edge-preserving regularization“how to sew”Shape-Time composite“inside-out”Input sequenceZ-cam, made by 3DVhttp://www.3dvsystems.com3DV camera operationhttp://www.3dvsystems.com3DV camera operationhttp://www.3dvsystems.comv3DV camera operationhttp://www.3dvsystems.comRGB imageZ imageshapetime video imageZitnick et al, Siggraph 2004Show Michael Cohen slides, a selection from: http://research.microsoft.com/~cohen/FindingMagicInAnImageStack.pdfDemonstrate MSR group shot program, downloadable from http://research.microsoft.com/~cohen/orhttp://research.microsoft.com/projects/GroupShot/Analyzing and synthesizing motionBill FreemanFredo DurandMIT Computational Photography, 6.882April 20, 2006Aperture Problem and Normal FlowAperture Problem and Normal FlowAperture Problem and Normal FlowAperture Problem and Normal FlowAperture Problem and Normal FlowAperture Problem and Normal FlowOptical flow constraint equation),,(),,( tyxItttvytuxI Brightness should stay constant as you track motion),,(),,( tyxItItIvtIutyxItyx1st order Taylor series, valid for small t0tyxIvIuIConstraint equation “BCCE” - Brightness Change Constraint EquationAperture Problem and Normal Flow00-UIIvIuItyxThe gradient constraint:Defines a line in the (u,v) spaceuvIIIIutNormal Flow:Combining Local Constraintsuv11tIUI -22tIUI -33tIUI -etc.Lucas-Kanade (a good, generic motion analysis method): Integrate gradients over a patch yxtyxIvyxIuyxIvuE,2),(),(),(Setting derivative w.r.t. (u, v) equal to zero gives:Assume a single velocity, u, v, for all pixels within an image patch. Find the (u, v) that minimizes the BCCE squared residual over the patch:tytxyyxyxxIIIIvuIIIIII22Note similarity of LHS matrix to Harris corner detector. When full-rank (corner-like), specifies a unique (u, v).Motion without movementJoint work with Ted Adelson and David Heeger, MIT1991A linear combination of quadrature-phase filters can advance the local phaseConvolved with an image, the image data now modulates the local amplitude. People mis-attribute the phase advance to translation.(Steerable filters allow synthesizing motion in arbitrary directions.)Motion without movement videohttp://www.cs.yorku.ca/~kosta/Motion_Without_Movement/Motion_Without_Movement.htmlKonstantinos G. Derpanis http://www.cs.yorku.ca/~kosta/Motion_Without_Movement/Motion_Without_Movement.htmlMotion Magnification(go to other


View Full Document

MIT 6 098 - Computational photography

Download Computational photography
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Computational photography and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Computational photography 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?