Unformatted text preview:

1Computational photographyMIT 6.098, 6.882Bill Freeman, Fredo Durand• Finish digital forensics• Analyzing multiple images– Shapetime photography– Image stacks• Analysing and synthesizing motion sequences– Motion without movement– Motion magnificationApril 20, 2006Analyzing multiple imagesBill FreemanFredo DurandMIT Computational Photography, 6.882April 20, 2006Multiple-exposure images by MareyStrobe photograph by EdgertonOther photographs by Doc EdgertonWhat hardware was needed to make these photographs?Strobe light, capacitor, thyristor…2Computational photographySurely we can update those photographic techniques, adding the generality and flexibility of digital methods. Analyze and re-render the images.Computational photographyFredo and Bill describing computational photography:• Fredo: using computation to make better quality photographs—to enhance.• Bill: using computation to reveal things about the world that we otherwise couldn’t see—to reveal.How display a single-frame summary of multiple frames?Typical frame Average over 50 frames3Median filter over timeVector median filter (20x20 patchs)2x2 vector median 2x2 vector least medianShapetime photographyJoint work with Hao Zhang, U.C. Berkeley2002Video framesMultiple-exposureLayer-By-TimeShape-Time4Frame index of each displayed pixelResulting composite imageFrame index of each displayed pixelResulting composite imageWith edge-preserving regularization“how to sew”Shape-Time composite“inside-out”Input sequence5Z-cam, made by 3DVhttp://www.3dvsystems.com3DV camera operationhttp://www.3dvsystems.com3DV camera operationhttp://www.3dvsystems.comv3DV camera operationhttp://www.3dvsystems.comRGB image Z image6shapetime video imageZitnick et al, Siggraph 2004Show Michael Cohen slides, a selection from:http://research.microsoft.com/~cohen/FindingMagicInAnImageStack.pdf78Demonstrate MSR group shot program, downloadable from http://research.microsoft.com/~cohen/orhttp://research.microsoft.com/projects/GroupShot/Analyzing and synthesizing motionBill FreemanFredo DurandMIT Computational Photography, 6.882April 20, 2006Aperture Problem and Normal FlowAperture Problem and Normal FlowAperture Problem and Normal Flow9Aperture Problem and Normal FlowAperture Problem and Normal FlowAperture Problem and Normal FlowOptical flow constraint equation),,(),,( tyxItttvytuxI =+++δδδBrightness should stay constant as you track motion),,(),,( tyxItItIvtIutyxItyx=+++δδδ1storder Taylor series, valid for small tδ0=++tyxIvIuIConstraint equation “BCCE” - Brightness Change Constraint EquationAperture Problem and Normal Flow00=•∇=++UIIvIuItyxrThe gradient constraint:Defines a line in the (u,v) spaceuvIIIIut∇∇∇−=⊥Normal Flow:Combining Local Constraintsuv11tIUI −=•∇22tIUI −=•∇33tIUI −=•∇etc.10Lucas-Kanade(a good, generic motion analysis method):Integrate gradients over a patch()∑Ω∈++=yxtyxIvyxIuyxIvuE,2),(),(),(Setting derivative w.r.t. (u, v) equal to zero gives:Assume a single velocity, u, v, for all pixels within an image patch. Find the (u, v) that minimizes the BCCE squared residual over the patch:⎟⎟⎠⎞⎜⎜⎝⎛−=⎟⎟⎠⎞⎜⎜⎝⎛⎥⎥⎦⎤⎢⎢⎣⎡∑∑∑∑∑∑tytxyyxyxxIIIIvuIIIIII22Note similarity of LHS matrix to Harris corner detector. When full-rank (corner-like), specifies a unique (u, v).Motion without movementJoint work with Ted Adelson and David Heeger, MIT1991A linear combination of quadrature-phase filters can advance the local phaseConvolved with an image, the image data now modulates the local amplitude. People mis-attribute the phase advance to translation.(Steerable filters allow synthesizing motion in arbitrary directions.)Motion without movement videohttp://www.cs.yorku.ca/~kosta/Motion_Without_Movement/Motion_Without_Movement.html11Konstantinos G. Derpanishttp://www.cs.yorku.ca/~kosta/Motion_Without_Movement/Motion_Without_Movement.htmlMotion Magnification(go to other


View Full Document

MIT 6 098 - Computational photography

Download Computational photography
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Computational photography and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Computational photography 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?