DOC PREVIEW
UCSD CSE 152 - Optical Flow and On to Recognition

This preview shows page 1-2 out of 6 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

1CSE152, Spr 04 Intro Computer VisionOptical Flow and On to RecognitionIntroduction to Computer VisionCSE 152Lecture 17CSE152, Spr 04 Intro Computer VisionAnnouncements• Assignment 4: Due Thursday• Assignment 5: To be posted on Thursday• Read: Trucco & Verri, Chapter 8 on Motion• Final Exam: Wed, 6/9/04, 11:30-2:30, WLH 2207 (here)CSE152, Spr 04 Intro Computer VisionVirtual Cinematography: Making 'The Matrix' SequelsGeorge BorshukovVFX Technology Supervisor, ESC EntertainmentFriday, June 4, 20041:00 p.m. to 2:30 p.m.[Pizza lunch will precede the event from noon to 1 p.m.]Main Auditorium, San Diego Supercomputer CenterThe presentation will cover the key technologies that had to be developed and deployed to create the synthetic human sequences in the Matrix sequels including Universal Capture - image-based facial animation, realistic human face rendering, and use of measured BRDF in film production. It will also feature a breakdown of The Superpunch shot (pictured above) from "The Matrix Revolutions" (the bullet time punch that Neo delivers to Agent Smith during the film's last face-off). This difficult, important, expensive, and challenging shot was entirely computer generated and showcased the technological developments of 3.5+ years at their best by showing a full-frame close-up of a known human actor. CSE152, Spr 04 Intro Computer VisionCSE152, Spr 04 Intro Computer VisionSimplest Idea for video processingImage Differences• Given image I(u,v,t) and I(u,v, t+δt), compute I(u,v, t+δt) - I(u,v,t).• This is partial derivative: • At object boundaries, is large and is cue for segmentation• Doesn’t tell which way stuff is movingtI∂∂tI∂∂CSE152, Spr 04 Intro Computer VisionOptical Flow:Where do pixels move to?2CSE152, Spr 04 Intro Computer VisionThe Motion FieldCSE152, Spr 04 Intro Computer VisionRigid Motion: General CasePosition and orientation of a rigid bodyRotation Matrix & Translation vectorRigid Motion:Velocity Vector: TAngular Velocity Vector: ω (or Ω)PpTp ×+=ω&p&CSE152, Spr 04 Intro Computer VisionMotion Field Equation• T: Components of 3-D linear motion• ω: Angular velocity vector• (u,v): Image point coordinates• Z: depth• f: focal lengthfvfuvufZfTvTvfufuvvfZfTuTuxyzxyzyxzyxz22ωωωωωωωω−−−+−=−++−−=&&CSE152, Spr 04 Intro Computer VisionfvfuvufZfTvTvfufuvvfZfTuTuxyzxyzyxzyxz22ωωωωωωωω−−−+−=−++−−=&&Pure Translationω = 0 CSE152, Spr 04 Intro Computer VisionPure TranslationRadialabout FOEParallel( FOE point at infinity) TZ= 0Motion parallel to image planeCSE152, Spr 04 Intro Computer VisionPure Rotation: T=0• Independent of TxTyTz• Independent of Z• Only function of (u,v), f and ωfvfuvufZfTvTvfufuvvfZfTuTuxyzxyzyxzyxz22ωωωωωωωω−−−+−=−++−−=&&3CSE152, Spr 04 Intro Computer VisionRotational MOTION FIELDThe “instantaneous” velocity of points in an imagePURE ROTATIONω = (0,0,1)TCSE152, Spr 04 Intro Computer VisionMotion Field Equation: Estimate DepthIf T, ω, and f are known or measured, then for each image point (u,v), one can solve for the depth Z given measured motion (du/dt, dv/dt) at (u,v).fvfuvufZfTvTvfufuvvfZfTuTuxyzxyzyxzyxz22ωωωωωωωω−−−+−=−++−−=&&CSE152, Spr 04 Intro Computer VisionEstimating the motion field from images1. Feature-based (Sect. 8.4.2 of Trucco & Verri)1. Detect Features (corners) in an image2. Search for the same features nearby (Feature tracking).2. Differential techniques (Sect. 8.4.1)CSE152, Spr 04 Intro Computer VisionDefinition of optical flowOPTICAL FLOW = apparent motion of brightness patternsIdeally, the optical flow is the projection of the three-dimensional velocity vectors on the imageCSE152, Spr 04 Intro Computer VisionMathematical formulation[Note change of notation: image coordinates now (x,y), not (u,v)]I (x,y,t) = brightness at image point (x,y) at time tOptical flow constraint equation :0 =∂∂+∂∂+∂∂=tIdtdyyIdtdxxIdtdIConsider scene (or camera) to be moving, so x(t), y(t)),,(),,( tyxItttdtdyytdtdxxI =+++δδδBrightness constancy assumption:0=dtdICSE152, Spr 04 Intro Computer VisionSolving for flow• We can measure • We want to solve for• One equation, two unknowns Optical flow constraint equation :0 =∂∂+∂∂+∂∂=tIdtdyyIdtdxxIdtdItIyIxI∂∂∂∂∂∂,, dtdydtdx,4CSE152, Spr 04 Intro Computer Vision, ∂∂=yIIx, ∂∂=yIIytIIt∂∂=Measurements, =dtdxudtdyv =Flow vectorCSE152, Spr 04 Intro Computer VisionWhat is the correspondence of P & P’Contour plots of image intensity in two imagesCSE152, Spr 04 Intro Computer VisionNormal FlowIllusion Works Barber Pole IllusionCSE152, Spr 04 Intro Computer VisionApparently an aperture problemÎCSE152, Spr 04 Intro Computer VisionTwo ways to get flow1. Think globally, and regularize over image2. Look over window and assume constant motion in the windowCSE152, Spr 04 Intro Computer Vision()()∑∑=++==++=02),(02),(tyxytyxxIvIuIIdvvudEIvIuIIduvudEΩxy5CSE152, Spr 04 Intro Computer Vision• M is zero matrix in constant intensity regionCSE152, Spr 04 Intro Computer VisionRecognitionCSE152, Spr 04 Intro Computer VisionRecognitionGiven a database of objects and an image determine what, if any of the objects are present in the image.CSE152, Spr 04 Intro Computer VisionRecognitionGiven a database ofobjects and an imagedetermine what, if anyof the objects are present in the image.CSE152, Spr 04 Intro Computer VisionRecognitionGiven a database ofobjects and an imagedetermine what, if anyof the objects are present in the image.CSE152, Spr 04 Intro Computer VisionObject Recognition: The ProblemGiven: A database D of “known” objects and an image I:1. Determine which (if any) objects in D appear in I2. Determine the pose (rotation and translation) of the objectSegmentation(where is it 2D)Recognition(what is it)Pose Est.(where is it 3D)WHAT AND WHERE!!!6CSE152, Spr 04 Intro Computer VisionFaceCamelBugHuman/FelixQuadrupedBarbaraSteeleProblem:Recognizing instancesRecognizing categoriesCSE152, Spr 04 Intro Computer VisionRecognition Challenges• Within-class variability– Different objects within the class have different shapes or different material characteristics– Deformable– Articulated– Compositional• Pose variability: – 2-D Image transformation (translation, rotation, scale)– 3-D Pose Variability (perspective, orthographic projection)• Lighting– Direction (multiple sources & type)–Color–Shadows•


View Full Document

UCSD CSE 152 - Optical Flow and On to Recognition

Download Optical Flow and On to Recognition
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Optical Flow and On to Recognition and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Optical Flow and On to Recognition 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?