1CSE152, Spr 06 Intro Computer VisionStereo Vision IIIntroduction to Computer VisionCSE 152Lecture 14CSE152, Spr 06 Intro Computer VisionAnnouncements• Midterm returned• Next HW assigned tomorrowCSE152, Spr 06 Intro Computer VisionStereo Vision Outline• Offline: Calibrate cameras & determine “epipolar geometry”• Online1. Acquire stereo images2. Rectify images to convenient epipolar geometry3. Establish correspondence 4. Estimate depthABCDCSE152, Spr 06 Intro Computer VisionBINOCULAR STEREO SYSTEMEstimating DepthZX(0,0) (d,0)Z=fXLXRDISPARITY(XL-XR)Z = (f/XL) XZ= (f/XR) (X-d)(f/XL) X = (f/XR) (X-d)X = (XLd) / (XL-XR)Z = d f(XL-XR)X = d XL(XL-XR)(Adapted from Hager)CSE152, Spr 06 Intro Computer VisionReconstruction: General 3-D case• Linear Method: find P such that• Non-Linear Method: find Q minimizingCSE152, Spr 06 Intro Computer VisionRandom Dot Stereograms2CSE152, Spr 06 Intro Computer VisionEpipolar Geometry• Epipolar Plane• Epipoles• Epipolar Lines• BaselineCSE152, Spr 06 Intro Computer VisionFamily of epipolar PlanesFamily of planes π and lines l and l’Intersection in e and e’OO’CSE152, Spr 06 Intro Computer VisionEpipolar Constraint: Calibrated CaseEssential Matrix(Longuet-Higgins, 1981)⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡−−−=×000][xyxzyztttttttwhereCSE152, Spr 06 Intro Computer VisionCalibrationDetermine intrinsic parameters and extrinsic relation of two camerasCSE152, Spr 06 Intro Computer VisionThe Eight-Point Algorithm (Longuet-Higgins, 1981)|F |=1.Minimize:under the constraint2Set F33to 1CSE152, Spr 06 Intro Computer VisionEpipolar geometry example3CSE152, Spr 06 Intro Computer VisionExample: converging camerascourtesy of Andrew ZissermanCSE152, Spr 06 Intro Computer VisionExample: forward motionee’courtesy of Andrew ZissermanCSE152, Spr 06 Intro Computer VisionRectificationGiven a pair of images, transform both images so that epipolar lines are scan lines.CSE152, Spr 06 Intro Computer VisionRectificationAll epipolar lines are parallel in the rectified image plane.CSE152, Spr 06 Intro Computer VisionImage pair rectificationsimplify stereo matching by warping the imagesApply projective transformation so that epipolar linescorrespond to horizontal scanlineseemap epipole e to (1,0,0)try to minimize image distortionHe001=⎥⎥⎦⎤⎢⎢⎣⎡Note that rectified images usually not rectangularCSE152, Spr 06 Intro Computer VisionRectificationGiven a pair of images, transform both images so that epipolar lines are scan lines.Input Images4CSE152, Spr 06 Intro Computer VisionRectificationGiven a pair of images, transform both images so that epipolar lines are scan lines.Rectified ImagesSee Section 7.3.7 for specific methodCSE152, Spr 06 Intro Computer VisionFeatures on same epipolar lineTruco Fig. 7.5CSE152, Spr 06 Intro Computer VisionMobi: Stereo-based navigationCSE152, Spr 06 Intro Computer VisionEpipolar correspondenceThis version is feature-based: detect edges in 1-D signal, and use dynanic progrmaming toe find correspondences that minimize an error function.CSE152, Spr 06 Intro Computer VisionSymbolic MapCSE152, Spr 06 Intro Computer VisionA challenge: Multiple InterpretationsEach feature on left epipolar line match oneand only one feature on right epipolar line.5CSE152, Spr 06 Intro Computer VisionMultiple InterpretationsEach feature on left epipolar line match oneand only one feature on right epipolar line.CSE152, Spr 06 Intro Computer VisionMultiple InterpretationsEach feature on left epipolar line match oneand only one feature on right epipolar line.CSE152, Spr 06 Intro Computer VisionMultiple InterpretationsEach feature on left epipolar line match oneand only one feature on right epipolar line.CSE152, Spr 06 Intro Computer VisionCorrespondence: Photometric constraint• Same world point has same intensity in both images (Constant Brightness Constraint)– Lambertian fronto-parallel– Issues:•Noise• Specularity• ForeshorteningCSE152, Spr 06 Intro Computer VisionUsing epipolar & constant Brightness constraints for stereo matchingFor each epipolar lineFor each pixel in the left image• compare with every pixel on same epipolar line in right image• pick pixel with minimum match cost• This will never work, so:Improvement: match