1CSE152, Spr 06 Intro Computer VisionStereo Vision IIntroduction to Computer VisionCSE 152Lecture 13CSE152, Spr 06 Intro Computer VisionAnnouncements• Midterm to be returned on Thursday• Discussion of midterm answers thenCSE152, Spr 06 Intro Computer VisionShape-from-X(i.e., Reconstruction)• Methods for estimating 3-D shape from image data. X can be one (or more) of many cues.– Stereo (two or more views, known viewpoints)– Motion (moving camera or object)– Shading– Changing lighting (Photometric Stereo)– Texture variation– Focus/blur– Prior knowledge/context– structured light/lasersCSE152, Spr 06 Intro Computer VisionBinocular Stereopsis: MarsGiven two images of a scene where relative locations of cameras are known, estimate depth of all common scene points.Two images of MarsCSE152, Spr 06 Intro Computer VisionAn Application: Mobile Robot NavigationThe Stanford Cart,H. Moravec, 1979.The INRIA Mobile Robot, 1990.Courtesy O. Faugeras and H. Moravec.CSE152, Spr 06 Intro Computer VisionCommercial Stereo HeadsTrinocular stereoTrinocular stereoBinocular stereoBinocular stereo2CSE152, Spr 06 Intro Computer VisionNeed for correspondenceTruco Fig. 7.5CSE152, Spr 06 Intro Computer VisionTriangulationNalwa Fig. 7.2CSE152, Spr 06 Intro Computer VisionStereo Vision Outline• Offline: Calibrate cameras & determine “epipolar geometry”• Online1. Acquire stereo images2. Rectify images to convenient epipolar geometry3. Establish correspondence 4. Estimate depthABCDCSE152, Spr 06 Intro Computer VisionBINOCULAR STEREO SYSTEMEstimating DepthZX(0,0) (d,0)Z=fXLXRDISPARITY(XL-XR)Z = (f/XL) XZ= (f/XR) (X-d)(f/XL) X = (f/XR) (X-d)X = (XLd) / (XL-XR)Z = d f(XL-XR)X = d XL(XL-XR)(Adapted from Hager)CSE152, Spr 06 Intro Computer VisionReconstruction: General 3-D case• Linear Method: find P such that• Non-Linear Method: find Q minimizingCSE152, Spr 06 Intro Computer VisionTwo Approaches1. Feature-Based– From each image, process “monocular” image to obtain cues (e.g., corners, lines).– Establish correspondence between2. Area-Based– Directly compare image regions between the two images.3CSE152, Spr 06 Intro Computer VisionHuman Stereopsis: Binocular FusionHow are the correspondences established?Julesz (1971): Is the mechanism for binocular fusiona monocular process or a binocular one??• There is anecdotal evidence for the latter (camouflage).• Random dot stereograms provide an objective answerCSE152, Spr 06 Intro Computer VisionRandom Dot StereogramsCSE152, Spr 06 Intro Computer VisionRandom Dot StereogramsCSE152, Spr 06 Intro Computer VisionEpipolar Constraint• Potential matches for p have to lie on the corresponding epipolar line l’.• Potential matches for p’ have to lie on the corresponding epipolar line l.CSE152, Spr 06 Intro Computer VisionEpipolar Geometry• Epipolar Plane• Epipoles• Epipolar Lines• BaselineCSE152, Spr 06 Intro Computer VisionFamily of epipolar PlanesFamily of planes π and lines l and l’Intersection in e and e’OO’4CSE152, Spr 06 Intro Computer VisionEpipolar Constraint: Calibrated CaseEssential Matrix(Longuet-Higgins, 1981)⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡−−−=×000][xyxzyztttttttwhereCSE152, Spr 06 Intro Computer VisionProperties of the Essential Matrix• E p’ is the epipolar line associated with p’.• ETp is the epipolar line associated with p.• E e’=0 and ETe=0.• E is singular.• E has two equal non-zero singular values(Huang and Faugeras, 1989).TTCSE152, Spr 06 Intro Computer VisionCalibrationDetermine intrinsic parameters and extrinsic relation of two camerasCSE152, Spr 06 Intro Computer VisionThe Eight-Point Algorithm (Longuet-Higgins, 1981)|F |=1.Minimize:under the constraint2Set F33to 1CSE152, Spr 06 Intro Computer VisionEpipolar geometry exampleCSE152, Spr 06 Intro Computer VisionExample: converging camerascourtesy of Andrew Zisserman5CSE152, Spr 06 Intro Computer VisionExample: motion parallel with image plane(simple for stereo → rectification)courtesy of Andrew ZissermanCSE152, Spr 06 Intro Computer VisionExample: forward motionee’courtesy of Andrew