3D VisionStereo VisionA Stereo PairMore Images…Slide 5Slide 6Slide 7Slide 8Lecture OutlineStereo GeometryA Simple Stereo SystemDisparity EquationDisparity vs. BaselineDepth AccuracyStereo with Converging CamerasSlide 16Slide 17Slide 18Slide 19Slide 20Slide 21Parameters of a Stereo SystemEpipolar GeometrySlide 24Essential MatrixSlide 26Fundamental MatrixSlide 28Computing F: The Eight-point AlgorithmLocating the Epipoles from FStereo RectificationSlide 32Slide 33Slide 34Next3D Computer Visionand Video Computing3D Vision3D VisionTopic 7 of Part 2Stereo Vision (I)CSC I6716Spring 2004Zhigang Zhu, NAC 8/203Ahttp://www - cs.engr.ccny.cuny.edu /~zhu/VisionCourse-2004.html3D Computer Visionand Video ComputingStereo VisionStereo VisionProblemInfer 3D structure of a scene from two or more images taken from different viewpointsTwo primary Sub-problemsCorrespondence problem (stereo match) -> disparity mapSimilarity instead of identityOcclusion problem: some parts of the scene are visible only in one eyeReconstruction problem -> 3DWhat we need to know about the cameras’ parametersOften a stereo calibration problemLectures on Stereo VisionStereo Geometry – Epipolar Geometry (*) Correspondence Problem (*) – Two classes of approaches3D Reconstruction Problems – Three approaches3D Computer Visionand Video ComputingA Stereo PairA Stereo PairProblemsCorrespondence problem (stereo match) -> disparity mapReconstruction problem -> 3DCMU CIL Stereo Dataset : Castle sequencehttp://www-2.cs.cmu.edu/afs/cs/project/cil/ftp/html/cil-ster.html?3D?3D Computer Visionand Video ComputingMore Images…More Images…ProblemsCorrespondence problem (stereo match) -> disparity mapReconstruction problem -> 3D3D Computer Visionand Video ComputingMore Images…More Images…ProblemsCorrespondence problem (stereo match) -> disparity mapReconstruction problem -> 3D3D Computer Visionand Video ComputingMore Images…More Images…ProblemsCorrespondence problem (stereo match) -> disparity mapReconstruction problem -> 3D3D Computer Visionand Video ComputingMore Images…More Images…ProblemsCorrespondence problem (stereo match) -> disparity mapReconstruction problem -> 3D3D Computer Visionand Video ComputingMore Images…More Images…ProblemsCorrespondence problem (stereo match) -> disparity mapReconstruction problem -> 3D3D Computer Visionand Video ComputingLecture OutlineLecture OutlineA Simple Stereo Vision SystemDisparity Equation Depth ResolutionFixated Stereo SystemZero-disparity HoropterEpipolar GeometryEpipolar lines – Where to search correspondencesEpipolar Plane, Epipolar Lines and Epipoleshttp://www.ai.sri.com/~luong/research/Meta3DViewer/EpipolarGeo.htmlEssential Matrix and Fundamental MatrixComputing E & F by the Eight-Point AlgorithmComputing the EpipolesStereo Rectification3D Computer Visionand Video ComputingStereo GeometryStereo GeometryConverging Axes – Usual setup of human eyesDepth obtained by triangulationCorrespondence problem: pl and pr correspond to the left and right projections of P, respectively.Object pointCentralProjectionRaysVergence AngleplprP(X,Y,Z)3D Computer Visionand Video ComputingA Simple Stereo SystemA Simple Stereo SystemZw=0 LEFT CAMERALeft image:referenceRight image:targetRIGHT CAMERAElevation ZwdisparityDepth Zbaseline3D Computer Visionand Video ComputingDisparity EquationDisparity EquationP(X,Y,Z)pl(xl,yl)Optical Center Olf = focal lengthImage planeLEFT CAMERAB = BaselineDepthStereo system with parallel optical axesf = focal lengthOptical Center Orpr(xr,yr)Image planeRIGHT CAMERAdxBfDZ Disparity: dx = xr - xl3D Computer Visionand Video ComputingDisparity vs. BaselineDisparity vs. BaselineP(X,Y,Z)pl(xl,yl)Optical Center Olf = focal lengthImage planeLEFT CAMERAB = BaselineDepthf = focal lengthOptical Center Orpr(xr,yr)Image planeRIGHT CAMERAdxBfDZ Disparity dx = xr - xl Stereo system with parallel optical axes3D Computer Visionand Video ComputingDepth AccuracyDepth AccuracyGiven the same image localization errorAngle of cones in the figureDepth Accuracy (Depth Resolution) vs. BaselineDepth Error  1/B (Baseline length)PROS of Longer baseline, better depth estimationCONSsmaller common FOVCorrespondence harder due to occlusionDepth Accuracy (Depth Resolution) vs. DepthDisparity (>0)  1/ DepthDepth Error  Depth2Nearer the point, better the depth estimationAn Examplef = 16 x 512/8 pixels, B = 0.5 mDepth error vs. depthZ2Two viewpointsZ2>Z1Z1Z1OlOr)(Z 2dxfBZ)(ZZ dxfBZAbsolute errorRelative error3D Computer Visionand Video ComputingStereo with Converging CamerasStereo with Converging CamerasStereo with Parallel Axes Short baselinelarge common FOVlarge depth errorLong baselinesmall depth errorsmall common FOVMore occlusion problemsTwo optical axes intersect at the Fixation Pointconverging angle The common FOV IncreasesFOVLeft right3D Computer Visionand Video ComputingStereo with Converging CamerasStereo with Converging CamerasStereo with Parallel Axes Short baselinelarge common FOVlarge depth errorLong baselinesmall depth errorsmall common FOVMore occlusion problemsTwo optical axes intersect at the Fixation Pointconverging angle The common FOV IncreasesFOVLeft right3D Computer Visionand Video ComputingStereo with Converging CamerasStereo with Converging CamerasTwo optical axes intersect at the Fixation Pointconverging angle The common FOV IncreasesDisparity propertiesDisparity uses angle instead of distanceZero disparity at fixation pointand the Zero-disparity horopterDisparity increases with the distance of objects from the fixation points>0 : outside of the horopter<0 : inside the horopterDepth Accuracy vs. DepthDepth Error  Depth2Nearer the point, better the depth estimationFOVLeft rightFixation point3D Computer Visionand Video ComputingStereo with Converging CamerasStereo with Converging CamerasTwo optical axes intersect at the Fixation Pointconverging angle The common FOV IncreasesDisparity propertiesDisparity uses angle instead of distanceZero disparity at fixation pointand the Zero-disparity horopterDisparity increases with the distance of objects from the fixation points>0 :