Unformatted text preview:

1CSE152, Spr 06 Intro Computer VisionStereo Vision IIIIntroduction to Computer VisionCSE 152Lecture 15CSE152, Spr 06 Intro Computer VisionAnnouncements• Next HW assigned, due May 30CSE152, Spr 06 Intro Computer VisionRandom Dot StereogramsCSE152, Spr 06 Intro Computer VisionEpipolar geometry exampleCSE152, Spr 06 Intro Computer VisionRectificationGiven a pair of images, transform both images so that epipolar lines are scan lines.Input ImagesCSE152, Spr 06 Intro Computer VisionRectificationGiven a pair of images, transform both images so that epipolar lines are scan lines.Rectified ImagesSee Section 7.3.7 for specific method2CSE152, Spr 06 Intro Computer VisionA challenge: Multiple InterpretationsEach feature on left epipolar line match oneand only one feature on right epipolar line.CSE152, Spr 06 Intro Computer VisionUsing epipolar & constant Brightness constraints for stereo matchingFor each epipolar lineFor each pixel in the left image• compare with every pixel on same epipolar line in right image• pick pixel with minimum match cost• This will never work, so:Improvement: match windows(Seitz)CSE152, Spr 06 Intro Computer VisionFinding CorrespondencesW(pl) W(pr)CSE152, Spr 06 Intro Computer VisionSlide Window to different disparities to find best matchBest Match amounts to minimizing (or maximizing some) Match MetricCSE152, Spr 06 Intro Computer VisionComparing Windows:==??ffggMostMostpopularpopular(Camps)For each window, match to closest window on epipolar line in other image.CSE152, Spr 06 Intro Computer VisionCorrespondence Search Algorithm For i = 1:nrowsfor j=1:ncolsbest(i,j) = -1for k = mindisparity:maxdisparityc = Match_Metric(I1(i,j),I2(i,j+k),winsize)if (c > best(i,j))best(i,j) = cdisparities(i,j) = kendendendendO(nrows * ncols * disparities * winx * winy)I1I2uvdI1I2uvd3CSE152, Spr 06 Intro Computer VisionMatch Metric Summary()()()()()()()()∑∑∑−+⋅−−+⋅−vuvuvuIvduIIvuIIvduIIvuI,,222211,2211,,,,() ( )()∑+−vuvduIvuI,221,,()()()()()()()()∑∑∑⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎝⎛−+−+−−−vuvuvuIvduIIvduIIvuIIvuI,2,22222,21111,,,,() ( )∑+−vuvduIvuI,21,,() ( )()∑+−vuvduIvuI,'2'1,,() ( ) ()∑<=nmkkkvuInmIvuI,',,,()( )()∑+vuvduIvuIHAMMING,'2'1,,,() ( ) ()()vuInmIBITSTRINGvuIkknmk,,,,'<=MATCH METRIC DEFINITIONNormalized Cross-Correlation (NCC)Sum of Squared Differences (SSD)Normalized SSDSum of Absolute Differences (SAD)Zero Mean SADRankCensusThese two are actually the same() ( )∑−+−−vuIvduIIvuI,_22_11),(),(CSE152, Spr 06 Intro Computer VisionStereo resultsGround truthScene– Data from University of Tsukuba(Seitz)CSE152, Spr 06 Intro Computer VisionResults with window correlationWindow-based matching(best window size)Ground truth(Seitz)CSE152, Spr 06 Intro Computer VisionResults with better methodState of the art methodBoykov et al., Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September 1999.Ground truth(Seitz)CSE152, Spr 06 Intro Computer VisionSome Issues• Ambiguity• Window size• Window shape• Lighting• Half occluded regionsCSE152, Spr 06 Intro Computer VisionAmbiguity4CSE152, Spr 06 Intro Computer VisionWindow sizeW = 3 W = 20Better results with adaptive window• T. Kanade and M. Okutomi, A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991. • D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998 • Effect of window size(Seitz)CSE152, Spr 06 Intro Computer VisionWindow Shape and ForshorteningCSE152, Spr 06 Intro Computer VisionWlWrWpU1U2Window Shape: Fronto-parallel ConfigurationCSE152, Spr 06 Intro Computer VisionLighting Conditions (Photometric Variations)W(Pl) W(Pr)CSE152, Spr 06 Intro Computer VisionProblem of OcclusionCSE152, Spr 06 Intro Computer VisionStereo ConstraintsCONSTRAINTBRIEF DESCRIPTION1-D Epipolar SearchArbitrary images of the same scene may be rectified based on epipolar geometry such that stereo matches lie along one-dimensional scanlines. This reduces the computational complexity and also reduces the likelihood of false matches.Monotonic OrderingPoints along an epipolar scanline appear in the same order in both stereo images, assuming that all objects in the scene are approximately the same distance from the cameras.Image Brightness ConstancyAssuming Lambertian surfaces, the brightness of corresponding points in stereo images are the same.Match UniquenessFor every point in one stereo image, there is at most one corresponding point in the other image.Disparity ContinuityDisparities vary smoothly (i.e. disparity gradient is small) over most of the image. This assumption is violated at object boundaries.Disparity LimitThe search space may be reduced significantly by limiting the disparity range, reducing both computational complexity and the likelihood of false matches.Fronto-Parallel SurfacesThe implicit assumption made by area-based matching is that objects have fronto-parallel surfaces (i.e. depth is constant within the region of local support). This assumption is violated by sloping and creased surfaces.Feature SimilarityCorresponding features must be similar (e.g. edges must have roughly the same length and orientation).Structural GroupingCorresponding feature groupings and their connectivity must be consistent.(From G. Hager)5CSE152, Spr 06 Intro Computer VisionStereo Matching using Dynamic Programming Reprinted from “Stereo by Intra- and Intet-Scanline Search,” by Y. Ohta and T. Kanade, IEEE Trans. on Pattern Analysis and MachineIntelligence, 7(2):139-154 (1985). © 1985 IEEE.(Ohta and Kanade, 1985)CSE152, Spr 06 Intro Computer VisionStereo matchingOptimal path(dynamic programming )Similarity measure(SSD or NCC)Constraints• epipolar• ordering• uniqueness• disparity limit• disparity gradient limitTrade-off• Matching cost (data)• Discontinuities (prior)(Cox et al. CVGIP’96; Koch’96; Falkenhagen´97;Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)(From Pollefeys)CSE152, Spr 06 Intro Computer VisionSome Challenges & Problems• Photometric issues:– specularities– strongly non-Lambertian BRDF’s• Surface structure– lack of texture– repeating texture within horopter bracket• Geometric ambiguities– as surfaces turn away, difficult to get accurate reconstruction (affine approximate can help)– at the occluding


View Full Document

UCSD CSE 152 - Stereo Vision III

Download Stereo Vision III
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Stereo Vision III and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Stereo Vision III 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?