Lecture 10: StereoTuesday, Oct 2Grad student extension ideas for problem set 2• Implement textons approach for texture recognition [Leung & Malik]– Possible data sources: Vistex, Curetdatabases• Build a shape-based object detector using the generalized Hough transform• Clustering approach to video shot boundary detection• Build a deformable contour trackerExam• Next Tuesday, Oct 9, in class• Bring one handwritten 8.5 x 11”, one-sided sheet with any notes• Closed book/laptop/calculatorReview all material covered so far• Image formation– Perspective, orthographic projection properties, equations, effects– Pinhole cameras– Thin lens– Field of view, depth of field• Color–BRDF– Spectral power distribution– Color mixing– Color matching – Color spaces– Human perception• Binary image analysis– Histograms and thresholding– Connected components– Morphological operators– Region properties and invariance– Distance transform, Chamfer distance• Filters– Application/effects of– Convolution properties– Noise models– Mean, median, Gaussian, derivative filters– Separability• Edges, pyramids, sampling– Image gradients– Effects of noise– Derivative of Gaussian, Laplacian filters– Canny edge detection– Corner detection– Sampling and aliasing– Pyramids – construction and applications•Texture– Analysis vs. synthesis– Representations• Grouping– Gestalt principles– Clustering: agglomerative, k-means, mean shift, graph-based– Graphs and affinity matrices•Fitting– Hough transform– Generalized Hough transform– Least squares– Incremental line fitting, k-means– Robust fitting: RANSAC, M-estimators– Deformable contours, energy functions• Stereo visionOutline• Brief review of deformable contours• Fundamentals of stereo vision • Epipolar geometryLast time: deformable contoursinitialintermediatefinala.k.a. active contours, snakesSnake energy functionThe total energy of the current snake defined asexintotalEEE+=Internal energy encourages smoothness or any particular shapeInternal energy incorporates priorknowledge about object boundary, which allows a boundary to be extracted even if some image data is missingExternal energy encourages curve onto image structures (e.g. image edges)We will want to iteratively minimize this energy for a good fit between the deformable contour and the target shape in the imageDiscrete energy terms• If the curve is represented by n pointsElasticity,Tension;Want to favor close pointsStiffnessCurvature;Want to favor smoothly shaped curve (not corners)10),( −==niyxiiiKν21iivdsdνν−≈+1111222)()(−+−++−=−−−≈iiiiiiidsdνννννννν∑−=−+++−+−=1021121|2|||niiiiiiinEνννβννα…Discrete energy terms• An external energy term for a (discrete) snake based on image edge2102|),(||),(|iiyniiixexyxGyxGE∑−=+−=Energy minimization• Many algorithms proposed to fit deformable contours– Greedy search– Gradient descent– Dynamic programming (for 2d snakes)Problems with snakes• Depends on number and spacing of control points• Snake may oversmooth the boundary• Not trivial to prevent curve self intersecting• Cannot follow topological changes of objectsProblems with snakes• May be sensitive to initialization, get stuck in local minimum• Accuracy (and computation time) depends on the convergence criteria used in the energy minimization techniqueProblems with snakes• External energy: snake does not really “see”object boundaries in the image unless it gets very close to it.image gradientsare large only directly on the boundaryI∇Depth unavailable in single viewsOptical centerP1P2P1’=P2’What cues can indicate 3d shape?Shading[Figure from Prados & Faugeras 2006]Focus/Defocus[Figure from H. Jin and P. Favaro, 2002]Texture[From A.M. Loh. The recovery of 3-D structure using visual texture patterns. PhD thesis]MotionFigures from L. Zhang http://www.brainconnection.com/teasers/?main=illusion/motion-shapeEstimating scene shape • Shape from X: Shading, Texture, Focus, Motion…•Stereo: – shape from motion between two views– infer 3d shape of scene from two (multiple) images from different viewpointsAccommodation and focusThe lens modifies the image focus by adjusting its focal length.Fixation, convergenceFixation, convergenceFrom Palmer, “Vision Science”, MIT PressHuman stereopsis: disparityDisparity occurs when eyes verge on one object; others appear at different visual anglesDisparity: d = r-l = D-F.d=0Human stereopsis: disparityAdapted from M. PollefeysRandom dot stereograms• Julesz 1960: Do we identify local brightness patterns before fusion (monocular process) or after (binocular)? • To test: pair of synthetic images obtained by randomly spraying black dots on white objectsRandom dot stereogramsForsyth & PonceRandom dot stereogramsRandom dot stereogramsFrom Palmer, “Vision Science”, MIT PressRandom dot stereograms• When viewed monocularly, they appear random; when viewed stereoscopically, see 3d structure.• Conclusion: human binocular fusion not directly associated with the physical retinas; must involve the central nervous system• Imaginary “cyclopean retina” that combines the left and right image stimuli as a single unitGenerating a random dot stereogramhttp://www.wellesley.edu/CS/LiDPC/OnParallaxis/Braunl.paper20.htmlAutostereogramsImages from magiceye.comExploit disparity as depth cue using single image(Single image random dot stereogram, Single image stereogram)Images from magiceye.comAutostereogramsImages from magiceye.comAutostereogramsStereo photography and stereo viewersInvented by Sir Charles Wheatstone, 1838Image courtesy of fisher-price.comTake two pictures of the same subject from two slightly different viewpoints and display so that each eye sees only one of the images.http://www.johnsonshawmuseum.orghttp://www.johnsonshawmuseum.orgPublic Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923http://www.well.com/~jimg/stereo/stereo_list.htmlStereo in machine vision systemsLeft : The Stanford cart sports a single camera moving in discrete increments along a straight line and providing multiple snapshots of outdoor scenesRight : The INRIA mobile robot uses three cameras to map its environmentForsyth & PonceStereo• Main issues– Geometry: what information is available, how do the camera views relate?– Correspondences: what feature in view 1 corresponds to feature
View Full Document
Unlocking...