DOC PREVIEW
cvpr09stereo

This preview shows page 1-2-3 out of 8 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Stereo Matching with Nonparametric Smoothness Priors in Feature SpaceBrandon M. SmithUniversity of Wisconsin–[email protected] ZhangUniversity of Wisconsin–[email protected] JinAdobe Systems [email protected] propose a novel formulation of stereo matching thatconsiders each pixel as a feature vector. Under this view,matching two or more images can be cast as matching pointclouds in feature space. We build a nonparametric depthsmoothness model in this space that correlates the image fea-tures and depth values. This model induces a sparse graphthat links pixels with similar features, thereby converting eachpoint cloud into a connected network. This network definesa neighborhood system that captures pixel grouping hierar-chies without resorting to image segmentation. We formulateglobal stereo matching over this neighborhood system and usegraph cuts to match pixels between two or more such net-works. We show that our stereo formulation is able to recoversurfaces with different orders of smoothness, such as thosewith high-curvature details and sharp discontinuities. Fur-thermore, compared to other single-frame stereo methods, ourmethod produces more temporally stable results from videosof dynamic scenes, even when applied to each frame indepen-dently.1. IntroductionStereo matching has been one of the core challenges incomputer vision for decades. Two categories of solutions havebeen proposed: local methods and global methods. Localmethods use a larger neighborhood (7 ×7 for example) aroundeach pixel; they have the flexibility to model parametric sur-faces (such as a quadratic patch) within the neighborhood, buthave difficulties in handling occlusion, which is a global prop-erty of the scene. Global methods use a smaller neighborhood(often a pair of pixels) to impose surface smoothness; theyare good at reasoning about occlusion but are often limitedto modeling piecewise planar scenes. In this work, we seekto combine the advantages of both approaches by designing aglobal stereo matching method that uses a large neighborhoodto define a depth smoothness prior.Using a large neighborhood gives us the opportunity tomodel complex local shapes; however, it is also challenging.Take the image in Figure 1 as an example. Different patcheshave different types of smoothness: flat planes, discontinuoussegments, and high-curvature folds. Assuming a single para-metric surface type would not be a robust solution in all cases,we argue that a nonparametric smoothness model should beused for a large neighborhood. Furthermore, it is well acceptedthat image features such as intensity edges [4] and color seg-Left Cloth3 image [17] Our depth map Woodford et al. [31]flat planediscontinuityhigh curvatureSmoothness types 2.01% bad pixels 6.33% bad pixelsFigure 1. Different image regions correspond to 3D surfaces withdifferent types of smoothness, as shown on the left. Such smooth-ness properties are often highly correlated with local image featuressuch as intensity gradients and shading. We propose a nonparametricsmoothness prior for global stereo matching that models the correla-tion between image features and depth values. Depth maps estimatedusing this model preserve both high-curvature surfaces and sharp dis-continuities at object boundaries, as shown in the middle. Our methodcompares favorably to an existing state-of-the-art method [31] thatuses a fixed (2nd) order smoothness prior, shown on the right. Ourmethod also has the advantage of being able to generate stable depthmaps for videos of dynamic scenes. Bad pixels (black) are thosewhose absolute depth errors are greater than one. Best viewed incolor.ments [25] provide important cues for depth estimation. Wetherefore hope that this nonparametric model will also be ableto represent the correlation between image features and depthvalues in a large neighborhood.Toward this end, we build a nonparametric depth smooth-ness prior model that correlates the image features and depthvalues. Our key idea is to consider each pixel as a feature vec-tor and view each image as a point cloud in this feature space.In general, the feature vector for each pixel can include itsposition, shading, texture, filter bank coefficients, etc., whichprovide cues that are often correlated with surface continu-ity, curvatures, etc. Under this view, matching two imagescan be cast as matching two point clouds in feature space. Inthis space, we introduce a nonparametric model that correlatesfeature vectors and depth values. For each image, this model1induces a dense graph with weighted edges that connect pix-els.Given a pair of such graphs that represent two images, wematch pixels between them using the graph cuts method [5].Our work makes the following three major contributions.• Nonparametric Smoothness in a Large NeighborhoodWe propose to use kernel density estimation in a largeneighborhood to correlate image features with depth val-ues. Using this correlation prior in a global matchingframework, our method is able to preserve both high-curvature shape details and sharp discontinuities at objectboundaries, as shown in Figure 1.• Sparse Graph Approximation Our large neighborhoodsmoothness prior yields an energy function defined overa dense graph that is challenging to minimize. We pro-pose novel techniques to simplify the energy function andapproximate the dense graph with a sparse one that con-tains its dominant edges. Applying graph cuts for stereomatching over such sparse graphs has the same computa-tional complexity as matching over regular image grids.• Stereo Matching with Implicit Segmentation Oursparse graph differs from the original image grid in thatit connects pixels with similar feature vectors. Such agraph encodes an image segmentation hierarchy. In prac-tice, matching pixels over such graphs preserves the dis-continuity boundaries well, but without requiring seg-mentation as a preprocessing step. Segmentation is of-ten temporally inconsistent when applied to videos; byavoiding it as a preprocessing step, our method recov-ers a more temporally stable depth estimate for dynamicscenes, even when applied to each frame independently.Our method is simple to implement: by replacing rectangularimage grids with our sparse graphs, one can use our idea inmost of the global stereo matching methods. We use it in clas-sic graph cuts stereo [5] in this paper. We show that our stereoformulation clearly improves upon existing methods on


cvpr09stereo

Download cvpr09stereo
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view cvpr09stereo and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view cvpr09stereo 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?