Lecture 5: Edges, Corners, Sampling, PyramidsThursday, Sept 13With some slides by S. Seitz, D. Frolova, and D. SimakovNormalized cross correlation• Normalized correlation: normalize for image region brightness • Windowed correlation search: inexpensive way to find a fixed scale pattern• (Convolution = correlation if filter is symmetric)Best matchTemplateFilters and scenesFilters and scenes• Scenes have holistic qualities• Can represent scene categories with global texture•UseSteerable filters, windowed for some limited spatial information• Model likelihood of filter responses given scene category as mixture of Gaussians, (and incorporate some temporal info…)[Torralba, Murphy, Freeman, and Rubin, ICCV 2003][Torralba & Oliva, 2003]Steerable filters• Convolution linear -- synthesize a filter of arbitrary orientation as a linear combination of “basis filters”• Interpolated filter responses more efficient than explicit filter at arbitrary orientation[Freeman & Adelson, The Design and Use of Steerable Filters, PAMI 1991]Steerable filters==Freeman & Adelson, 1991Basis filters for derivative of Gaussian[Torralba, Murphy, Freeman, and Rubin, ICCV 2003]Probability of the scene given global features[Torralba, Murphy, Freeman, and Rubin, ICCV 2003]Contextual priors• Use scene recognition Æ predict objects present• For object(s) likely to be present, predict locations based on similarity to previous images with the same place and that object[Torralba, Murphy, Freeman, and Rubin, ICCV 2003]Scene categorySpecific place(black=right, red=wrong)Blue solid circle: recognition with temporal infoBlack hollow circle: instantaneous recognition using global feature onlyCross: true locationImage gradientThe gradient of an image: The gradient points in the direction of most rapid change in intensityThe gradient direction (orientation of edge normal) is given by:The edge strength is given by the gradient magnitudeSlide credit S. SeitzEffects of noiseConsider a single row or column of the image• Plotting intensity as a function of position gives a signalWhere is the edge?Where is the edge? Solution: smooth firstLook for peaks in Derivative theorem of convolutionThis saves us one operation:Laplacian of GaussianConsider Laplacian of GaussianoperatorWhere is the edge? Zero-crossings of bottom graph2D edge detection filters• is the Laplacian operator:Laplacian of GaussianGaussian derivative of GaussianThe Canny edge detectororiginal image (Lena)The Canny edge detectornorm of the gradientThe Canny edge detectorthresholdingNon-maximum suppressionCheck if pixel is local maximum along gradient direction, select single max across width of the edge• requires checking interpolated pixels p and rThe Canny edge detectorthinning(non-maximum suppression)Predicting the next edge pointAssume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s). (Forsyth & Ponce)Hysteresis ThresholdingReduces the probability of false contours and fragmented edgesGiven result of non-maximum suppression:For all edge points that remain,- locate next unvisited pixel where intensity > thigh- start from that point, follow chains along edge and add points where intensity < tlowEdge detection by subtractionoriginalEdge detection by subtractionsmoothed (5x5 Gaussian)Edge detection by subtractionsmoothed – original(scaled by 4, offset +128)Why doesthis work?Gaussian - image filterLaplacian of GaussianGaussiandelta functionCauses of edgesAdapted from C. RasmussenIf the goal is image understanding, what do we want from an edge detector?Learning good boundaries• Use ground truth (human-labeled) boundaries in natural images to learn good features• Supervised learning to optimize cue integration, filter scales, select feature typesWork by D. Martin and C. Fowlkes and D. Tal and J. Malik, Berkeley Segmentation Benchmark, 2001[D. Martin et al. PAMI 2004]Human-marked segment boundariesFeature profiles (oriented energy, brightness, color, and texture gradients) along the patch’s horizontal diameter[D. Martin et al. PAMI 2004]What features are responsible for perceived edges?What features are responsible for perceived edges?Learning good boundaries[D. Martin et al. PAMI 2004]OriginalBoundary detectionHuman-labeledBerkeley Segmentation Database, D. Martin and C. Fowlkes and D. Tal and J. Malik[D. Martin et al. PAMI 2004]Edge detection and corners• Partial derivative estimates in x and y fail to capture cornersWhy do we care about corners?Case study: panorama stitching[Brown, Szeliski, and Winder, CVPR 2005]How do we build panorama?• We need to match (align) images[Slide credit: Darya Frolova and Denis Simakov]Matching with Features• Detect feature points in both imagesMatching with Features• Detect feature points in both images• Find corresponding pairsMatching with Features• Detect feature points in both images• Find corresponding pairs• Use these pairs to align imagesMatching with Features• Problem 1:– Detect the same point independently in both imagesno chance to match!We need a repeatable detectorMatching with Features• (Problem 2:– For each point correctly recognize the corresponding one)?We need a reliable and distinctive descriptorMore on this aspect later!Corner detection as an interest operator• We should easily recognize the point by looking through a small window• Shifting a window in any direction should give a large change in intensity“flat” region:no change in all directions“edge”:no change along the edge direction“corner”:significant change in all directionsC.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988Corner detection as an interest operatorCorner Detection22,(, )xxyxyxy yIIIMwxyIII⎡⎤=⎢⎥⎢⎥⎣⎦∑M is a 2×2 matrix computed from image derivatives:Sum over image region – area we are checking for cornerGradient with respect to x, times gradient with respect to yCorner DetectionEigenvectors of M: encode edge directionsEigenvalues of M: encode edge strengthλ1, λ2 – eigenvalues of Mdirection of the slowest changedirection of the fastest changeλmaxλminCorner Detectionλ1λ2“Corner”λ1and λ2are large,λ1 ~ λ2;E increases in all directionsλ1and λ2are small;E is almost constant in all directions“Edge”λ1>> λ2“Edge”λ2>> λ1“Flat”regionClassification of image points
View Full Document
Unlocking...