Slide 1Last Time: Feature Detection and MatchingToday: TextureIncludes: more regular patternsIncludes: more random patternsScale: objects vs. textureWhy analyze texture?Texture-related tasksShape from textureSlide 10Texture-related tasksSlide 12Slide 13Color vs. texturePsychophysics of textureSlide 16Slide 17Slide 18JuleszTexture representationTexture representation: exampleTexture representation: exampleTexture representation: exampleTexture representation: exampleTexture representation: exampleTexture representation: exampleSlide 27Texture representation: exampleTexture representation: window scaleFilter banksd-dimensional featuresFilter banksInfluential paper:Bergen and Adelson, Nature 1988Malik and PeronaSlide 36Slide 37Slide 38Slide 39Slide 40Histogram matching algorithmSlide 42Heeger-Bergen texture synthesis algorithmBergen and HeegerBergen and Heeger resultsBergen and Heeger failuresDe Bonet (and Viola)DeBonetDeBonetSlide 50What we’ve learned from the previous texture synthesis methodsThe Goal of Texture AnalysisThe Goal of Texture SynthesisTexture synthesisSlide 55The ChallengeSlide 57Efros and LeungMarkov ChainsMarkov Chain Example: TextText synthesisText synthesisSynthesizing Computer Vision textSynthesized textMarkov Random FieldTexture SynthesisTexture synthesis: intuitionSynthesizing One PixelReally Synthesizing One PixelNeighborhood WindowVarying Window SizeSynthesis resultsSlide 73Slide 74Failure CasesHole FillingExtrapolationSlide 78Slide 79Slide 80Image Quilting [Efros & Freeman 2001]Slide 82Minimal error boundarySlide 84Slide 85Slide 86Slide 87Slide 88Slide 89Slide 90Texture TransferSlide 92Slide 93Slide 94(Manual) texture synthesis in the mediaSlide 96Slide 97Slide 98SummarySlide 100Next time: Image StitchingC280, Computer VisionProf. Trevor [email protected] 7: TextureLast Time: Feature Detection and Matching•Local features•Pyramids for invariant feature detection•Invariant descriptors•MatchingToday: TextureWhat defines a texture?Includes: more regular patternsIncludes: more random patternsScale: objects vs. textureOften the same thing in the world can occur as texture or an object, depending on the scale we are considering.Why analyze texture?Importance to perception:•Often indicative of a material’s properties•Can be important appearance cue, especially if shape is similar across objects•Aim to distinguish between shape, boundaries, and textureTechnically: •Representation-wise, we want a feature one step above “building blocks” of filters, edges.Texture-related tasks•Shape from texture–Estimate surface orientation or shape from image textureShape from texture•Use deformation of texture from point to point to estimate surface shapePics from A. Loh: http://www.csse.uwa.edu.au/~angie/phdpics1.htmlTexture-related tasks•Shape from texture–Estimate surface orientation or shape from image texture•Segmentation/classification from texture cues–Analyze, represent texture–Group image regions with consistent texture•Synthesis–Generate new texture patches/images given some exampleshttp://animals.nationalgeographic.com/Color vs. textureRecall: These looked very similar in terms of their color distributions (when our features were R-G-B)But how would their texture distributions compare?Psychophysics of texture•Some textures distinguishable with preattentive perception– without scrutiny, eye movements [Julesz 1975]Same or different?Julesz•Textons: analyze the texture in terms of statistical relationships between fundamental texture elements, called “textons”. •It generally required a human to look at the texture in order to decide what those fundamental units were...Texture representation•Textures are made up of repeated local patterns, so:–Find the patterns•Use filters that look like patterns (spots, bars, raw patches…)•Consider magnitude of response–Describe their statistics within each local window•Mean, standard deviation•Histogram•Histogram of “prototypical” feature occurrencesTexture representation: exampleoriginal imagederivative filter responses, squaredstatistics to summarize patterns in small windows mean d/dx value mean d/dy value Win. #1 4 10…Texture representation: exampleoriginal imagederivative filter responses, squaredstatistics to summarize patterns in small windows mean d/dx value mean d/dy value Win. #1 4 10Win.#2 18 7…Texture representation: exampleoriginal imagederivative filter responses, squaredstatistics to summarize patterns in small windows mean d/dx value mean d/dy value Win. #1 4 10Win.#2 18 7…Texture representation: exampleoriginal imagederivative filter responses, squaredstatistics to summarize patterns in small windows mean d/dx value mean d/dy value Win. #1 4 10Win.#2 18 7Win.#9 20 20……Texture representation: examplestatistics to summarize patterns in small windows mean d/dx value mean d/dy value Win. #1 4 10Win.#2 18 7Win.#9 20 20……Dimension 1 (mean d/dx value)Dimension 2 (mean d/dy value)Texture representation: examplestatistics to summarize patterns in small windows mean d/dx value mean d/dy value Win. #1 4 10Win.#2 18 7Win.#9 20 20……Dimension 1 (mean d/dx value)Dimension 2 (mean d/dy value)Windows with small gradient in both directionsWindows with primarily vertical edgesWindows with primarily horizontal edgesBothTexture representation: exampleoriginal imagederivative filter responses, squaredvisualization of the assignment to texture “types”Texture representation: examplestatistics to summarize patterns in small windows mean d/dx value mean d/dy value Win. #1 4 10Win.#2 18 7Win.#9 20 20……Dimension 1 (mean d/dx value)Dimension 2 (mean d/dy value)Far: dissimilar texturesClose: similar texturesTexture representation: window scale•We’re assuming we know the relevant window size for which we collect these statistics.Possible to perform scale selection by looking for window scale where texture description not changing.Filter banks•Our previous example used two filters, and resulted in a 2-dimensional feature vector to describe texture in a window.–x and y derivatives revealed something about local structure.•We can generalize to apply a collection of multiple (d) filters: a “filter bank”•Then our feature vectors will be