Texture SynthesisTextureSlide 3The ChallengeEfros & Leung AlgorithmSome DetailsNeighborhood WindowVarying Window SizeSynthesis ResultsMore ResultsHomage to ShannonHole FillingExtrapolationSummaryImage Quilting [Efros & Freeman]Slide 16Minimal error boundaryOur PhilosophySlide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Political Texture Synthesis!MS Digital Image Pro (DEMO)Fill OrderSlide 32Exemplar-based Inpainting demoApplication: Texture TransferTexture TransferSlide 36Slide 37Slide 38Image AnalogiesSlide 40Slide 41Blur FilterEdge FilterArtistic FiltersColorizationTexture-by-numbersSuper-resolutionSuper-resolution (result!)Video Matching [Sand & Teller, 2004]Slide 50Texture Synthesis15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005© Darren Green (www.darrensworld.com)Texture•Texture depicts spatially repeating patterns•Many natural phenomena are texturesradishes rocks yogurtTexture Synthesis•Goal of Texture Synthesis: create new samples of a given texture•Many applications: virtual environments, hole-filling, texturing surfacesThe Challenge•Need to model the whole spectrum: from repeated to stochastic texturerepeatedstochasticBoth?Efros & Leung Algorithm•Assuming Markov property, compute P(p|N(p))–Building explicit probability tables infeasible ppSynthesizing a pixelnon-parametricsamplingInput image –Instead, we search the input image for all similar neighborhoods — that’s our pdf for p–To sample from this pdf, just pick one match at randomSome Details•Growing is in “onion skin” order–Within each “layer”, pixels with most neighbors are synthesized first–If no close match can be found, the pixel is not synthesized until the end•Using Gaussian-weighted SSD is very important– to make sure the new pixel agrees with its closest neighbors–Approximates reduction to a smaller neighborhood window if data is too sparseNeighborhood WindowinputVarying Window SizeIncreasing window sizeSynthesis Resultsfrench canvas rafia weaveMore Resultswhite bread brick wallHomage to ShannonHole FillingExtrapolationSummary•The Efros & Leung algorithm–Very simple–Surprisingly good results–Synthesis is easier than analysis!–…but very slowppImage Quilting [Efros & Freeman]•Observation: neighbor pixels are highly correlatedInput image non-parametricsamplingBBIdea:Idea: unit of synthesis = block unit of synthesis = block•Exactly the same but now we want P(B|N(B))•Much faster: synthesize all pixels in a block at once•Not the same as multi-scale!Synthesizing a blockInput textureB1 B2Random placement of blocks blockB1B2Neighboring blocksconstrained by overlapB1 B2Minimal errorboundary cutmin. error boundaryMinimal error boundaryoverlapping blocks vertical boundary__==22overlap errorOur Philosophy•The “Corrupt Professor’s Algorithm”:–Plagiarize as much of the source image as you can–Then try to cover up the evidence•Rationale: –Texture blocks are by definition correct samples of texture so problem only connecting them togetherFailures(ChernobylHarvest)input imagePortilla & SimoncelliWei & Levoy Our algorithmXu, Guo & ShumPortilla & SimoncelliWei & Levoy Our algorithmXu, Guo & Shuminput imagePortilla & SimoncelliWei & Levoy Our algorithminput imageXu, Guo & ShumPolitical Texture Synthesis!MS Digital Image Pro (DEMO)Fill Order•In what order should we fill the pixels?Fill Order•In what order should we fill the pixels?–choose pixels that have more neighbors filled–choose pixels that are continuations of lines/curves/edgesCriminisi, Perez, and Toyama. “Object Removal by Exemplar-based Inpainting,” Proc. CVPR, 2003.Exemplar-based Inpainting demohttp://research.microsoft.com/vision/cambridge/i3l/patchworks.htm++==Application: Texture Transfer•Try to explain one object with bits and pieces of another object:Texture Transfer ConstraintTexture sample•Take the texture from one image and “paint” it onto another object Texture TransferSame as texture synthesis, except an additional constraint:1. Consistency of texture 2. Similarity to the image being “explained”==++Image AnalogiesAaron Hertzmann1,2Chuck Jacobs2Nuria Oliver2Brian Curless3David Salesin2,31New York University2Microsoft Research3University of WashingtonImage AnalogiesA A’B B’Blur FilterEdge FilterA A’B B’Artistic FiltersColorizationTexture-by-numbersA A’B B’Super-resolutionAA’Super-resolution (result!)BB’Video Matching [Sand & Teller, 2004]Motion MagnificationMotion MagnificationCe Liu Antonio Torralba William T. FreemanFrédo Durand Edward H. AdelsonComputer Science and Artificial Intelligence LaboratoryMassachusetts Institute of
View Full Document
Unlocking...