Texture Synthesis by Non parametric Sampling Image Quilting for Texture Synthesis Transfer by Efros and Leung Efros and Freeman ICCV 99 SIGGRAPH 01 Presentation by Gyozo Gidofalvi Computer Science and Engineering Department University of California San Diego gyozo cs ucsd edu November 15 2001 Outline Background information on texture Growing texture pixel by pixel using a simple non parametric method Image quilting a very simple texture synthesis algorithm Simple extension to image quilting for texture transfer Applications Summary What is a texture What features and statistics are characteristics of a texture pattern so that texture pairs that share the same features and statistics cannot be told apart by pre attentive human visual perception Julesz 1960s 1980s The concept of texture is intuitively obvious but has no precise definition something consisting of mutually related elements On can describe texture by tone and structure Tone is based on pixel intensity properties Structure describes spatial relationships of primitives Texture Analysis Techniques Statistical methods gather information about textures by exploiting pixel first and second order statistics Structural methods describe textures as composed of well defined texture primitives texels which are placed according to some syntactic rules Model based methods construct a generative or stochastic model of textures called random field models Transform methods represent an image in a new form in which the characteristics of the texture become more easily accessible Some examples include Fourier transforms and multi resolution methods What is texture synthesis Given an input sample texture synthesize a texture that is sufficiently different from the given sample texture yet appears perceptually to be generated by the same underlying stochastic process input image SYNTHESIS True infinite texture generated image Classification of texture Traditionally textures has been classified as regular repeated texels stochastic without explicit texels Most real world textures are mixtures of these basic types Challenge is to model the whole spectrum from regular to stochastic texture regular stochastic both Some previous approaches multi scale filter response histogram matching Heeger and Bergen 95 sampling from conditional distribution over multiple scales DeBonet 97 filter histograms with Gibbs sampling Zhu et al 98 matching 1st and 2nd order properties of wavelet coefficients Simoncelli and Portilla 98 New method by Efros et al goals preserve local structure model wide range of real textures ability to do constrained synthesis method inspired by N gram language model of Shannon texture is modelled as Markov Random Field MRF texture is grown one pixel at a time conditional pdf of a pixel given its neighbors synthesized thus far is estimated by searching the the sample image for similar neighborhoods N gram model of the English language Shannon Model language as a generalized Markov chain where a set of n letters words completely determine the pdf of the next letter word Results using alt singles corpus Mark V Shaney Shaney One morning I shot an elephant in my arms and kissed him I spent an interesting evening recently with a grain of salt Assuming Markov property texture can be modeled as a MRF Synthesizing one pixel SAMPLE finite sample image p Generated image To synthesize a pixel p search the sample image for pixels with similar neighborhood to p construct a histogram for the distribution of these pixels finally sample this distribution to obtain a value for p Similarity is based on the Gaussian weighted sum squared difference to preserve local structure Growing texture on pixel at the time User defined window size indicates the randomness of the texture To grow from from scratch a 3x3 random seed from the sample is used Unless no close match is found pixels with most neighbors are synthesized first Importance of Gaussian weighted similarity measure Neighborhood window size Randomness parameter More Synthesis Results Increasing window size Results reptile skin aluminium wire More results French canvas rafia weave More results wood granite More results white bread brick wall Constrained synthesis Visual comparison Synthetic tilable texture DeBonet 97 Simple tiling Our approach Failure cases Growing garbage Verbatim copying Homage to Shannon Constrained text synthesis What we have so far An algorithm that is simple models a wide range of real world textures naturally well suited for constrained texture synthesis but it very slow sometimes grows garbage How can it be improved Why only synthesize on pixel at the time For most complex textures only a very few pixels actually have a choice of values wasted search effort Example Pattern of circles on the plane Once the algorithm starts synthesizing a particular circle the values of the remaining pixels are completely determined Unit of synthesis should be more than just a pixel Texture synthesis would be like jigsaw puzzle Questions What are the patches How to put them together patch Chaos Mosaic Xu Guo Shum 00 input idea result Process 1 tile input image 2 pick random blocks and place them in random locations 3 Smooth edges Used in Lapped Textures Praun et al 00 Chaos Mosaic Xu Guo Shum 00 input result The approach works well on stochastic textures but fails on structures Image Quilting non parametric sampling p B Input image Synthesizing a block Idea let s combine random block placement of Chaos Mosaic with spatial constraints of Efros Leung Unit of synthesis is a block Exactly the same but now we want P B N B Much faster synthesize all pixels in a block at once block Input texture B1 B2 B1 B2 B1 B2 Minimal error Random placement Neighboring blocks constrained by overlap boundary cut of blocks Minimal error boundary overlapping blocks vertical boundary 2 overlap error min error boundary Image Quilting algorithm 1 2 Pick size of block and size of overlap Synthesize blocks in raster order 1 Search input texture for block that satisfies overlap constraints above and left Easy to optimize using NN search Liang et al 01 Paste new block into resulting texture use dynamic programming to compute minimal error boundary cut 2 Comparison Portilla Simoncelli Xu Guo Shum input image Wei Levoy Image Quilting Comparison Portilla Simoncelli Xu Guo Shum input image Wei Levoy Image Quilting Homage to Shannon Portilla Simoncelli Xu Guo Shum input image Wei Levoy Image Quilting Failures Chernobyl
View Full Document
Unlocking...