Patch Based Texture Synthesis in 2D and 3D Presented by Junwen Wu CSE291 J Feb 27 2003 Introduction to Texture Synthesis Problem Description Given an input texture sample image Iin Target an unlimited amount of image data Iout which is perceived by the human observer as the same texture Picking the right set of statistics to match Finding an algorithm to match them Introduction to Texture Synthesis Two classes of textures Stochastic Texture V S Structured Texture Histogram matching Conditional distribution preserving under multi scale Patch pasting Pixel wise non parametric sampling Pixel wise non parametric sampling Greedy strategy Slow Patch based Texture Synthesis Patch based Texture Synthesis in Images Alexei A Efros William T Freeman Image quilting for Texture Synthesis and Transfer SIGGRAPH 2001 Image Quilting Lin Liang Ce Liu Ying qing Xu Baining Guo and Heung yeung Shum Real time texture synthesis by patch based sampling ACM Transaction on Graphics Vol 20 No 3 July 2001 Pages 127 150 Microsoft Paper Basic Idea Basic Idea Image Quilting Minimum Error Boundary cut Microsoft Paper Image Feathering blending Be performed along seams Makes the sharp changes occurring at the cut line appear more gradual Minimum Error Boundary Cut E11 E12 E13 E14 E21 E22 E23 E24 e22 Min E11 E12 E13 eij Pin i j PMatch i j 2 eij i 1 Eij eij Min Ei 1 j 1 Ei 1 j Ei 1 j 1 i 1 Minimum of last row Image Feathering Pout i j ij Pin i j 1 ij PMatch i j Using ij to control the blending effect One possible criterion to select ij is by their distance to the cutting line How to choose the matching patch Image Quilting The most similar patch in the input image which is defined as Bopt arg min dist Bin Bout Bin I in Bin Bin and Bout are the overlapped region in the patch Bin and Bout Microsoft Paper case SMatch Randomly select from the set SMatch In select the most similar patch S Match Bin dist Bin Bout d max 1 2 1 d max Pout i j 2 A i j Pin out i j Bin out A is the area of Bin out Common Parameters in Both Approaches Patch size Smaller means more matching possibility yet weaker statistical constraint From left to right Iin Iout by patch size 16x16 24x24 32x32 Width of Overlapped Band Wider band implies stronger statistical constraints yet less matching possibilities More Parameters in Microsoft Paper Distance tolerance dmax Controlled by Control the similarity between the synthesis texture and the input texture Smaller More similarity in local structures Greater Discontinuous transition between patches From left to right Iin Iout with 0 0 2 1 respectively Three Steps of Accelerating Technique Step 1 Optimized KD tree for the data points P4 P1 P1 P3 P2 P2 P3 P4 Three Steps of Accelerating Technique Step 1 Contd Optimized KD tree for the data points Partition the data space into hypercubes by axis orthogonal hyperplanes Node hypercubes enclosing a set of data points Construction rule Sliding mid point rule V S standard KDtree splitting rule Both will produce cubes with high aspect ratio High aspect ratio will increase error Sliding mid point rule can prevent it from causing problems Three Steps of Accelerating Technique Step 2 Quadtree Pyramid Take advantage of image data Gaussian Pyramid Reduce Expand Calculated over Iin all other data points can be extracted from filtered Iin At every level four children higher level images are computed over images with different shifting along x and y directions Compare with Gaussian Pyramid Same reduce Smoothing sub sampling and expand interpolating different Gaussian pyramid cannot always find the corresponding pixel in the higher level to the lower level patches Three Steps of Accelerating Technique Step 3 PCA Data dimension reduction A lower dimension representation Expanded by the first several eigenvectors of the covariance matrix Results Results from Liang s paper From left the right Input sample texture images Synthesized texture from Liang s approach Synthesized texture from Efros and Freeman s approach Texture Transfer Texture Transfer Image quilting is suitable for it image quilting is based on local image information A desired correspondence map should be satisfied as well as the texture synthesis requirement Correspondence map C Source Image Controlling Target Image Spatial Map of some Corresponding Quantity Error term is modified to be e ij Pin i j Pout i j 2 1 C Pin i j C Pout i j 2 Results of Image Transfer Correspondence map Luminance value Correspondence map Blurred Luminance value QUESTIONS
View Full Document
Unlocking...