Image Pyramids and BlendingImage PyramidsWhat are they good for?Gaussian pyramid constructionImage sub-samplingImage sub-samplingSamplingReally bad in videoAlias: n., an assumed nameGaussian pre-filteringSubsampling with Gaussian pre-filteringCompare with...Image BlendingFeatheringAffect of Window SizeAffect of Window SizeGood Window SizeWhat is the Optimal Window?What if the Frequency Spread is WideWhat does blurring take away?What does blurring take away?High-Pass filterBand-pass filteringLaplacian PyramidPyramid BlendingPyramid BlendingLaplacian Pyramid: BlendingBlending RegionsHorror PhotoSeason Blending (St. Petersburg)Season Blending (St. Petersburg)Simplification: Two-band Blending2-band BlendingLinear Blending2-band BlendingGradient DomainGradient Domain blending (1D)Gradient Domain Blending (2D)Comparisons: Levin et al, 2004Perez et al., 2003Perez et al, 2003Don’t blend, CUT!Davis, 1998Efros & Freeman, 2001Minimal error boundaryGraphcutsGraph cuts (simple example à la Boykov&Jolly, ICCV’01)Kwatra et al, 2003Lazy Snapping (Li el al., 2004)Putting it all togetherImage Pyramids and Blending15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005© Kenneth KwanImage PyramidsKnown as a Gaussian Pyramid [Burt and Adelson, 1983]• In computer graphics, a mip map [Williams, 1983]• A precursor to wavelet transformA bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s noseFigure from David ForsythWhat are they good for?Improve Search• Search over translations– Like homework– Classic coarse-to-fine strategy• Search over scale– Template matching– E.g. find a face at different scalesPrecomputation• Need to access image at different blur levels• Useful for texture mapping at different resolutions (called mip-mapping) Image Processing• Editing frequency bands separately• E.g. image blending…Gaussian pyramid constructionfilter maskRepeat•Filter•SubsampleUntil minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid)The whole pyramid is only 4/3 the size of the original image!Image sub-samplingThrow away every other row and column to create a 1/2 size image-calledimage sub-sampling1/41/8Image sub-sampling1/4 (2x zoom) 1/8 (4x zoom)Why does this look so bad?1/2SamplingGood sampling:•Sample often or,•Sample wiselyBad sampling:•see aliasing in action!Really bad in videoAlias: n., an assumed namePicket fence recedingInto the distance willproduce aliasing…Input signal:x = 0:.05:5; imagesc(sin((2.^x).*x))Matlab output:WHY?Aj-aj-aj:Alias!Not enough samplesGaussian pre-filteringG 1/4G 1/8Gaussian 1/2Solution: filter the image, then subsample• Filter size should double for each ½ size reduction. Why?Subsampling with Gaussian pre-filteringG 1/4 G 1/8Gaussian 1/2Solution: filter the image, then subsample• Filter size should double for each ½ size reduction. Why?• How can we speed this up?Compare with...1/4 (2x zoom) 1/8 (4x zoom)1/2Image BlendingFeathering0101+=Encoding transparencyI(x,y) = (αR, αG, αB, α) Iblend= Ileft+ IrightAffect of Window Size01leftright01Affect of Window Size0101Good Window Size01“Optimal” Window: smooth but not ghostedWhat is the Optimal Window?To avoid seams• window >= size of largest prominent featureTo avoid ghosting• window <= 2*size of smallest prominent featureNatural to cast this in the Fourier domain• largest frequency <= 2*size of smallest frequency• image frequency content should occupy one “octave” (power of two)FFTWhat if the Frequency Spread is WideIdea (Burt and Adelson)• Compute Fleft= FFT(Ileft), Fright= FFT(Iright)• Decompose Fourier image into octaves (bands)–Fleft= Fleft1+ Fleft2+ …• Feather corresponding octaves Fleftiwith Frighti– Can compute inverse FFT and feather in spatial domain• Sum feathered octave images in frequency domainBetter implemented in spatial domainFFTWhat does blurring take away?originalWhat does blurring take away?smoothed (5x5 Gaussian)High-Pass filtersmoothed – originalBand-pass filteringLaplacian Pyramid (subband images)Created from Gaussian pyramid by subtractionGaussian Pyramid (low-pass images)Laplacian PyramidHow can we reconstruct (collapse) this pyramid into the original image?Need this!OriginalimagePyramid Blending010101Left pyramid Right pyramidblendPyramid Blendinglaplacianlevel4laplacianlevel2laplacianlevel0left pyramid right pyramid blended pyramidLaplacian Pyramid: BlendingGeneral Approach:1. Build Laplacian pyramids LA and LB from images A and B2. Build a Gaussian pyramid GR from selected region R3. Form a combined pyramid LS from LA and LB using nodes of GR as weights:• LS(i,j) = GR(I,j,)*LA(I,j) + (1-GR(I,j))*LB(I,j)4. Collapse the LS pyramid to get the final blended imageBlending RegionsHorror Photo© prof. dmartinSeason Blending (St. Petersburg)Season Blending (St. Petersburg)Simplification: Two-band BlendingBrown & Lowe, 2003• Only use two bands: high freq. and low freq.• Blends low freq. smoothly• Blend high freq. with no smoothing: use binary maskLow frequency (λ > 2 pixels)High frequency (λ < 2 pixels)2-band BlendingLinear Blending2-band BlendingGradient DomainIn Pyramid Blending, we decomposed our image into 2ndderivatives (Laplacian) and a low-res imageLet us now look at 1stderivatives (gradients):• No need for low-res image – captures everything (up to a constant)• Idea: – Differentiate– Blend– ReintegrateGradient Domain blending (1D)TwosignalsRegularblendingBlendingderivativesbrightdarkGradient Domain Blending (2D)Trickier in 2D:• Take partial derivatives dx and dy (the gradient field)• Fidle around with them (smooth, blend, feather, etc)• Reintegrate– But now integral(dx) might not equal integral(dy)• Find the most agreeable solution– Equivalent to solving Poisson equation– Can use FFT, deconvolution, multigrid solvers, etc.Comparisons: Levin et al, 2004Perez et al., 2003Perez et al, 2003Limitations:• Can’t do contrast reversal (gray on black -> gray on white)• Colored backgrounds “bleed through”• Images need to be very well alignededitingDon’t blend, CUT!So far we only tried to blend between two images. What about finding an optimal seam?Moving objects become ghostsDavis, 1998Segment the mosaic• Single source image per segment• Avoid artifacts along boundries– Dijkstra’s algorithmInput textureB1 B2Random placement of blocks blockB1B2Neighboring
View Full Document
Unlocking...