Today: non-linear filters, and uses for the filters and representations from last timeReadingMid-term examImage representationsImage pyramidsWavelet/QMF representationLinear image transformationsSchematic pictures of each matrix transformFourier transformGaussian pyramidLaplacian pyramidWavelet (QMF) transformSteerable pyramidMatlab resources for pyramids (with tutorial)Why use these representations?Image statistics (or, mathematically, how can you tell image from noise?)Bayesian MAP estimator for clean bandpass coefficient valuesBayesian MAP estimatorBayesian MAP estimatorNoise removal resultsImage textureTextureThe Goal of Texture SynthesisThe Goal of Texture AnalysisPre-attentive texture discriminationPre-attentive texture discriminationPre-attentive texture discriminationPre-attentive texture discriminationPre-attentive texture discriminationPre-attentive texture discriminationJuleszInfluential paper:Bergen and Adelson, Nature 1988Malik and PeronaRepresenting texturesBergen and HeegerBergen and Heeger resultsBergen and Heeger failuresDe Bonet (and Viola)DeBonetDeBonetPortilla and SimoncelliPortilla and SimoncelliZhu, Wu, & Mumford, 1998Zhu, Wu, & MumfordEfros and LeungWhat we learned from Efros and Leung regarding texture synthesisEfros & Leung ’99Efros & Leung ’99 extendedImage QuiltingMinimal error boundaryOur PhilosophyAlgorithmTexture TransferSummary of image quiltingMedian filterDegraded imageRadius 1 median filterCCD color samplingColor sensing, 3 approachesTypical errors in temporal multiplexing approachTypical errors in spatial multiplexing approach.CCD color filter patternThe cause of color moireBlack and white edge falling on color CCD detectorColor sampling artifactTypical color moire patternsColor sampling artifactsHuman PhotoreceptorsBrewster’s colors example (subtle).Median Filter InterpolationTwo-color sampling of BW edgeR-G, after linear interpolationR – G, median filtered (5x5)Recombining the median filtered colorsDidn’t get a chance to show:12Today: non-linear filters, and uses for the filters and representations from last time• Review pyramid representations• Non-linear filtering• Textures3Reading• Related to today’s lecture: – Chapter 9, Forsyth&Ponce..• For next Thursday’s lecture:– Horn, Ch. 12– Bishop chapter 1 (handout from last lecture)4Mid-term examProblem set 3 given out today– Open book, open web.– Work by yourself. This problem set is a mid-term exam, and you can’t: talk about it, e-mail about it, give hints, etc, with others.– Due Tuesday, Oct. 22 (in 12 days).5Image representations• Fourier basis• Image pyramids6Image pyramidsShows the information added in Gaussian pyramid at each spatial scale. Useful for noise reduction & coding.Progressively blurred and subsampled versions of the image. Adds scale invariance to fixed-size algorithms.Shows components at each scale and orientation separately. Non-aliased subbands. Good for texture and feature analysis.Bandpassed representation, complete, but with aliasing and some non-oriented subbands.• Gaussian• Laplacian• Wavelet/QMF• Steerable pyramid7Wavelet/QMF representation8Linear image transformations• In analyzing images, it’s often useful to make a change of basis.Fourier transform, orWavelet transform, orSteerable pyramid transformfUFrr=transformed imageVectorized image9Schematic pictures of each matrix transform• Shown for 1-d images• The matrices for 2-d images are the same idea, but more complicated, to account for vertical, as well as horizontal, neighbor relationships.10Fourier transform=*Fourier transformFourier bases are global: each transform coefficient depends on all pixel locations.pixel domain image11Gaussian pyramid=Overcomplete representation. Low-pass filters, sampled appropriately for their blur.*Gaussian pyramidpixel image12Laplacian pyramid=Overcomplete representation. Transformed pixels represent bandpassed image information.*Laplacianpyramidpixel image13Wavelet (QMF) transformWavelet pyramid=*Ortho-normal transform (like Fourier transform), but with localized basis functions. pixel image14=Multiple orientations at one scale Multiple orientations at the next scale the next scale… Steerable pyramid*Steerablepyramidpixel imageOver-complete representation, but non-aliased subbands.15Matlab resources for pyramids (with tutorial)http://www.cns.nyu.edu/~eero/software.html16Why use these representations?• Handle real-world size variations with a constant-size vision algorithm.• Remove noise• Analyze texture• Recognize objects• Label image features17Image statistics (or, mathematically, how can you tell image from noise?)18P(x)Bayesian MAP estimator for clean bandpasscoefficient valuesLet x = bandpassed image value before adding noise.Let y = noise-corrupted observation.By Bayes theoremP(x|y) = k P(y|x) P(x)P(y|x)yP(y|x)P(x|y)P(x|y)19P(x)Let x = bandpassed image value before adding noise.Let y = noise-corrupted observation.By Bayes theoremP(x|y) = k P(y|x) P(x)P(y|x)yP(y|x)P(x|y)P(x|y)Bayesian MAP estimator20P(x)Let x = bandpassed image value before adding noise.Let y = noise-corrupted observation.By Bayes theoremP(x|y) = k P(y|x) P(x)P(y|x)yP(y|x)P(x|y)P(x|y)Bayesian MAP estimator21Noise removal resultshttp://www-bcs.mit.edu/people/adelson/pub_pdfs/simoncelli_noise.pdfSimoncelli and Adelson, Noise Removal via Bayesian Wavelet Coring22Image texture23Texture• Key issue: representing texture– Texture based matching• little is known– Texture segmentation• key issue: representing texture– Texture synthesis• useful; also gives some insight into quality of representation– Shape from texture• cover superficially24The Goal of Texture SynthesisTrue (infinite) textureSYNTHESISgenerated imageinput image• Given a finite sample of some texture, the goal is to synthesize other samples from that same texture– The sample needs to be "large enough“25The Goal of Texture AnalysisTrue (infinite) textureANALYSISgenerated imageinput image“Same” or “different”Compare textures and decide if they’re made of the same “stuff”.26Pre-attentive texture discrimination27Pre-attentive texture discrimination28Pre-attentive texture discriminationSame or different textures?29Pre-attentive texture discrimination30Pre-attentive texture discrimination31Pre-attentive texture discriminationSame or different textures?32Julesz• Textons: analyze the texture in terms of statistical relationships between