Readings (handouts)HomeworksImage representationLinear image transformationsAn example of such a transform: the Fourier transformFourier transform magnitudeWhoops; this was a mistakeMasking out the fundamental and harmonics from periodic pillarsScaled representationsImage pyramidsThe Gaussian pyramidThe computational advantage of pyramidsConvolution and subsampling as a matrix multiply (1-d case)Next pyramid levelb * a, the combined effect of the two pyramid levelsThe Laplacian PyramidApplication to image compressionWavelets/QMF’sFrequency characteristics of the high and low-pass filtersGood and bad features of wavelet/QMF filtersSteerable pyramidsMatlab resources for pyramids (with tutorial)Matlab resources for pyramids (with tutorial)Image statistics (or, mathematically, how can you tell image from noise?)Pixel representation image histogrambandpass filtered imagebandpassed representation image histogramPixel domain noise image and histogramBandpass domain noise image and histogramNoise-corrupted full-freq and bandpass imagesBayes theoremBayesian MAP estimator for clean bandpass coefficient valuesBayesian MAP estimatorBayesian MAP estimatorMAP estimate, , as function of observed coefficient value, yNoise removal resultsSpatial resolution and colorBlurring the G componentBlurring the R componentBlurring the B componentLab componentsBlurring the L Lab componentBlurring the a Lab componentBlurring the b Lab componentApplication to image compressionBandwidth (transmission resources) for the components of the television signal12Readings (handouts)• Pyramids and multi-scale representations paper, in “Representations of Vision”• Chapter 1 of Bishop’s book, esp. sect. 1.83Homeworks• Problem set 2 due today.• Problem set 3 given out Thursday– Open book, open web.– Closed mouth, ie, this problem set is a mid-term exam, and you can’t talk about it, pass notes, etc, with other people.4Image representation• Fourier basis• Image pyramids• Image statistics• Color and spatial frequency effects5Linear image transformations• In analyzing images, it’s often useful to make a change of basis.Fourier transform, orWavelet transform, orSteerable pyramid transformfUFrr=transformed imageVectorized image6An example of such a transform: the Fourier transformdiscrete domain∑∑−=−=+−=1010ln],[],[MkNlNMkmielkfnmFπForward transform∑∑−=−=++=1010ln],[1],[MkNlNMkmienmFMNlkfπInverse transform7To get some sense of what basis elements look like, we plot a basis element --- or rather, its real part ---as a function of x,y for some fixed u, v. We get a function that is constant when (ux+vy) is constant. The magnitude of the vector (u, v) gives a frequency, and its direction gives an orientation. The function is a sinusoid with this frequency along the direction, and constant perpendicular to the direction. uv()vyuxie+−π()vyuxie+π8uv()vyuxie+−π()vyuxie+πHere u and v are larger than in the previous slide.9uv()vyuxie+−π()vyuxie+πAnd larger still...10Fourier transform magnitude11Whoops; this was a mistakeI’d inadvertently zeroed out the DC component12Masking out the fundamental and harmonics from periodic pillars13Scaled representations• Alternative:– Apply filters of fixed size to images of different sizes– Typically, a collection of images whose edge length changes by a factor of 2 (or root 2)– This is a pyramid (or Gaussian pyramid) by visual analogy• Big bars (resp. spots, hands, etc.) and little bars are both interesting– Stripes and hairs, say• Inefficient to detect big bars with big filters– And there is superfluous detail in the filter kernel14Example application: CMU face detectorFrom: http://www.ius.cs.cmu.edu/IUS/har2/har/www/CMU-CS-95-158R/15Image pyramids• Gaussian• Laplacian• Wavelet/QMF• Steerable pyramid16The Gaussian pyramid• Smooth with gaussians, because– a gaussian*gaussian=another gaussian • Synthesis – smooth and sample•Analysis– take the top image• Gaussians are low pass filters, so repn is redundant17The computational advantage of pyramidshttp://www-bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf18http://www-bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf19http://www-bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf2021Convolution and subsampling as a matrix multiply (1-d case)U1 =1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 022Next pyramid levelU2 =1 4 6 4 1 0 0 00 0 1 4 6 4 1 00 0 0 0 1 4 6 40 0 0 0 0 0 1 423b * a, the combined effect of the two pyramid levels>> U2 * U1ans =1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 0 0 0 0 00 0 0 0 1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 00 0 0 0 0 0 0 0 1 4 10 20 31 40 44 40 30 16 4 00 0 0 0 0 0 0 0 0 0 0 0 1 4 10 20 25 16 4 024The Laplacian Pyramid• Synthesis– preserve difference between upsampledGaussian pyramid level and Gaussian pyramid level– band pass filter - each level represents spatial frequencies (largely) unrepresented at other levels•Analysis– reconstruct Gaussian pyramid, take top layer25http://www-bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf262728Application to image