9/28/09 CS 461, Copyright G.D. Hager Computer Vision Linear Filtering and Edge Detection Professor Hager http://www.cs.jhu.edu/~hager9/28/09 CS 461, Copyright G.D. Hager Outline • Image noise models • Filtering by Convolution • Properties of Convolution • Derivative Operators Goal: To understand the properties of common linear and nonlinear filtering operations on gray-scale images as a basis for many solutions in computer vision.9/28/09 CS 461, Copyright G.D. Hager IMAGE NOISE Cameras are not perfect sensors and Scenes never quite match our expectations9/28/09 CS 461, Copyright G.D. Hager Noise Models • Noise is commonly modeled using the notion of “additive white noise.” – Images: I(u,v,t) = I*(u,v,t) + n(u,v,t) – Note that n(u,v,t) is independent of n(u’,v’,t’) unless u’=u,u’=u,t’=t. – Typically we assume that n (noise) is independent of image location as well --- that is, it is i.i.d – Typically we assume the n is zero mean, that is E[n(u,v,t)]=0 • A typical noise model is the Gaussian (or normal) distribution parametrized by π and σ • This implies that no two images of the same scene are ever identical9/28/09 CS 461, Copyright G.D. Hager Gaussian!Noise:!sigma=1!9/28/09 CS 461, Copyright G.D. Hager Gaussian Noise: sigma=16!9/28/09 CS 461, Copyright G.D. Hager Properties of Noise Processes • Properties of temporal image noise: Mean µ(i,j) = Σ I(u,v,t)/n Standard Deviation σi,j = Sqrt( Σ ( µ(ι,ϕ) – I(u,v,t) )2/n ) Signal-to-noise Ratio σi,j µ (i,j)9/28/09 CS 461, Copyright G.D. Hager Image Noise • An experiment: take several images of a static scene and look at the pixel values mean = 38.6 std = 2.99 Snr = 38.6/2.99 ≈ 13 max snr = 255/3 ≈ 859/28/09 CS 461, Copyright G.D. Hager PROPERTIES OF TEMPORAL IMAGE NOISE (i.e., successive images) Sqrt(n) σ If standard deviation of grey values at a pixel is s for a pixel for a single image, then the laws of statistics states that for independent sampling of grey values, for a temporal average of n images, the standard deviation is: For example, if we want to double the signal to noise ratio, we could average 4 images.9/28/09 CS 461, Copyright G.D. Hager Temporal vs. Spatial Noise • It is common to assume that: – spatial noise in an image is consistent with the temporal image noise – the spatial noise is independent and identically distributed • Thus, we can think of a neighborhood of the image itself as approximated by an additive noise process • Averaging is a common way to reduce noise – instead of temporal averaging, how about spatial? • For example, for a pixel in image I at i,j )','(9/1),('11'11'jiIjiIiiijjj∑ ∑+−=+−==9/28/09 CS 461, Copyright G.D. Hager Convolution (I * K)(x,y) = Σi Σj I(x-i,y-j) K(i,j) Note these indices run backwards --- this can sometimes fool you down the road! We often write I’ = I*K to represent the convolution of I by K. K is referred to as the kernel of the convolution (or sometimes the “stencil” in the discrete case). It is also the impulse response of the filter. Convolution is the generalization of this “averaging” process. As we’ll see, it can do more than average.9/28/09 CS 461, Copyright G.D. Hager DISCRETE CONVOLUTION Template ‘Kernel’ T1 T2 T3 T4 T5 T6 T7 T8 T9 I1 I2 I3 I4 I5 I6 I7 I8 I9 Image I = T9 x I1 + T8 x I2 + T7 x I3 + T6 x I4 + T5 x I5 + T4 x I6 + T3 x I7 + T2 x I8 + T1 x I9 3x3 Template Local Image Neighborhood9/28/09 CS 461, Copyright G.D. Hager Convolution Image (I) 2 4 2 Kernel (K) * Note: Typically Kernel is relatively small in vision applications. 2 1 1 2 1 1 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Convolution: R= K*I I R ∑ ∑−= −=−−=2/2/2/2/),(),(),(mmhmmkkjhiIkhKjiRKernel size is m+1 by m+1 m=2 borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager Impulse Response 1 * = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 borrowed from D. Kriegman Note that the “self-replicating” property of the impulse response is what require the “flip” in the template when it is applied9/28/09 CS 461, Copyright G.D. Hager Smoothing by Averaging Kernel: borrowed from D. Kriegman9/28/09 CS 461, Copyright G.D. Hager (Swiped from Bill Freeman)9/28/09 CS 461, Copyright G.D. Hager (Swiped from Bill Freeman)9/28/09 CS 461, Copyright G.D. Hager (Swiped from Bill Freeman)9/28/09 CS 461, Copyright G.D. Hager (Swiped from Bill Freeman)9/28/09 CS 461, Copyright G.D. Hager (Swiped from Bill Freeman)9/28/09 CS 461, Copyright G.D. Hager (Swiped from Bill Freeman)9/28/09 CS 461, Copyright G.D.