1Lighting affects appearance2Image NormalizationGlobalHistogram Equalization. Make two images have same histogram. Or, pick a standard histogram, and make adjust each image to have that histogram.Apply monotonic transform to intensities.Additive and multiplicative normalization.Subtract mean intensity, divide by total magnitude of result.LocalNormalized cross-correlation: Normalize windows and then compare with SSD.Normalize intensity and first derivatives -> direction of gradient.Normalize filter outputs: eg., Gabor Jets.3Histogram: H: I in R^2 -> f:Z->R. That is, computing a histogram takes in an image as input, and returns a function as output that maps integer intensity values to a frequency. In this case, f(i) = sum_x I(x)==i, where I(x)==i acts like an indicator function.Histogram equalization: I in R^2,f: (R->R) -> J in R^2. That is, histogram equalization takes an image and a desired histogram as input, and produces a new image. We have J(x) = g(I(x)), where x is an index into the image. J(x) is a histogram equalized version of I(x) if H(J) = f (that is, the J has the desired histogram, f) and g is a montonically increasing function.Histogram equalization undoes any montonic change to the intensities. That is, suppose h is a montonically increasing function. Suppose also that I and I’are images, such that I’(x) = h(I(x)). Then, for any target histogram, f, H(I’,f) = H(I,f). That is, I and I’ will be the same after histogram equalization.Normalization. Suppose I is an image. Let mean(I) denote the mean intensity of I. Then I’ = I - mean(I) is normalized to have zero mean. Let std(I’) be the standard deviation of intensities in I. Then I’’ = (I-mean(I))/std(I) is normalized to have zero mean and unit standard deviation. This removes additive and multiplicative changes to the image.Direction of gradient. Let grad(I(x)) denote the image gradient of I at point x. Then we can represent the image with the direction of the gradient, D(x) = grad(I)/||grad(I)||. This is equivalent to normalizing the image very locally, since additive and multiplicative changes to the local image can map the intensity and magnitude of the image gradient to arbitrary values.How do we represent light? (1)Ideal distant point source:- No cast shadows- Light distant- Three parameters- Example: lab with controlled light4How do we represent light? (2)Environment map: l(θ,φ)- Light from all directions- Diffuse or point sources- Still distant- Still no cast shadows.- Example: outdoors (sky and sun)Sky`5Lambertian + Point SourceSurface normalLightθθθθlˆnormal surface isˆ isradiance is)ˆ(,0max(light ofintensity islight ofdirection isnalbedoinlilllllλλ•= •=Lambertian, point sources, no shadows. (Shashua, Moses)WhiteboardSolution linearLinear ambiguity in recovering scaled normalsLighting not known.Recognition by linear combinations.6Linear basis for lightingλZλYλXA brief Detour: Fourier Transform, the other linear basisAnalytic geometry gives a coordinate system for describing geometric objects.Fourier transform gives a coordinate system for functions.7BasisP=(x,y) means P = x(1,0)+y(0,1)Similarly: ++++=)2sin()2cos()sin()cos()(22122111θθθθθaaaafNote, I’m showing non-standard basis, these are from basis using complex functions.Exampleθθθsincos)cos(:such that ,,2121aacaac+=+∃∀8Orthonormal Basis||(1,0)||=||(0,1)||=1(1,0).(0,1)=0Similarly we use normal basis elements eg:While, eg:=πθθθθθ202cos)cos()cos()cos(d=πθθθ200sincos d2D Example910Convolution−=∗=000)()()(dxxhxxghgxfImagine that we generate a point in f by centering h over the corresponding point in g, then multiplying g and h together, and integrating.Convolution TheoremGFTgf* 1−=⊗• F,G are transform of f,gThat is, F contains coefficients, when we write f as linear combinations of harmonic basis.11Examples?)3cos1.2cos2.(cos?cos?2coscos?coscos=⊗++=⊗=⊗=⊗ffθθθθθθθθLow-pass filter removes low frequencies from signal. Hi-pass filter removes high frequencies. Examples?ShadowsAttached ShadowCast Shadow1290.797.296.399.5#988.596.395.399.1#784.794.193.597.9#576.388.290.294.4#342.867.953.748.2#1ParrotPhoneFaceBall(Epstein, Hallinan and Yuille; see also Hallinan; Belhumeur and Kriegman)5 2D±Dimension:With Shadows: PCADomainDomainLambertianEnvironment mapnlθ lλmax (cosθ, 0)13Images...LightingReflectance0 1 2 300.510 1 2 300.511.52 rLighting to Reflectance: Intuition14++++++++++++15Spherical HarmonicsOrthonormal basis, , for functions on the sphere. n’th order harmonics have 2n+1 components.Rotation = phase shift (same n, different m). In space coordinates: polynomials of degree n.S.H. used for BRDFs (Cabral et al.; Westin et al;). (See also Koenderink and van Doorn.)φθπφθimnmnmePmnmnnh )(cos)!()!(4)12(),(+−+=nmnmnnmnmzdzdnzzP )1(!2)1()(222−−=++nmhS.H. analog to convolution theorem• Funk-Hecke theorem: “Convolution” in function domain is multiplication in spherical harmonic domain.• k is low-pass filter.k16Harmonic Transform of Kernel00( ) max(cos ,0)n nnk k hθ θ∞== =12( 2)! (2 1)13( 1) 2, even2 ( 1)!( 1)!0 2, od22d20nnnn nnn nknnnπππ+− +− ≥− ==≥+=0.8860.5910.2220.0370.0140.00700.511.50 1 2 3 4 5 6 7 8Amplitudes of KernelnnA17Energy of Lambertian Kernel in low order harmonicsAccumulated Energy37.587.599.299.81 99.93 99.970204060801001200 1 2 4 6 8Reflectance Functions Near Low-dimensional Linear Subspace0( )nnm nm nmn m nr k l K L h= =−∞= ∗ = Yields 9D linear subspace. = −=≈20)(nnnmnmnmnmhLK18How accurate is approximation?Point light sourceAmplitude of k-0.500.511.522.533.50 1 2 4 6 8Amplitude of l = point source00.20.40.60.811.20 1 2 4 6 89D space captures 99.2% of energy  = −=∞= −=≈=∗=200)()(nnnmnmnmnmnnnmnmnmnmhLKhLKlkrHow accurate is approximation? (2)Worst case.9D space captures 98% of energyDC component as big as any other.1st and 2nd harmonics of light could have zero energyAmplitude of k-0.500.511.522.533.50 1 2 4 6 8Amplitude of l = point source00.20.40.60.811.20 1 2 4 6 819Forming Harmonic Imagesnm nmb (p)=r (X,Y,Z)λ λZ λYλX2 2 22(Z -X -Y )λXZ λYZ2 2(X -Y )λXYCompare this to 3D SubspaceλZλYλX20Accuracy of Approximation of ImagesNormals present to varying amounts.Albedo makes some pixels more important.Worst case approximation arbitrarily bad.“Average” case approximation