ColorShape-from-shadinglinear filters (Thursday & next Tuesday)Color matching experimentColor matching functions for a particular set of monochromatic primariesUsing the color matching functions to predict the primary match for a new spectral signalColor metamerismMetameric lightsMetamerism applicationsColor appearanceSpatial effects on color appearancedemosOutlineWhat does a color reaching your eye tell you about the world?General strategiesBasis functions for Macbeth color checkerLow-dimensional models for color spectraThe rendering model (using linear combinations of spectral basis functions to represent both surfaces and lights)Count unknowns and equationsA possible assumption: know the mean surface valueShape from shadingGeneric reflectance mapRemember, this model doesn’t handle properly:Linear shading mapLinear shading: 1st order terms of Lambertian shadingLinear shading approximationSimplified lighting assumptionSimplified linear shape from shading computation, for I(n) = S(n+1)-S(N)Illustrates a general problemShape-from-shading as prototypical vision computationShape-from-shading as prototypical vision computationShape-from-shading as prototypical vision computationLook for stationary pointsSet derivatives = 0Set derivatives = 0Results using this approach1Color• Reading: – Chapter 6, Forsyth & Ponce• Optional reading:– Chapter 4 of Wandell, Foundations of Vision,Sinauer, 1995 has a good treatment of this.Oct. 1, 2002MIT 6.801/6.866Profs. Freeman and Darrell2Shape-from-shading• Reading: – Chapter 11 (esp. 11.1, 11.7), HornOct. 1, 2002MIT 6.801/6.866Profs. Freeman and Darrell3linear filters (Thursday & next Tuesday)• Reading: – Chapter 7, F&P• Recommended Reading: – Chapter 7, 8 HornOct. 1, 2002MIT 6.801/6.866Profs. Freeman and Darrell4Color matching experimentFoundations of Vision, by Brian Wandell, Sinauer Assoc., 19955Color matching functions for a particular set of monochromatic primariesp1= 645.2 nmp2= 525.3 nmp3= 444.4 nmFoundations of Vision, by Brian Wandell, Sinauer Assoc., 19956Using the color matching functions to predict the primary match for a new spectral signal=)()()()()()(313212111NNNccccccCλλλλλλLLLStore the color matching functions in the rows of the matrix, C=)()(1NtttλλMrLet the new spectral signal to be characterized be the vector t.Then the amounts of each primary needed to match t are:tCr7CIE XYZ: Color matching functions are positive everywhere, but primaries are imaginary. Usually draw x, y, where x=X/(X+Y+Z)y=Y/(X+Y+Z)Foundations of Vision, by Brian Wandell, Sinauer Assoc., 199589Color metamerism Two spectra, t and s, perceptually match whenwhere C are the color matching functions for some set of primaries.sCtCr=rCtrCsr=Graphically,10Metameric lightsFoundations of Vision, by Brian Wandell, Sinauer Assoc., 199511Metamerism applicationsThe word “VOID” wasn’t visible on the check, when viewed by eye, only after xerox copying did it appear.12Color appearance • Or, color matching outside of the controlled experimental setup.13Spatial effects on color appearance14Forsyth & Ponce15Forsyth & Ponce16demoslightnesscolor17Outline• Color physics.• Color perception and color matching.• Inference about the world from color observations.18What does a color reaching your eye tell you about the world?=.*Foundations of Vision, by Brian Wandell, Sinauer Assoc., 199519General strategies• (a) Determine what image would look like under white light, or (b) surface reflectances• Assume – that we are dealing with flat frontal surfaces– We’ve identified and removed specularities– no variation in illumination• We need some form of reference– brightest patch is white– spatial average is known– gamut is known–specularities– prior probabilities for lights and surfacesForsyth & Ponce20Low-dimensional models for color spectra=321321)()()()(ωωωλλλλMMMMMMMMEEEe21Basis functions for Macbeth color checkerFoundations of Vision, by Brian Wandell, Sinauer Assoc., 199522Low-dimensional models for color spectraFoundations of Vision, by Brian Wandell, Sinauer Assoc., 199523The rendering model (using linear combinations of spectral basis functions to represent both surfaces and lights)skpSpectral response of sensors.*∑∑∑=λλψλφλiiijjsjkskercp )()()(  sjr)(λφjie)(λψi)(λkcSurface spectral basis functionsSurface basis coefficientsIlluminant spectral basis functionsilluminant basis coefficients=Response of kth sensor to the jth surface patch24Count unknowns and equations• Suppose you use a 3-dimensional linear model for both the illuminant and surfaces (two different linear models of the same dimensionality). Suppose have N surfaces.• 3 + 3N unknowns• 3N measurements25A possible assumption: know the mean surface value∑∑∑=λλψλφλiiijjsjkskercp )()()(Sum over and j to writeλ∑∑∑=λλψλφλiiijjsjkskercp )()()(Sensor responses, as function of surfaces and lights, from before:Average over all the N surfaces:Aep =In general, A will be invertible and the estimated illuminant, e, under the “gray world” assumption, is:epA-1=26Shape from shading27Horn, 1986),(),(),,,(iieeeeiiELfBRDFφθφθφθφθ==28Generic reflectance mapHorn, 198629Remember, this model doesn’t handle properly:• Occluding edges• Albedo changes• Perspective effects (small)• Interreflections• Material changes across surfaces in the image30Freeman, 199431Freeman, 199432Linear shading mapHorn, 198633Linear shading: 1storder terms of Lambertian shading2222111),(ssssqpqpqqppkqpR++++++=Lambertian point sourceqqqpRppqpRkqpqp 0,00,02),(),( ====∂∂+∂∂+≈1storder Taylor series about p=q=0)1( 2qqppkss++=See Pentland, IJCV vol. 1 no. 4, 1990.34Linear shading approximationrange imageLambertian shadingquadratic terms higher-order termslinear shadingPentland 1990, Adelson&Freeman, 199135Simplified lighting assumption• For simplicity, and without loss of generality, let’s consider light source from the left.• Then rendered image is proportional to the derivative of the range image along a row36Simplified linear shape from shading