1Randomly sample image points from a large set of natural outdoor images and determine the responses of the L, M and S cones to each such image point. This gives a three dimensional probability distribution: p(L,M,S). This distribution is non-Gaussian with high correlations. However, p(logL,logM,logS) is approximately Gaussian with high correlations. This distribution could be efficiently encoded by assigning the most bits to the 1stprinciple axis, the next most bits to the 2ndprinciple axis and the next most bits to the 3rdprinciple axis. These principle axes can be found using principle components analysis.2Principle components analysis (PCA). PCA can sometimes be useful for finding compact descriptions of data.3Ruderman principle components for cone receptor responses to rural (no man-made objects) natural images.4Distributions of the three principle components of the logarithms of the cone absorptions for natural images (dashed curves are Gaussian functions).5Wavelength tuning function of a neuron (M cell) in the LGN. DeValois (1965)6Wavelength tuning function of a neuron (P cell) in the LGN. DeValois (1965)7Wavelength tuning function of a neuron (P or K cell) in the LGN. DeValois (1965)8Opponent channels model of human color vision. This schematic illustrates what subjective percepts the channels supposedly represent.9Spectral sensitivities of the hypothesized color opponent mechanisms in the human visual system. These color opponent mechanisms (or ones very similar to these) are consistent with many aspects of human color vision performance.1011Psychophysical evidence for opponent processing model of color vision. Suppose a just detectable amount of red light produces the responses shown on the left for each color mechanism. Similarly a just detectable amount of blue-green light produces similar responses (but of opposite sign in the opponent channels). The prediction is that adding the just detectable red and green lights together should be undetectable because the responses in the red-green and yellow-blue mechanisms would be cancelled out. (The total response would be 0.3 which is less than the 0.4 produced by each light alone.) This kind of experiment (done by S. L. Guth) is very strong evidence for opponent processing. Other earlier evidence was obtained in hue cancellation experiments bu Hurvich and Jameson (see text).12Psychophysical evidence for opponent color channels in humans. (S.L. Guth)13A big part of the illumination problem is that the light reaching the eye from an object is greatly affected by the spectrum of the light falling on the object. I will come back to this a little at the end, but first we need to discuss the basics of color vision.14The light reaching the eye is the product of the illumination and the reflectance of the material; it can also depends on the position of the eye.1516A natural reflectance function (i) x a natural illuminant (ii) = a natural radiance function (iii).17Natural irradiance spectra for different conditions in a rainforest.18Natural daylight spectra.19Natural reflectance spectra.20Natural reflectance spectra of fruits and leaves in a monkey’s natural habitat. Notice that wavelengths beyond 650-700 nm are not really relevant because they fall outside the range of L cone.21(C) Best fit of first principle component to natural reflectance spectra.(B) Best fit of first two principle components to natural reflectance spectra.(A) Best fit of first three principle components to natural reflectance spectra.From Wandell, B. (1995) Foundations of Vision. Sinauer Press.22First three principle components for Krinov natural reflectance spectra.23Solving the color constancy problem involves (at least implicitly) recovering the reflectance of the surfaces and the illumination falling on the surfaces. There are several ideas about how the brain might accomplish this. They are based on assumptions about the properties of the environment. Generally, these have not been rigorously analyzed in natural scenes.24Maloney and Wandell (1986) analysis of the color constancy problem. The unknown parameters are the sigmas and the epsilons. They show that 4 photoreceptor types are sufficient under certain natural conditions to solve the constancy problem.252627Possible cost functions. MMSE (minimum mean squared error), MAP (maximum a posteriori), MLM (maximum local mass). The first two are commonly used but can have bad effects in some situations. MMSE penalizes more strongly as the magnitude of the error increases. MAP penalizes all errors equally. MLM penalizes errors more strongly as the magnitude of the error increases, but once the error gets above a certain magnitude all errors are equally bad.28Distribution of weights for a linear model (from PCA) of the “Munsell” paper surface reflectance functions. The shape of the basis functions are not shown here, but are similar to those of the Krinov spectra shown earlier. This constitutes a statistical model of natural reflectance functions. As similar analysis can be done for natural illuminant functions to obtain a statistical model of natural illuminants. Combining these statistical models with a loss function can be used to generate a Bayesian estimate of the illuminant and surface reflectance functions.29Solid curve is actual illuminate of a simulated scene containing patches of natural surfaces. The individual dashed curves show the estimated function for individual scenes (with different randomly selected patches of natural surface). (a) MLM cost (utility) function. (d) MAP. (f) Maloney & Wandell subspace model. Note that if the illuminate is correctly estimated then the reflectance functions can be identified quite accurately.30MLM estimates for other illuminants.31Steve Shevell’s demonstration of the effect spatial interactions on perceived color. It is one of many demonstrations showing that color perception is affected by the spatial context color and form. These mechanisms probably play an important role in “color constancy,” the ability of humans to judge the reflectance of a material fairly accurately independent of the color of the illumination. Another mechanism that contributes to the solution to illumination problem is separate light adaptation in each type of photoreceptor (this is called von Kries adaptation).One way to test what human understand about the spatial structure of images (e.g., surface structure) is to ask them to