Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 2: Limits to VisionIntroduction‡Last timeLast time, we got an overview of we are going. We'd like to understand how we "know what is where just by looking", how we infer "states of the world", such as scene and object parameters, and how vision enables useful actions. How to get there? We saw the need for an interdisciplinary approach, and analysis at several different levels. Computa-tional theory provides a level of analysis that can produce a quantitative account of behavior, and helps to bridge our understanding of how the neural mechanisms of vision enable that behavior. The goal of computational theory is to understand the problems of vision at a level that enables us to specific a procedure to obtain quantitative solutions to estimating "scenes" or "states of the world" S from image input information I. From there we can relate this knowledge to human behavior and underlying neural processes. We saw that the same object can give rise to an infinite variety of images, and a given image could potentially have come from an infinite variety of objects. To make the problem concrete, think of an object property, say the shape of a face. What do we mean by "shape"? We'll return to this later in the course, but for the moment assume that "shape" is a geometrical description, like a contour map. The shape could belong to one of two people. The image of a face is light intensity, I(x,y), where (x,y) are coordinates in retinal space. As we saw in the last lecture, intensities are not related to shape in any simple fashion. In fact, we could say that the shape a signal which is encrypted in the image. Further, the image can vary because of differing lighting conditions N = {illumination1, illumination 2, ...}. If you are trying to recognize the face, these variations in illumination constitute a form of "noise". So the image is an encrypted and noisy function (f) of the shape signal: I(x,y) = f(shape S, illumination N). The goal is to recover the signal as best we can. Call the recovered signal S'.This example underscores the importance of understanding how image information is generated, i.e. characterzing the "generative model". The example also suggestst that we might get some mileage from thinking about the computational problem in terms of "communication"--image measurements I contain information about a signal S that the world is "sending" us (e.g. scene variable such as object shape, size, or brightness) but the signal is encrypted and muddled by unwanted variability N. This kind of problem has been studied a lot over the past sixty years or so by communication engineers and psychologists, and is called the Signal Detection Theory (SDT).This example underscores the importance of understanding how image information is generated, i.e. characterzing the "generative model". The example also suggestst that we might get some mileage from thinking about the computational problem in terms of "communication"--image measurements I contain information about a signal S that the world is "sending" us (e.g. scene variable such as object shape, size, or brightness) but the signal is encrypted and muddled by unwanted variability N. This kind of problem has been studied a lot over the past sixty years or so by communication engineers and psychologists, and is called the Signal Detection Theory (SDT).In general, your eye's retinal image of a signal (say your friend's face) is never the same from one time to the next because of changes in illumination, viewpoint, makeup, hair, or even age. But this is very complicated to analyze, so let's start off with perhaps the simplest case of variability in vision--variations in the intensity of a single spot of light. ‡VariabilityImagine you have a flashlight with a high and low beam setting, and you want to send a signal (S = high or low) to a distant friend by flashing a light with either the high or the low beam on (e.g. "high if by land, low if by sea"). In a perfect world,, this would be a one bit communication system. But the world isn't perfect and suppose the flash light is a bit flaky. The result is that occasionally the amount light emitted with the high beam is lower than for the low beam. Your friend won't be able to reliably tell whether a flash corresponded to the high or the low setting. You'd be stuck with a communica-tion system that on average transmits less than one bit of information.Now suppose you work really hard to make the most reliable flashlight possible. Physics has shown that no matter what you do, you'll always be a bit short of the ideal one bit information capacity. Normally you wouldn't notice, because you could make the error rate very tiny by increasing the intensity difference between the high and low settings, but if the intensity differences for the two switch settings are small, you'd always find some residual variability due to photon fluctuations. The generative model is illustrated by the graph with the solid arrows in the figure below.2 2_LimitsToVision.nb‡DiscriminationWe are going to ask the psychophysical question: How well can one discriminate a slight change in intensity of a spot of light given that from trial to trial the number of photons landing in the eye (or transduced by the retina) is not consistently the same? This is the problem of intensity discrimination, and we'll develop a model to account for human discrimination thresholds given variability. Intensity discrimination can be viewed as an "inverse" inference problem--the dashed line in the above figure (Is S' = "high" or "low"?. We'll see how photon fluctuations fundamentally limit human discrimination thresholds (also called difference thresholds).‡DetectionA special case of discrimination is detection, where we are interested in the transition from invisible to visible. How well you can tell whether there was a flash of light or not? On the best of nights when the sky is clear and moonless, you can see about 2000 stars. Why can't you see more? There are millions more there, each emitting photons that land on earth, but they are just too dim for us to see. In other words, what are the limits to just detecting a faint point of light in the dark? This question was addressed in a classic study by Hecht, Schlaer and Pirenne in what may be the first study that combined psychophysics, neuroscience, and what we would call a computational theory in vision. Hecht, Schlaer and