Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 4: Ideal Observer Analysis‡Initialize standard library files:In[2]:=H<< "BarCharts`"; << "Histograms`"; << "PieCharts`"L;Off@General::"spell1"D;GoalsOne of the basic take-home messages of this course is that the accuracy and reliability of perceptual decisions are limited by two primary sources: 1) inherent uncertainty in the stimulus information for a specific task2) limitations of the human observer.This is a critical distinction when drawing conclusions about the underlying neural mechanisms of the brain from behavioral/psychophysical data. The pattern of errors that a human observer shows when detecting patterns, discriminating vernier targets, finding targets in clutter, recognizing objects, reaching for objects, could in principle largely reflect the uncertainty in the task itself. If this is the case, then our simplest conclusion about the neural mechanism is that it behaves like an ideal observer, i.e. as a very efficient utilizer of the information available. On the one hand the ideal observer provides a good model of human perceptual behavior; on the other, it limits our ability to draw conclusions about the neural mechanisms, which are often best revealed by sub-ideal behavior. Human perceptual performance is near optimal for some tasks, and not for other tasks. Historically Hecht et al. and later Barlow and others used this comparison to good advantage. Hecht et al. argued that the variation in the proportion of hits was largely due to photon fluctations, with only smaller contributions from limitations of the human observer, suggesting that the variability was due to a high efficiency of photon transduction.We'd like to further develop our tools of signal detection theory, and extend them to perceptual decisions more generally, so that we can quantitatively compare humans to ideal observers. We call this comparison ideal observer analysis.Develop tools to analyze the information available in a task using ideal observer theory: ‡From dots to patterns: What does the eye see best?‡From 2D image discrimination to object perception: How should we formulate object perception tasks?Last time‡External variability and the Ideal Observer for light discriminationIntroduced the idea of an ideal observer that models "external variability" and optimal decisions given that variability or uncertainty. We stated that the ideal observer should choose the hypothesis (e.g. the switch setting in our prototypical light discrimination task) with the highest posterior probability, given the data (photon count). This decision process is an example of Bayesian inference. Bayesian theories of vision provide a quantitative model of the information available in a task. From there, we can test how well other observers (human, animal, or neuro) compare. The ideal can be used as a benchmark to measure the performance of humans, machines, and neurons for more complicated problems like pattern detection.I asked you to accept (for the moment) that the Bayesian maximum a posteriori (MAP) decision rule is optimal in the sense of minimizing the total probability of error. Later we will prove this.Today2 4_IdealObserverAnalysis.nbTodayIn this lecture we complete our introduction to classical signal detection theory (SDT). SDT provides an important set of tools for measuring and modeling the sensitivity of human and neural perceptual decisions. (Later we'll generalize further to "statistical decision theory"--same acronym!) We will: o Understand how to summarize ideal (and human performance) in the yes/no task in terms of hit and false alarm rates, and to relate these to a sensitivity measure called d'. To do this, we will introduce the (standard) Gaussian approximation, and apply it to variability in light levels.o Introduce other tasks. In particular, the two-alternative forced-choice task o Understand how to quantitatively compare human and ideal performance. o Measure your own statistical efficiency in a 2AFC task The standard Gaussian approximationMost inference modeling is done using Gaussian models of variability. One reason is theoretical convenience. A deeper theoretical reason rests on the Central Limit Theorem, which says that a sum of independently drawn random variables (from a non-Gaussian distribution) looks more and more Gaussian the more elements that are in the sum. Empirically, many experiments on human signal detection have been well-fit by assuming Gaussian distributions. However, as we will see later (when we measure statistics on natural images), the Gaussian assumption/approximation for some random variables is a bad approximation. It is always important to test this assumption. We'll first show that the Gaussian approxi-mation provides a good approximation to the Poisson distribution.Some terminology. We've adopted the convention of treating the high light as a "signal". Similarly, we can think of the chance low switch settings as "noise" to be take into account. We will continue with this here, and begin to use the terms "signal" and "noise". But remember that this is just a convention--the problem is symmetric, and we could be talking about whether a measurement is from hypothesis A vs. hypothesis B.Gaussian approximation for signal and noise distributionsAs the mean a gets large, the frequency of occurrance of a Poisson distributed random variable can be approximated very well by the Gaussian distribution:p HX = xL =‰-Hx-mL22 s22 p s. The mean or expectation of X is : E HXL = m,and the variance is : var HXL = s2This approximation is useful to estimate probability values for large a. If a is large enough, the probability of negative values (which is meaningless for a Poisson distribution) is very small. For computational convenience and for later generality, we will usually use the Gaussian approximation. 4_IdealObserverAnalysis.nb 3This approximation is useful to estimate probability values for large a. If a is large enough, the probability of negative values (which is meaningless for a Poisson distribution) is very small. For computational convenience and for later generality, we will usually use the Gaussian approximation. Let's compare the forms of the Poisson and Gaussian distributions:ManipulateBndist = NormalDistributionBmean, mean F;pdist = PoissonDistribution@meanD;p1 = Table@PDF@pdist, xD, 8x, -5, 50<D;g1 = ListPlot@p1, PlotStyle Ø RGBColor@1, 0, 0DD;p2 = Table@PDF@ndist, xD, 8x, -5, 50<D;g2 =