Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 6‡Initialize standard library files:In[1]:=<< "BarCharts`"; << "Histograms`";Off@General::spell1D;GoalsLast timeDeveloped signal detection theory for characterizing an ideal observer for detecting a "known" pattern in additive gaussian noise.The statistical treatment is a special case of Bayesian inference.Showed how human and ideal performance can be quantitatively compared by their respective sensitivities, d's.This timeExtend the tools of signal detection theory to "pattern inference theory", to begin to deal with more complex tasks.How to manage complex pattern inference tasks?‡Two main observations for simplification: Graphical models of influenceTask dependence: Bayesian inference theory -> Bayesian decision theory, to take into account what information is impor-tant and what is not. I.e. what is signal and what is noise.(Related to "integrating out", below.)Graphical Models of dependenceGraphical Models of dependenceThe generative model in the previous lecture was simple. The signals were two fixed images (e.g. a sinusoidal grating and a uniform image), and the image variability was solely due to additive noise. What about natural images? The "universe" of possible factors generating an image could be expressed by constructing the joint probability on all possible combinations of descriptions:p(scene,object class, environment lighting,object reflectivity, object shape, global features, local features, haptic)where each of the variable classes is itself a high-dimensional description, but this is hopelessly large, because of the combinatorial problem. Natural images are complex, and in general it is difficult and often impractical to build a detailed quantitative generative model.But natural images do have regularities, and we can get insight into the problem by considering how various factors might produce natural images.One way to begin simplifying the problem is to note that not all variables have a direct influence on each other. So draw a graph in which lines only connect variables that influence each other. In particular, we will use directed graphs to repre-sent conditional probabilities.2 6_BayesDecisionTheory.nbGraphs: causal structure and conditional independenceThe idea is that natural image pattern formation is specified by a high-dimensional joint probability, requiring an elabora-tion of the causal structure that is more complex than the simple SDT model. We represent the probabilistic structure of the joint distribution P(S,L,I) by a Bayes net (e.g. Ripley, 1996}, which is a graphical model that expresses how (random) variables influence each other. There are three basic building blocks: converging, diverging, and intermediate nodes. For example, multiple (e.g. scene) variables causing a given image measurement, a single variable producing multiple image measurements, or a cause indirectly influencing an image measurement through an intermediate variable. These types of influence provide a first step towards modeling the joint distribution and the means to compute probabilities of the unknown variables given known values.Components of the generative structure for image patterns involve converging, diverging,and intermediate nodes. For example,these could correspond to:multiple (scene) causes {shape S1, illumination S2 giving rise to the same image measurement, I ; one cause, S influencing more than one image measurement, {color, I1, brightness, I2}; a scene (or other) cause S, {object identity, S} influencing an image measurement (image contour) through an intermediate variable L (3D shape) .A basic rule of probability is the product rule, in which the joint probability p(A,B) = p(A|B)p(B) (see Probability-Overview.nb).The arrows above is a graphical shorthand that tells us how to factor a joint probability into conditionals. So for the three examples above, we have:p(S1,S2,I)=p(I|S1,S2)p(S1)p(S2)p(S,I1,I2)=p(I1|S)p(I2|S)p(S)p(S,L,I)=p(I|L)p(L|S)p(S)Basic rules: Condition on what is known, and integrate out what you don't care about‡Condition on what is known:Given a scene description S = {S1,...,SN}, and image features I={I1,...,IM}, the "universe" of possibilities is:(1)p HS, ILIf we know (i.e. the visual system has measured some image feature I), the joint can be turned into a conditional (posterior):6_BayesDecisionTheory.nb 3(2)p HS »IL = p HS, ILêp HIL‡Integrate out what you don't care aboutWe don't care to estimate the noise (or other generic, "nuisance", or "secondary" variables):(3)p ISsignal»IM =‚Snoisep ISsignal, Snoise»IM,or if continuous =‡Snoisep ISsignal, Snoise»IM „ SnoiseCalled "integrating out" or "marginalization". For example, suppose I want to calculate the ideal observer for recognizing one of 6 objects, but each object could appear in one of 12 "poses". A "pose" means a specific position relative to the camera. I'd want to set up my problem so that I can integrate over the poses, to effectively discount that source of variation. In other words I really don't want a precise estimate of the pose parameters, but I do want to be as accurate as possible in deciding which object I've seen. This kind of ideal observer analysis of human object recognition was done by Tjan et al. in 1995.Graphical models and general inference‡Three types of nodes in a graphical model: known, unknown to be estimated, unknown to be integrated out (marginalized)We have three basic states for nodes in a graphical model: -- known-- unknown to be estimated-- unknown to be integrated out (marginalization). We have causal state of the world S, that gets mapped to some image data I, perhaps through some intermediate parameters L, i.e. S->L->I. So for example, face identity S determines facial shape L, which in turn determines the image I itself. Consider three very broad types of task:‡Image data inference: synthesis Image synthesis (forward, generative model): We want to model I through p(I|S). In our example, we want to specify "Bill", and then p(I|S="Bill") can be implemented as an algorithm to spit out images of Bill. If there is an intermediate variable, L, it gets integrated out.4 6_BayesDecisionTheory.nb‡Hypothesis ("inverse") inferenceHypothesis inference: we want to model samples for S: p(S|I). Given an image,we want to spit out likely object identities,so that we can minimize risk, or for example, do MAP classification for accurate object identification. Again there is an