Natural Scene Statistics and Perception W S Geisler 1 Some Important Visual Tasks Identification of objects and materials Navigation through the environment Estimation of motion trajectories and speeds Estimation of physical dimensions and shape Object manipulation Visual communication The first point I want to make is that the human visual system is designed to perform a number of important tasks It is designed to perform these tasks though evolution as well as though learning development during the life span I have listed some of these tasks in this slide These are very complex and difficult tasks that require complex neural machinery 2 Fundamental Premise The human visual system is the result of evolution by natural selection and hence its design must incorporate detailed knowledge of the physical regularities of the natural environment Now everyone who studies biological vision systems knows that this statement must be true Nonetheless vision scientists have only begun to take this fact seriously in recent years This premise suggests that a potentially powerful strategy for gaining insight into the human visual system is by measuring and analyzing the physical regularities of the natural environment The power of this approach is that if we can characterize the physical regularities in the visual environment then we will gain insight into how those regularities could be exploited to perform visual tasks Those insights in turn allow us to know what to look for in the brain and how to interpret what we find For example to understand how the brain can create 3D representations from 2D retinal images it is crucial to understand the regularities in the retinal image produced by perspective projection of natural scenes Because of the complexity of the natural environment and the loss of information due to projection onto a 2D retina the physical regularities relevant for vision are best described in statistical terms and the best way to understand what those statistical properties imply for vision is with statistical analysis 3 Two General Kinds of Natural Scene Statistics It is useful to distinguish between two general types of natural scene statistics which can be measured at various levels along the pathway from environment to behavior Within domain statistics are useful for understanding coding and representation However they say nothing about the relationship between the different levels and hence they are not as useful for understanding the information relevant for specific tasks In most natural tasks the goal is to use the retinal image or some neural representation of the retinal image to make inferences about properties of the physical environment In other words the statistics one needs to know is the joint probability of the environment and image properties These are what I am calling across domain statistics 4 5 Two Examples one if I run out of time Contour Perception Fixation Selection I will describe two recent examples from my lab of measuring physical properties of the environment and analyzing their relationship to the design of the human visual system one example concerns contour perception and the other gaze selection 6 Humans have a remarkably ability to see meaningful non accidental structure in images they have never seen before Much of this structure is contained in the contours created by object shadow lighting and material boundaries We have been interested in measuring the statistical properties of these contours in natural scenes and in trying to understand the relationship between those properties and human ability to see structure in images One area of research has been to examine the 2D geometrical relationships between local edge elements extracted from natural scenes 7 Contour Completion Task Do contour elements intersecting an occluding surface belong to the same or different contour same contour different contour s distance direction orientation contrast polarity We have used these statistics to characterize the information available to support performance of certain simple tasks such as the contour completion and contour grouping 8 Measuring Bayesian Contour Statistics Each red pixel in the right image is a edge element location The orientation of each element was measured but is not shown here Two observers then assigned edge elements to physical contours sources observers regarded boundary contours lighting contours and surface marking contours as distinct This assignment information was assumed to provide approximate ground truth 9 Decimated edge samples with orientation shown 10 1 0 L s 1 p s c p s c p d c p d c The geometrical and contrast polarity relationship between two edge elements is given by 4 parameters Once images are hand segmented it is straight forward to estimate the likelihood and prior probability distributions In the specific task we consider next the prior probabilities are forced to be equal so the relevant function is the likelihood ratio distribution which is shown on the right The reference is in the middle distance is given by the ring direction by the angle around the ring orientation difference by the orientation of the plotted line segment polarity by the particular half of the diagram and likelihood ratio by the color of the plotted line segment For an earlier version of this analysis without contrast polarity see Geisler Perry Super Gallogly 2001 Vision Research 41 711 724 These average Bayesian pair wise statistics make it to possible to determine optimal performance in the contour completion task 11 In the contour completion task a pair of edge elements is selected at random from a natural image and an occluder is placed between them The task the display is shown in B is to indicate whether the pair of elements is from the same or different physical contour where the prior probability is 0 5 Three occluder diameters No feedback is given for the first 600 trials then 600 trials with feedback then 600 trials with no feedback 12 Comparison of human symbols and ideal solid curves performance with green and without red contrast polarity information Human efficiency is high and parallel to ideal Average data for four observers two experienced two na ve 13 Ideal Distributions of hits misses false alarms and correct rejections for ideal in the contour completion task Plotting conventions the same as in the plot of pair wise contour statistics presented earlier except here the line segments are vectors covering the range
View Full Document
Unlocking...