Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 14: Intro to Scenes from ImagesInitializeOutlineLast time: Continue with discussion of the two views of the function of early visual coding‡Oriented filters: efficient coding vs. Edge/bar detection--Efficient coding means fewer bits required to encode imageExamples: PCA->dimension reduction->quantization. Decorrelates filter outputs. Filters localized in space and spatial frequency do too (e.g. wavelets).Sparseness--high kurtosis histograms for filter outputs--Edge/bar detection: local image measurements that correlate well with useful surface properties‡Problems with edge detectionNoise & scaleVarious scene causes can give rise to identical image intensity gradients--no local information to "disambiguate" an edgeToday‡Next homework‡Mathematica Demonstrations‡Mathematica Demonstrations Illusions‡Extrastriate cortex--overview‡Scenes from images, scene-based modeling of imagesOverview of extrastriate cortexWe've seen how to model the processing of spatial visual information in V1. Thirty years ago, one might have thought that a thorough understanding of primary visual cortex would produce a thorough understanding of visual perception. Not so. Since then, neurophysiologists have shown that primate visual processing has only just begun in V1. Much of this work is based on studies of the macaque monkey, but in the past decade and half, scientists have used brain imaging techniques to distinguish visual areas in the human cortex. ‡Extra-striate cortexPrimary visual cortex sends visual information to many other visually sensitive cortical areas (current estimates are about 30 visual areas in the macaque). These areas have been identified through anatomical, histological, and physiological techniques with the early work by Samuel Zeki at the University of London, and David Van Essen and colleagues. Areas have been delineated by:Function: physiology, neurons in different brain areas selective for different aspects of their inputsArchitecture: cytoarchitecture (e.g. cell size,cell density, density of axons, layering, discov-ered using different kinds of stains).Connections: anatomical connections traced using retrograde and anterograde tracers.Topography: retinotopic maps in each of several of the early visual areas (V1-V8).Primary visual cortex has a fairly precise topographic map of the visual field--nearby points in the image map to nearby cells in V1. Other areas have less precise topographic maps of the visual field. 2 14.ScenesfromImages.nbFrom Van Essen et al. 199214.ScenesfromImages.nb 3‡Human visual areasFrom Scientific American, 4 14.ScenesfromImages.nb‡Visual hierarchyOne of the remarkable discoveries about extra-striate cortex is that these areas are organized hierarchically (See Felleman and Van Essen, 1991; DeYoe and Van Essen, 1988; DeYoe et al., 1994), and involve multiple parallel pathways. A general pattern of connectivity between areas has emerged in which one sees:• feedforward connections from superficial layers (I, II, III) to IV• feedback connections originating in deep (V, VI) and superficial layers and terminating in and outside layer IV.14.ScenesfromImages.nb 5Functions?What are these extra-striate visual areas of cortex doing? At a general level, these areas turn image information into useful behavior, such as recognition, visuo-motor control, and navigation. Below we outline current views on two large-scale functional pathways. But it is also important to begin to look for detailed computations that extra-striate areas are doing. At the current time, we have only a few ideas, some of which we will look at in the lectures on motion perception.For example, the very large receptive fields found in extra-striate areas (e.g. MT cells can have receptive fields as large as 100 deg!) bring together information from distant parts of the visual field. One idea is that information which likely belongs to same object, or have the same cause, is what is being brought together. A few problems are: stereovisionmotion disambiguationcolor constancyobject contours & regionsSome indication of possible functional distinctions are illustrated below for smaller scale pathways.‡Large scale functional pathwaysThe flow of visual information follows two dominant streams. In the dorsal or parietal stream, information flows from primary cortex to parietal cortex. A substream that has been studied for motion processing is: V1 <-> MT <-> MST. The temporal stream carries information from primary visual cortex to infero-temporal cortex. A sub-stream which has been studies for object recognition is: V1 <-> V2 <-> V4 <-> IT. ‡Dominant functional streams6 14.ScenesfromImages.nb‡Dominant functional streamsBased on studies of the behavior of monkeys and man with lesions, and work using electrophysiological techniques, it is thought that the parietal stream has to do with navigation, and view-centered representations of the visual world. It is sometimes called the "where" system (Mishkin and Ungerleider, 1983). Although it may more to do with "how" (Goodale & Milner 1992).The temporal stream is sometimes called the "what" system. It is believed to be important for non-viewer centered represen-tations useful for object recognition. Form and color of objects is thought to be extracted by interacting modules in the temporal stream.Current working hypotheses regarding function:dorsal / parietal areas: e.g. V1 -> MT -> MST"where out there?"navigation, viewer centered representationmotion for layout, heading (MST)...and for driving motor actions such as reachingtemporal: e.g. V1 -> V2 -> V4"what is it?"processing for non-viewer or object-centered representationmaterial color and shape & form...and further downstream, temporal areas (IT) for object recognition14.ScenesfromImages.nb 7‡Smaller scale pathways: >Fig. 3. Schematic diagram of anatomical connections and neuronal selectivities of early visual areas in the macaque monkey. LGN = lateral geniculate nucleus (parvocellular and magnocellular divisions). Divisions of Vl and V2: blob = cytochrome oxidase blob regions; interblob = cytochrome oxidase--poor regions surrounding the blobs; 4B = lamina 4B; thin = thin (narrow) cytochrome oxidase strips; interstripe = cytochrome oxidase-poor regions between the thin and thick strips; thick = thick (wide) cytochrome oxidasestrips; V3 = visualarea 3; V4 = visualarea(s) 4; MT = middle temporal area. Areas V2,