104Object Recognition-Segregation of function-Visual hierarchy-What and where (ventral and dorsal streams)-Single cell coding and ensemble coding-Distributed representations of object categories-Face recognition-Object recognition as a computational problemHigher Perceptual Functions105Segregation of function exists already in theearly visual system:M channel (magnocellular): from M-type retinalganglion cells to magnocellular LGN layers to layer IVBof V1; wavelength-insensitive in LGN, orientationselectivity in V1 (“simple cells”), binocularity anddirection selectivity in layer IVB; processing visualmotion.P channel (parvocellular): from P-type retinal ganglioncells to parvocellular LGN layers to interblob regions oflayer III in V1; many cells in LGN show coloropponency, cells in interblob regions of V1 have strongorientation selectivity and binocularity (“complexcells”), channel is also called P-IB; processing visualobject shape.Functional Segregation106Segregation of function can also be found atthe cortical level:- within each area: cells form distinct columns.- multiple areas form the visual hierarchy …Functional Segregation107The Visual Hierarchyvan Essen and Maunsell, 1983108van Essen et al., 1990The Visual Hierarchy109-functional segregation of visual features into separate(specialized) areas.-increased complexity and specificity of neuralresponses.- columnar groupings, horizontal integration withineach area.-larger receptive fields at higher levels.-visual topography is less clearly defined at higherlevels, or disappears altogether.-longer response latencies at higher levels.- large number of pathways linking each segregatedarea to other areas.- existence of feedforward, as well as lateral andfeedback connections between hierarchical levels.The Visual Hierarchy110The Architecture of Visual CortexMishkin and Ungerleider, 1983Lesion studies in the macaque monkey suggest that there aretwo large-scale cortical streams of visual processing:Dorsal stream (“where”)Ventral stream (“what”)111What and WhereMishkin and Ungerleider, 1983Object discrimination taskLandmark discrimination taskBilateral lesion of the temporallobe leads to a behavioral deficitin a task that requires thediscrimination of objects.Bilateral lesion of the parietallobe leads to a behavioral deficitin a task that requires thediscrimination of locations(landmarks).112The Architecture of Visual CortexmotioncolorformLateral views of the macaque monkey brain113Single Cells and RecognitionWhat is the cellular basis for visual recognition (visuallong-term memory)?1. Where are the cellular representationslocalized?2. What processes generate theserepresentations?3. What underlies their reactivation during recalland recognition?114Single Cells and RecognitionVisual recognition involves the inferior temporal cortex(multiple areas). These areas are part of a distributednetwork and are subject to both bottom-up (feature driven)and top-down (memory driven) influences.Miyashita and Hayashi, 2000115Single Cells and RecognitionCharacteristics of neural responses in IT:1. Object-specific (tuned to object class), selectivefor general object features (e.g. shape)2. Non-topographic (large RF)3. Long-lasting (100’s ms)Columnar organization (“object feature columns”)Specificity has often rather broad range(distributed response pattern)116Distributed RepresentationsAre there specific, dedicated modules (or cells) foreach and every object category?No. – Why not?117Distributed RepresentationsEvidence → feature based and widely distributedrepresentation of objects across (ventral) temporalcortex.What is a distributed representation?118Distributed RepresentationsExperiments conducted by Ishai et al.:Experiment 1:1. fMRI during passive viewing2. fMRI during delayed match-to-sampleExperiment 2:1. fMRI during delayed match-to-sample withphotographs2. fMRI during delayed match-to-sample with linedrawingsThree categories: houses, faces, chairs.119Distributed RepresentationsFindings:Experiment 1:Consistent topography in areas that most stronglyrespond to each of the three categories.Modules?No - Responses are distributed (more so for non-facestimuli)Experiment 2:Are low-level features (spatial frequency, texture etc.)responsible for the representation?No – line drawings elicit similar distributions of responses120Distributed RepresentationsFrom Ishai et al., 1999121Distributed RepresentationsFrom Ishai et al., 1999housesfaceschairs122Face recognition achieves a very high level ofspecificity – hundreds, if not thousands ofindividual faces can be recognized.Face RecognitionVisual agnosia specific to faces: prosopagnosia.High specificity of face cells → “gnostic units”,“grandmother cells”Many face cells respond to faces only – andshow very little response to other object stimuli.123Face RecognitionTypical neural responses in the primate inferior temporalcortex:Desimone et al., 1984124Face RecognitionFace cells (typically) do not respond to:1. “jumbled” faces2. “partial” faces3. “single components” of faces (although someface-component cells have been found)4. other “significant” stimuliFace cells (typically) do respond to:1. faces anywhere in a large bilateral visual field2. faces with “reduced” feature content (e.g. b/w,low contrast)Face cell responses can vary with: facialexpression, view-orientation125Face RecognitionFace cells are (to a significant extent) anatomicallysegregated from other cells selective forobjects. They are found in multiple subdivisionsacross the inferior temporal cortex (in particularin or near the superior temporal sulcus)126Face RecognitionFaces versus objects in a recent fMRI study (Halgren etal. 1999)127Object Recognition:Why is it a Hard Problem?Objects can be recognized over huge variations inappearance and context!Ability to recognize objects in a great number ofdifferent ways:object constancy (stimulus equivalence)Sources of variability:- Object position/orientation- Viewer position/orientation- Illumination (wavelength/brightness)- Groupings and context- Occlusion/partial views128Object Recognition:Why is it a Hard Problem?Examples for variability:field of viewTranslation invarianceRotation invariance129Object Recognition:Why is it a Hard Problem?More examples for variability:field of viewSize invarianceColor130Object Recognition:Why is it a Hard Problem?Variability in visual scenes:field of viewPartial occlusionand presence of other