Introduction to Neural NetworksU. Minn. Psy 5038Representation of visual informationNeural codes and representationOverview‡Final projects‡More on representation--tie with V1 orientation selectivity‡More on representation--contingent adaptation--tie with decorrelating neural networksExtra-striate cortical visual areasV1 plus extra-striate areas, ~40-50% of primate cortex.We've noted that cells in the primary visual cortical area send their visual information to an incredibly complex, and yet structured collection of extra-striate areas.‡Connectivity between areasFeedforward (ascending pathways) from mainly superficial layers (I,II, III) to layer IV of receiving areas. More diffuse feedback (descending pathways) from outside IV back to layers also outside IV (mainly I or VI). (~cm)But there are also feedforward/feedback local circuits at a finer grain, within columns (~mm)‡Two main large-scale functional pathways (involving mutliple areas and area-area connections)What kind of information is represented in these areas? Any hypothesized function of striate cortex must eventually take into account what the information is to be used for. Two primary functions of vision are: object perception and recognition--"within object" processing, and spatial processing (between object, and view-object relations). "Where" vs. "What" ‡Occipital-parietal pathway ("dorsal stream"): Spatial judgments, action ("where"/"how")?V1,MT,MST,LIP,...Viewer-centered computations"where" or "how" or "now"?Spatial computations,such as coordinate transformations for action‡Occipital-temporal pathway ("ventral stream"): Object perception, recognition ("what")?object perception, recognition pathway V1, V2, V4, Posterior IT, Anterior IT, ... view-independent, object-intrinsic, computationsReceptive field properties: Size grows: V1 = .5°-2°; TE (mean) = 26°Response properties become more difficult to characterize. Earlier RF responses often show approximate additivity--response is determined by the combination of responses to shape and orientation of parts or features. Neural RFs in later areas (TE), more difficult to characterize. Respond to highly specific features, sometimes interpretable (faces), sometimes more abstract ("toilet brush-like thing").Some information has to be discounted, and other information selected that is diagnostic for accurate classifiction. Invariances required for recognition: photometric: illumination level, direction, shadows geometrical: translation, size, orientation in depth category-related: levels of abstraction Combinatorial problem of object representation and classification grandmother cells, distributed codes, sparse codes‡Behavioral evidence for distinction between ventral and dorsal pathway functions2 Lect_16_VisualRepCode.nb‡Behavioral evidence for distinction between ventral and dorsal pathway functionsBehavioral evidence: Double dissociation of function.MonkeysIT Lesions (Mishkin) showed impaired ability in matching tasks (non-match-to-sample task) but can dospatial task.Posterior parietal lesions: impaired ability to perform spatial task but can do matchingtask.Human patients.Milner & Goodale's Patient D. F. (agnosic): Motor competence, good acuity, color, and motion perception. Normal intelligence and language comprehension.But can't recognize objects or faces. Further, DF cannot judge orientation or size. Motor interactions are close to normal. She can "post" a letter-like object into a slot, adjust grasp size correctly before touching the object.How can she “post” letter into an arbitrarily oriented slot with her hand but cannot tell you in advance whether the orientation of the slot is vertical or horizontal!(Other patients have been studied that show the opposite pattern of deficits)Neural codes and representation: Demonstration of orientation adaptationLet's return to the problem of representation at the level of quasi-homogeneous population of neurons, such as the collec-tion of simple cells making up a hypercolumn or a collection of hypercolomns in V1. What might be the relationship between a perceptual judgment of a stimulus property (like orientation) and the receptive field properties of neurons in V1? To motivate this problem and to introduce concepts of coarse coding, population or distributed codes, etc. (below), let's make a demo to study a well-known illusion involving adaptation.Make stimuliIn[1]:=width = 64; grating[x_,y_,xfreq_,yfreq_] := Cos[(2. Pi)*(xfreq*x + yfreq*y)];Lect_16_VisualRepCode.nb 3‡Left-slanted adapting gratingIn[136]:=xfreq = 4; theta = 0.8 * Pi ê2;yfreq = xfreq êTan@thetaD;gleft = DensityPlot@grating@x, y, xfreq, yfreqD, 8x, 0, 1<, 8y, 0, 1<,PlotPoints Ø 64, Mesh Ø False, Frame Ø False,ColorFunction Ø "GrayTones"D;‡Right-slanted adapting gratingIn[25]:=xfreq = 4; theta = 1.2 * Pi ê2;yfreq = xfreq êTan@thetaD;gright = DensityPlot@grating@x, y, xfreq, yfreqD, 8x, 0, 1<,8y, 0, 1<, PlotPoints Ø 64, Mesh Ø False, Frame Ø False,ColorFunction Ø "GrayTones"D;‡Vertical test gratingIn[29]:=xfreq = 4; theta = Pi ê2;yfreq = xfreq êTan@thetaD;gvertical = DensityPlot@grating@x, y, xfreq, yfreqD, 8x, 0, 1<,8y, 0, 1<, PlotPoints Ø 64, Mesh Ø False, Frame Ø False,ColorFunction Ø "GrayTones"D;‡Gray fixation barIn[23]:=gbar = [email protected]`D, Rectangle@80, 0.45`<, 81, 0.5`<D<,AspectRatio Ø14F;4 Lect_16_VisualRepCode.nbTest: Try itIn[139]:=Show@GraphicsGrid@88gleft, gvertical<, 8gbar, gbar<,8gright, gvertical<<DDOut[139]=Lect_16_VisualRepCode.nb 5What happens if you adapt with the left eye and test with the right eye?Can this effect be explained in terms of changes to neurons in the retina? LGN?Neural codes and representationCan we explain orientation adaptation in terms of neural networks? To do this, we have to grapple with several questions: What are the "languages" of the brain? How is information represented? What is the information in a train of action potentials?First some background concepts.Firing rateIn sensory systems, firing rate often correlates well with both subjective and physical "intensity". What does the train of action potentials mean elsewhere in the brain?"Labeled lines"Suppose that when a particular cell fires it means something in particular, i.e. when neuron S fires at a rate of f spikes per second, then the animal must be looking at and recognizing its grandmother. Or when neuron T fires at a rate of g spikes per second, then the animal must be seeing a