Prof. Greg Francis1PSY 310: Sensory and Perceptual ProcessesPurdue UniversityNeural codesPSY 310Greg FrancisLecture 12Is 100 billion neurons enough?Purdue UniversityCOC illusion The COC illusion looks like real squares because the neuralresponses are similarTrue squaresCOC squaresGanglion cell responsesPurdue UniversityVentral stream From visual cortex to the temporal lobe Involved in recognizing or identifying objectsPurdue UniversityCritical features Cells in inferotemporal cortex respond to complex features instimuli Not easy to measure neurophysiologically Why such bizarre patterns?Purdue UniversityReal stimuli Following the ideas about the COC stimulus, ourperceptual experience is determined by the neuralresponses to stimuli Suppose you see this picture of a catPurdue UniversityCell responses This IT cell might respond strongly And the other cells hardly at allProf. Greg Francis2PSY 310: Sensory and Perceptual ProcessesPurdue UniversityCell responses One might be tempted to suggest that this cell“codes” the perceptual experience of the cat But this would not be correct This cell would respond similarly to many differenttypes of stimuli But our perceptual experience is quite different!Purdue UniversityNeurophysiology and perception Does this mean that our perceptual experience isnot determined by neural responses? After all similar neural responses should give rise tosimilar percepts No. Similarity between ganglion responses andperceptual experience works for the ganglion cellsbecause of their anatomical location Everything else in the visual system is based on theirresponsesPurdue UniversityNeurophysiology and perception The rest of visual perception does not depend onthe responses of this IT cellPurdue UniversityNeurophysiology and perception The rest of visual perception does not depend on theresponses of this IT cell In fact, probably thousands of IT cells respond to thisstimulusPurdue UniversityNeurophysiology and perception The rest of visual perception does not depend on theresponses of this IT cell In fact, probably thousands of IT cells respond to thisstimulus Millions of ganglion cellsPurdue UniversityNeurophysiology and perception The rest of visual perception does not depend on theresponses of this IT cell In fact, probably thousands of IT cells respond to thisstimulus Millions of ganglion cells Millions of orientation sensitive cellsProf. Greg Francis3PSY 310: Sensory and Perceptual ProcessesPurdue UniversityNeurophysiology and perception The rest of visual perception does not depend on theresponses of this IT cell In fact, probably thousands of IT cells respond to thisstimulus Millions of ganglion cells Millions of orientation sensitive cells Dorsal streamPurdue UniversityRepresentation of objects Objects are unlikely to be represented by a single neuron Consider facesPurdue UniversityRepresentation of objects Objects are unlikely to be represented by a single neuron Consider facesWhat do you do here?Purdue UniversityRepresentation of objects Objects are unlikely to be represented by a single neuron Consider facesWhat do you do here?What do you do withdifferent expressions?Purdue UniversityRepresentation of objects Objects are unlikely to be represented by a single neuron Consider facesWhat do you do here?Since we perceive andrecognize all of thesefaces as beingdifferent. There mustbe a different neuralrepresentations.Purdue UniversityDistributed code Each cell codes some feature of the imageProf. Greg Francis4PSY 310: Sensory and Perceptual ProcessesPurdue UniversityDistributed code Each cell codes some feature of the imageThe representationof a face is apattern across thefeatures.No two faces havethe same pattern,so not the samepercept.Purdue UniversityDistributed code Each cell codes some feature of the imageChange ofexpression maychange only someof the features.Similar faces havesimilar patterns; sosimilar percepts.Purdue UniversityFeatures So what are the features? A difficult question to answer They need not be things that we would name Nose, eye, mouth, hair Could be Fourier components Or something completely differentPurdue UniversityFourier features Selecting certain Fourier components as features might notcorrespond to anything that we would name in the imagePurdue UniversityDistributed code How does it all get put together?We don’t seefeatures, we seefaces.Purdue UniversityRelated issue We earlier discussed how we can consider the visual system to consistof many retinotopic layers of activities from cells tuned to differentfeaturesImageRetinaGanglion cellsOrientationcellsProf. Greg Francis5PSY 310: Sensory and Perceptual ProcessesPurdue UniversityRelated issue There are actually many layers we never got a chance to talkabout Color, size, motion, depth, texture, many more To an extent, they process things independently - featuresImageRetinaGanglion cellsOrientationcellsColorMotionPurdue UniversityRelated issue If you see a red car go speeding by, you gets lots of responsesfrom different parts of the brain Which part is the percept? How do they get coordinated?ImageRetinaGanglion cellsOrientationcellsColorMotionPurdue UniversityRelated issue For that matter, if you are thinking of something else orlistening intently to something You may not perceive the car Even though the nervous system responds!ImageRetinaGanglion cellsOrientationcellsColorMotionPurdue UniversityAttention It’s not entirely clear what attention is or does But it seems to be involved in “pulling together” neuralresponses from different parts of the brain Perhaps by synchronizing action potentials Perhaps it selects the features that help solve a particular task E.g., recognize a face Attention seems to be necessary to actually perceivesomethingPurdue UniversityAttentional blink Suppose you have to identify rapidly presented (100 ms)letters e.g., detect J and/or K in a stream of lettersMPKRWSPurdue UniversityAttentional blink Turns out that detection of first letter tends to make detectionof the second letter very difficult if it immediately follows the first Attentional blinkMPKRJSProf. Greg Francis6PSY 310: Sensory and Perceptual ProcessesPurdue UniversityAttentional blink Measure frequency of detectionImplies thatdetecting firstletter causes you to