The Receptive Fields of Inferior Temporal Cortex Neurons inNatural ScenesEdmund T. Rolls, Nicholas C. Aggelopoulos, and Fashan ZhengUniversity of Oxford, Department of Experimental Psychology, South Parks Road, Oxford OX1 3UD, United KingdomInferior temporal cortex neurons have generally been found to have large visual receptive fields that typically include the fovea and extendthroughout much of the visual field. However, a problem of such a large receptive field is that it does not easily support object selection bysubsequent processing areas, in that all objects within such a large receptive field might activate inferior temporal cortex cells. To clarifythis, we recorded from inferior temporal cortex neurons while macaques searched for objects in complex natural scenes or in plainbackgrounds, as normally used. Inferior temporal cortex neuron receptive fields were much smaller in natural scenes (mean radius, 11°)than in plain backgrounds (39°). With two objects in a scene, one of which was a target for action (a touch), the firing rates were equallyhigh during foveation of the effective stimulus when it was the target and when it was the distractor in both the plain and the complexscenes. With a plain background and two objects present, the receptive fields were much larger (24°) for the stimulus when it was thetarget than when it was the distractor (9°). This effect of object-based attention was much less evident in the complex scene, when thereceptive fields were small both when the stimulus was a distractor and when it was a target. The results show that the temporal visualcortex provides an unambiguous representation in natural scenes by responding to the object shown at or close to the fixation point.Key words: visual search; translation invariance; attention; object recognition; rhesus monkey; active visionIntroductionInferior temporal cortex (IT) neurons of macaques have re-sponses that provide information about objects or faces (Gross etal., 1972; Perrett et al., 1982; Rolls, 1992, 2000; Booth and Rolls,1998; Rolls and Deco, 2002). The responses of these neurons areoften relatively invariant with respect to the position in the visualfield, size, and even view of the object (Gross et al., 1972; Rolls andBaylis, 1986; Tovee et al., 1994; Booth and Rolls, 1998). This is animportant property, because when areas that receive from theinferior temporal visual cortex such as the orbitofrontal cortex,amygdala, and hippocampus learn about one view, position, orsize of an object, the learning then generalizes to other views,positions, or sizes of the same object (Rolls and Treves, 1998;Rolls and Deco, 2002).Much visual neurophysiology is conducted with one visualstimulus present in an otherwise blank visual scene (Hubel andWiesel, 1982; Gross et al., 1985). Even in studies of the neuronalmechanisms of selective attention, there are usually only twosmall visual stimuli present in the visual field, which is otherwiseblank (Chelazzi et al., 1993, 1998; Desimone and Duncan, 1995;Chelazzi, 1998; Chelazzi and Corbetta, 2000). In conditions inwhich one visual stimulus is present, inferior temporal cortexneurons typically have large receptive fields, ⱖ50° in diameter,under anesthesia (Gross et al., 1972) and when performing avisual fixation task (Tovee et al., 1994). However, a problem ofsuch large inferior temporal cortex neuron receptive fields is thatthey do not easily support object selection by subsequent process-ing areas, in that all objects within such a large neuronal receptivefield might activate different inferior temporal cortex cells, sothat the output of the inferior temporal cortex might appear as a“tower of Babel.” For example, if multiple objects were presentwithin the large receptive field of inferior temporal cortex neu-rons, the orbitofrontal cortex and amygdala would retrieve manydifferent reward–punishment associations simultaneously, andthe rest of the brain would not know what to approach or avoid.Nor would there be any way of directing action at the correct goalobject in such a scene with multiple objects, or indeed in anycluttered natural scene. Therefore, the issue arises of how thevisual system operates in a natural and cluttered visual scene.The aim was to investigate the sizes of the receptive fields ofinferior temporal cortex neurons in natural scenes.Materials and MethodsWe measured the magnitude of the responses of inferior temporal cortexneurons when an effective stimulus was shown in blank scenes, in com-plex natural scenes, and in scenes with one other image present, as istypical in previous studies of attention. In the visual search task, in onecondition the effective image was the object of attention, in the sense thatthe monkey was required to search for that object on the screen and touchit. In another condition, the effective image for the neuron was not theobject of attention, in that the monkey was searching for another objectto touch.Recording techniques. The activity of single neurons was recorded withglass insulated tungsten microelectrodes in two macaque monkeys (Ma-caca mulatta; weight, ⬃4 – 6 kg) in a primate chair using techniquesdescribed previously (Rolls et al., 1990; Tovee et al., 1993; Booth andRolls, 1998). All preparative and subsequent procedures were performedin accordance with the National Institutes of Health Guide for the Careand Use of Laboratory Animals and were licensed under the UK Animals(Scientific Procedures) Act of 1986. The action potentials of single neu-rons were amplified (Rolls et al., 1979) and converted to digital pulsesusing the trigger circuitry of an oscilloscope and analyzed online using anReceived April 24, 2002; revised Oct. 9, 2002; accepted Oct. 18, 2002.This work was supported by the Wellcome Trust and by the Medical Research Council Interdisciplinary ResearchCentre for Cognitive Neuroscience.Correspondence should be addressed to Prof. E. T. Rolls, Department of Experimental Psychology, University ofOxford, South Parks Road, Oxford OX1 3UD, UK. E-mail: [email protected] © 2002 Society for Neuroscience 0270-6474/02/220339-10$15.00/0The Journal of Neuroscience, January 1, 2003 • 23(1):339 –348 • 339IBM-compatible personal computer. The isolation of single neurons wasensured using Brainwave enhanced Discovery data acquisition, usingcluster cutting for offline data analysis (DataWave Technologies, Long-mont, CO), and establishing that no spikes occurred very close