Eye Movements in Natural BehaviorAuthors: Mary Hayhoe & Dana BallardDepartments of Brain & Cognitive Science and Computer ScienceUniversity of Rochester, Rochester, NY 14627, [email protected]@cs.rochester.edu2Abstract Although it has been recognized since Yarbus’ classic experiments over 50 years ago,that saccadic eye movements reflect cognitive processes, it is only recently that a numberof separate lines of inquiry have merged to give an increasingly coherent understanding ofthe elaborate and intricate role of eye movements in cognitive function. There are threeprincipal advances. The first is the demonstration of the role of eye movements inexecuting everyday visually guided behaviors. The major findings of these studies havebeen the ubiquitous role of the task in such movements, and the importance of learningwhere and when to fixate. The second advance has been the recognition of the role ofinternal reward in guiding eye and body movements. This has been revealed especially inneurophysiological studies. The third important advance has been the theoreticaldevelopments in the fields of reinforcement learning and game theory, together withdevelopments in graphic simulation. Together, these developments have allowed thesimulation realistic models of eye movements over extended time scales. All of theseadvances are critical for understanding how behavioral programs control fixations and theselection of visual information.3 Over fifty years ago a Russian scientist Yarbus was able to capture the movementsof the eye by attaching a mirror system to the eyeball. Although he was not first tomeasure eye movements, his work most clearly called attention to their intrinsicallycognitive nature. Nowadays most vision scientists are familiar with his traces of a subjectexamining Ripken’s “The Unexpected Visitor” and the very different saccadic patternselicited by the different questions asked of the subject [1]. The significance of this findingwas that it revealed in a particularly compelling way that “seeing” is not a unitary process,and is inextricably linked to the observer’s cognitive goals. For example, the instruction toremember the position of the objects and people in the room might easily be taken as thejob of vision. The fact that other instructions produced strikingly different patterns meansthat vision is much more complex. Although the role of eye movements in cognitive taskshas been studied extensively since that time [2-4], it is only recently that a number ofseparate lines of inquiry have coalesced to give an increasingly coherent understanding ofthe elaborate and intricate role of eye movements in cognitive function. The intent of thisreview is to describe these advances.There are three principal advances that have had a direct bearing on the issue. Thefirst is the demonstration of the role of eye movements in executing everyday visuallyguided behaviors [5-9]. These measurements have been driven by the development ofportable eye trackers that can be worn by subjects engaged in behaviors that involvesubstantial body movements [10-12]. Figure 1 shows one such device. The major findingsof these studies have been the ubiquitous role of the task in such movements, and theimportance of learning where and when to fixate. The second advance has been the4recognition of the role of internal reward in guiding eye and body movements [13-17].This has been revealed especially in neurophysiological studies. The third importantadvance has been the theoretical developments in the field of reinforcement learning,together with tremendous developments in graphic simulation [18-20]. Together, thesedevelopments have allowed the simulation of reward-based systems that incorporaterealistic models of eye movements over extended time scales. The sum total of all of theseadvances is to shift the focus of experimental understanding of the role of eye movements.While the early focus has been on where the eyes fixate in an image, we now are moreinterested in why the eyes choose a location in a scene, and when they choose it.Eye Tracking in Natural BehaviorYarbus’ subjects had to suffer a mirror attached to a suction cup on the sclera, thathad an unpleasant bulk. Subsequent systems, such as eye coils and the Dual PurkinjeImage tracker were more comfortable, but still required the head to be held fixed.However the last ten years has seen the advent and rapid refinement of portable eyetrackers that allow eye tracking during free viewing. Michael Land built one of the first ofthese but it was hampered by having to have the eye position calculated off line [5-7].Modern eye trackers have fast software to do this and a recent version by Pelz, shown inFigure 1, is completely portable, using power from a backpack-mounted battery pack [12].The new eye trackers allow the study of eye movements over extended tasks in naturalsettings, where a much wider variety of natural coordinated behaviors is possible.5 The Importance of Task. Although Yarbus’ study revealed the importance of the instructionsin determining where subjects look, the particular fixations do not reveal much more thanthat the observer attended to these locations. While a given cognitive event might reliablylead to a particular fixation, the fixation itself does not uniquely specify the cognitiveevent (Box 2). This problem is particularly acute in situations where subjects passivelyview pictures where the experimenter often has little control of, and no access to, whatthe observer is doing. There are indeed some regularities in fixation patterns that can beexplained by image properties [21-23]. However, specific instructions to search or toremember objects lead to more specialized patterns [4,24-26,62]. Furthermore, if the taskvenue is set up so as to require sufficient visual acuity, then eye movements are almostinvariably locked to attention [27, 28]. Thus, recent experiments where the task structureis evident have been more easily interpreted, because the task provides an externalreferent for the internal computations [2, 29,30]. The most novel finding of such studies isthat the task dependent nature of vision is much more important than the native visualsaliency of the scene (Box 1). In a variety natural tasks, the eyes are positioned at a pointthat is not the most visually salient, but is the best for the spatio-temporal demands of thejob that needs to be done. This line of investigation has been used in
View Full Document
Unlocking...