Eye Movements in Natural Behavior Authors Mary Hayhoe Dana Ballard Departments of Brain Cognitive Science and Computer Science University of Rochester Rochester NY 14627 USA mary cvs rochester edu dana cs rochester edu Abstract Although it has been recognized since Yarbus classic experiments over 50 years ago that saccadic eye movements reflect cognitive processes it is only recently that a number of separate lines of inquiry have merged to give an increasingly coherent understanding of the elaborate and intricate role of eye movements in cognitive function There are three principal advances The first is the demonstration of the role of eye movements in executing everyday visually guided behaviors The major findings of these studies have been the ubiquitous role of the task in such movements and the importance of learning where and when to fixate The second advance has been the recognition of the role of internal reward in guiding eye and body movements This has been revealed especially in neurophysiological studies The third important advance has been the theoretical developments in the fields of reinforcement learning and game theory together with developments in graphic simulation Together these developments have allowed the simulation realistic models of eye movements over extended time scales All of these advances are critical for understanding how behavioral programs control fixations and the selection of visual information 2 Over fifty years ago a Russian scientist Yarbus was able to capture the movements of the eye by attaching a mirror system to the eyeball Although he was not first to measure eye movements his work most clearly called attention to their intrinsically cognitive nature Nowadays most vision scientists are familiar with his traces of a subject examining Ripken s The Unexpected Visitor and the very different saccadic patterns elicited by the different questions asked of the subject 1 The significance of this finding was that it revealed in a particularly compelling way that seeing is not a unitary process and is inextricably linked to the observer s cognitive goals For example the instruction to remember the position of the objects and people in the room might easily be taken as the job of vision The fact that other instructions produced strikingly different patterns means that vision is much more complex Although the role of eye movements in cognitive tasks has been studied extensively since that time 2 4 it is only recently that a number of separate lines of inquiry have coalesced to give an increasingly coherent understanding of the elaborate and intricate role of eye movements in cognitive function The intent of this review is to describe these advances There are three principal advances that have had a direct bearing on the issue The first is the demonstration of the role of eye movements in executing everyday visually guided behaviors 5 9 These measurements have been driven by the development of portable eye trackers that can be worn by subjects engaged in behaviors that involve substantial body movements 10 12 Figure 1 shows one such device The major findings of these studies have been the ubiquitous role of the task in such movements and the importance of learning where and when to fixate The second advance has been the 3 recognition of the role of internal reward in guiding eye and body movements 13 17 This has been revealed especially in neurophysiological studies The third important advance has been the theoretical developments in the field of reinforcement learning together with tremendous developments in graphic simulation 18 20 Together these developments have allowed the simulation of reward based systems that incorporate realistic models of eye movements over extended time scales The sum total of all of these advances is to shift the focus of experimental understanding of the role of eye movements While the early focus has been on where the eyes fixate in an image we now are more interested in why the eyes choose a location in a scene and when they choose it Eye Tracking in Natural Behavior Yarbus subjects had to suffer a mirror attached to a suction cup on the sclera that had an unpleasant bulk Subsequent systems such as eye coils and the Dual Purkinje Image tracker were more comfortable but still required the head to be held fixed However the last ten years has seen the advent and rapid refinement of portable eye trackers that allow eye tracking during free viewing Michael Land built one of the first of these but it was hampered by having to have the eye position calculated off line 5 7 Modern eye trackers have fast software to do this and a recent version by Pelz shown in Figure 1 is completely portable using power from a backpack mounted battery pack 12 The new eye trackers allow the study of eye movements over extended tasks in natural settings where a much wider variety of natural coordinated behaviors is possible 4 The Importance of Task Although Yarbus study revealed the importance of the instructions in determining where subjects look the particular fixations do not reveal much more than that the observer attended to these locations While a given cognitive event might reliably lead to a particular fixation the fixation itself does not uniquely specify the cognitive event Box 2 This problem is particularly acute in situations where subjects passively view pictures where the experimenter often has little control of and no access to what the observer is doing There are indeed some regularities in fixation patterns that can be explained by image properties 21 23 However specific instructions to search or to remember objects lead to more specialized patterns 4 24 26 62 Furthermore if the task venue is set up so as to require sufficient visual acuity then eye movements are almost invariably locked to attention 27 28 Thus recent experiments where the task structure is evident have been more easily interpreted because the task provides an external referent for the internal computations 2 29 30 The most novel finding of such studies is that the task dependent nature of vision is much more important than the native visual saliency of the scene Box 1 In a variety natural tasks the eyes are positioned at a point that is not the most visually salient but is the best for the spatio temporal demands of the job that needs to be done This line of investigation has been used in extended visuomotor tasks such as driving walking sports and making tea
View Full Document
Unlocking...