Task-Evoked Pupillary Response to Mental Workload in Human-Computer Interaction Shamsi T. Iqbal♦, Xianjun Sam Zheng† and Brian P. Bailey♦ Department of Computer Science♦ and Beckman Institute† University of Illinois Urbana, IL 61801, USA (siqbal, xzheng, bpbailey)@uiuc.edu Abstract Accurate assessment of a user’s mental workload will be critical for developing systems that manage user attention (interruptions) in the user interface. Empirical evidence suggests that an interruption is much less disruptive when it occurs during a period of lower mental workload. To provide a measure of mental workload for interactive tasks, we investigated the use of task-evoked pupillary response. Results show that a more difficult task demands longer processing time, induces higher subjective ratings of mental workload, and reliably evokes greater pupillary response at salient subtasks. We discuss the findings and their implications for the design of an attention manager. Categories & Subject Descriptors: H.5.2 [Information Interfaces and Presentation]: User Interfaces – Evaluation and Methodology; H.1.2 [Models and Principles]: User/Machine Systems – Human Information Processing. General Terms: Design; Experimentation; Human Factors. Keywords: Attention; Interruption; Mental Workload; Pupil Size; Task Models; User Studies. INTRODUCTION Productive interaction between humans and computers requires that a user must effectively manage her attention among the applications that are competing for it. A poorly timed notification (interruption) due to instant messages, incoming email, or system alert can disrupt a user’s task performance [3, 6] and emotional state [1, 3, 13]. An attractive solution is to develop an attention manager that manages where in a user’s task sequence an application can gain user attention. Empirical studies show that a less disruptive moment for an interruption is during a period of low mental workload in a user’s task sequence [6, 7, 11]. Thus, a significant challenge in developing an attention manager is to develop a reliable measure of mental workload for a dynamic task environment such as the desktop interface. Although pupil size is known to correlate well with the mental workload for discrete, non-interactive tasks [10], we investigated how well pupil size correlates with the mental workload demanded by interactive tasks representative of daily computer-based tasks that users often perform. In our study, a user performed an easier and more difficult task from several task categories and we measured pupil size using a head-mounted eye-tracker. We used task completion time and subjective ratings of difficulty to validate the mental workload imposed by the tasks. Our results show that a more difficult task demands longer processing time, induces higher subjective ratings of mental workload, and reliably evokes greater pupillary response at corresponding subtasks than a less difficult task. We discuss our empirical findings and their implications for the design of an attention manager. RELATED WORK Mental Workload Assessment Techniques Mental workload can be assessed with a number of techniques, including task performance on primary and secondary tasks [11], subjective ratings, and physiological measures (pupil size, heart rate, EEG) [8]. We believe that pupil size is the most promising single measure of mental workload because it does not disrupt a user’s ongoing activities, provides real-time information about the user’s mental workload, and is less intrusive than other physiological measures such as heart rate or EEG. Task-Evoked Pupillary Response The correlation of pupil size with mental workload has long been supported [4, 9, 10]. Research has shown that pupil dilations occur at short latencies following the onset of a task and subside quickly once the task is completed. More importantly, the magnitude of the pupillary dilation appears to be a function of processing load, or the mental effort required to perform the cognitive task. Note that other than the task factor, some environmental factors (e.g. ambient illumination and the near reflex), or emotional states may also induce pupillary response, producing changes of pupil size. Nonetheless, Beatty [4] has shown that task-evoked pupillary response uniquely reflects the momentary level of processing load and is not an artifact of non-cognitive confounding factors. In fact, the task-evoked pupillary response has been widely used as a tool to investigate various aspects of human information processing, such as perception, memory, Copyright is held by the author/owner(s). CHI 2004, April 24–29, 2004, Vienna, Austria. ACM 1-58113-703-6/04/0004. CHI 2004 ׀ Late Breaking Results Paper 24-29 April ׀ Vienna, Austria 1477reasoning and reading [4]. However, whether pupil size can provide a real-time measure of mental workload for more natural interactive tasks in human-computer interaction requires further investigation. USER STUDY We conducted a user study to answer two main questions: • How well does pupil size correlate with the mental workload that tasks from the same task category impose on a user? • Is this correlation pattern consistent across several categories of primary task? Experimental Design The study was a 4 Task Category (Object manipulation, Reading comprehension, Mathematical reasoning and Searching) x 2 Difficulty (Easy and Difficult) repeated measures within-subjects design. Equipment As a user performed tasks, we recorded his pupil data using a head-mounted SR Inc., Eyelink II eyetracker with a 250 HZ sample rate and 0.005 degree spatial resolution. User and Tasks Twelve users (6 female) volunteered in the user study. The average age of the users was 23.7 years (SD = 3.23). An important part of the study was to identity a meaningful set of task categories representative of daily work tasks. We determined task categories from a literature review, an informal questionnaire to several users, our own experience, and the consideration of user time for the study. Four task categories were developed, each with two difficulty levels (easy vs. difficult): • Reading Comprehension. A user read a given text and answered questions. The easier task belonged to grade 9 level and the more difficult task belonged to grade 17. • Mathematical Reasoning. A user performed math calculations. For the easier task, a user had to mentally add two four digit numbers and