Brain-Computer Interfaces for HCI and Games Abstract In this workshop we study the research themes and the state-of-the-art of brain-computer interaction. Brain-computer interface research has seen much progress in the medical domain, for example for prosthesis control or as biofeedback therapy for the treatment of neurological disorders. Here, however, we look at brain-computer interaction especially as it applies to research in Human-Computer Interaction (HCI). Through this workshop and continuing discussions, we aim to define research approaches and applications that apply to disabled and able-bodied users across a variety of real-world usage scenarios. Entertainment and game design is one of the application areas that will be considered. ACM Classification Keywords H5.2. Information interfaces and presentation (e.g., HCI): User Interfaces (D.2.2, H.1.2, I.3.6). Keywords Brain-computer interfaces, multimodal interaction, affective computing, games Introduction Advances in cognitive neuroscience and brain imaging technologies provide us with the increasing ability to Copyright is held by the author/owner(s). CHI 2008, April 5 – 10, 2008, Florence, Italy. ACM 978-1-60558-012-8/08/04. Anton Nijholt University of Twente, CTIT PO Box 217, 7500 AE Enschede the Netherlands [email protected] Brendan Allison IAT, University of Bremen Otto-Hahn-Allee NW1, N1151 28359 Bremen, Germany [email protected] Melody Moore Jackson GT BrainLab, School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA [email protected] Desney Tan Microsoft Research Redmond, WA, USA [email protected] José del R. Millán Rue du Simplon 4 1920 Martigny. Switzerland [email protected] Bernhard Graimann IAT, University of Bremen Otto-Hahn-Allee NW1, N1151 28359 Bremen, Germany [email protected] CHI 2008 Proceedings · Workshops April 5-10, 2008 · Florence, Italy3925interface directly with activity in the brain. Researchers have begun to use these technologies to build brain-computer interfaces. In these interfaces, humans intentionally manipulate their brain activity in order to directly control a computer or physical prostheses. The ability to communicate and control devices with thought alone has especially high impact for individuals with reduced capabilities for muscular response. In fact, applications for patients with severe motor disabilities have been the driving force of most brain-computer interface research. The Potential of Brain-Computer Interfaces Although removing the need for motor movements in computer interfaces is itself challenging and rewarding, we believe that the full potential of brain sensing technologies as an input mechanism lies in the extremely rich information it could provide about the state of the user [5,8]. Having access to this state information is valuable to human-computer interaction (HCI) researchers and opens up at least three distinct areas of research: Controlling Computers with Thought Alone. Much of the current BCI work aims to improve the lives of patients with severe neuromuscular disorders in which many patients lose control of their bodies, including simple functions such as eye-gaze. However, many of these patients retain full control of their higher level cognitive abilities. These disorders cause extreme frustration or social isolation caused by having no way to communicate with the external world. Providing these patients with brain-computer interfaces that allow them to control computers directly with their brain signals could dramatically increase their quality of life. The complexity of this control ranges from simple binary decisions, to moving a cursor on the screen, to more ambitious control of mechanical prosthetic devices. Nearly all current brain-computer interface research has been a logical extension of assistive methods in which one input modality is substituted for another (for detailed reviews of this work, see [2,5]). However, there now is the need to start thinking about brain-computer interface applications for users with no physical disabilities and where brain activity can be seen as one of many of the possible input modalities that can be used sequentially or parallel with other input modalities. Clearly, also able-bodied users can enter applications where they meet situational impairments. This includes applications in domains such as traditional communication and productivity tasks, as well as games and entertainment computing. Evaluating Interfaces and Systems. The cognitive or affective state derived from brain imaging could be used as an evaluation metric for either the user or for computer systems. Since we can measure the intensity of cognitive activity as a user performs certain tasks, we could potentially use brain imaging to assess cognitive aptitude based on how hard someone has to work on a particular set of tasks. With proper task and cognitive models, we might use these results to generalize performance predictions in a much broader range of tasks and scenarios. In addition to evaluating the human, we can understand how users and computers interact so that we can improve our computing systems. Thus far, we have been relatively successful in learning from performance metrics such as task completion times and CHI 2008 Proceedings · Workshops April 5-10, 2008 · Florence, Italy3926error rates. We have also used behavioral and physiological measures to infer cognitive processes, such as mouse movement and eye gaze as a measure of attention. However, there remain many cognitive processes that are hard to measure externally. For example, it is still extremely difficult to ascertain cognitive workloads or particular cognitive strategies used, such as verbal versus spatial memory encoding. Brain imaging can potentially provide measures that directly quantify the cognitive utility of our interfaces. This could potentially provide powerful measures that either corroborate external measures, or more interestingly, shed light on the interactions that we would have never derived from these measures alone. Building Adaptive User Interfaces. If we tighten the iteration between measurement, evaluation, and redesign, we could design interfaces that automatically adapt depending on the cognitive state of the user. Interfaces that adapt themselves to available resources in order to provide pleasant and optimal user experiences are not a new concept. In fact, we have put quite a bit of