Walking from thoughts: Not the muscles are crucial, but the brain waves! Robert Leeb1, Claudia Keinrath1, Doron Friedman2, Christoph Guger3, Christa Neuper1,4, Maia Garau2, Angus Antley2, Anthony Steed2, Mel Slater2 and Gert Pfurtscheller1,5 1Laboratory of Brain-Computer Interfaces, Institute for Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16a/II, A-8010 Graz, Austria 2Department of Computer Science, University College London, Gower Street, WC1E 6BT London, United Kingdom 3g.tec - Guger Technologies OEG, Herbersteinstrasse 60, A-8020 Graz, Austria 4Department of Psychology, University of Graz, Universtitaetsplatz 2, A-8010 Graz, Austria 5Ludwig-Boltzmann Institut für Medizinische Informatik und Neuroinformatik, Graz University of Technology, Inffeldgasse 16a/II, A-8010 Graz, Austria {[email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]} AbstractAble-bodied participants are able to move forward in a Virtual Environment (VE) by imagining movements of their feet. This is achieved by exploiting a Brain-Computer Interface (BCI) which transforms thought-modulated EEG signals into an output signal that controls events within the VE. The experiments were carried out in an immersive projection environment, commonly referred to as a "Cave” in which participants were able to move through a virtual street by foot imagery alone. Experiments of BCI feedback on a normal monitor, VE experiments with a head-mounted display (HMD) and in the Cave-VE are compared. Keywords — Virtual environment (VE), Brain-Computer Interface (BCI), walking, thoughts1. Introduction “Yes he was walking! The illusion was utterly convincing …” experienced the leading actor from Arthur C. Clark in the book 3001, the final odyssey [1], when he was wearing a “Braincap” connected to the “Brainbox”. Thereby he could experience this science fiction technology and explore different virtual and ancient real worlds. Has this dream gone real? Here we show that participants are able to move forward – “to walk” – in a Virtual Environment (VE) by imagining movements of their feet. The improvement of seamless and natural human-computer interfaces is an all-the-time necessary task in virtual reality (VR) development. An interesting research problem is to realize locomotion through a VE only by mental activity or "thought". Typically, participants navigate by using a hand-held device, such as a joystick or a wand. Unfortunately contradictory stimuli appear in such situations; on the one hand the world around them is moving, which generates the illusion of walking, but on the other hand the participant is thinking on his index finger, for pressing the button on the joystick. This results in a reduced sense of being present in the VE, and is one of the causes of simulation sickness [2]. A possible next step towards next-generation interfaces could be achieved by exploiting a Brain-Computer Interface (BCI) which represents a direct connection between the human brain and the computer [3]. The electroencephalogram (EEG) of the human brain encompasses different types of oscillatory activities, in which the oscillations in the alpha and beta band (event-related desynchronization, ERD [4]) are particularly important to discriminate between different brain states (e.g. imagination of movements). A BCI transforms thought-modulated EEG signals into an output signal [3] that can control events within that VE [5, 6]. The goal of this work is to demonstrate that it is possible to move through different VEs, e.g. a virtual street, without any muscular activity, when the participant only imagines the movement of both feet and to show the influences of different feedback modalities on the same task. VR provides an excellent testing ground for procedures that may apply later in reality. One important future application may the use of VE for people with disabilities. If it is possible to show that people can learn to control their movements through space within a VE, it would justify the much bigger expense of building physical devices as e.g. a robot arm controlled by a BCI. 2. Methods 2.1. Graz Brain-Computer Interface Direct Brain-Computer communication is a novel approach to develop an additional communication channel for human-machine interaction. The imagination of PRESENCE 200525different types of movements, e.g. right hand, left hand, foot or tongue movement, results in a characteristic change of the EEG over the sensorimotor cortex of a participant [4]. The Graz-BCI detects changes in the ongoing EEG during the imagination of hand or foot movements and transforms them into a control signal [7]. Three bipolar derivations, located 2.5 cm anterior and posterior to the electrode positions C3, Cz and C4 of the international 10/20 system [8] were recorded with a sampling frequency of 250 Hz (sensitivity was set to 50µV) and bandpass filtered between 0.5 and 30 Hz. The ground electrode was positioned on the forehead. The logarithmic bandpower (BP) was calculated for each channel by digitally band-pass filtering the EEG (using a Butterworth filter of order 5) in the upper alpha (10 - 12 Hz) and beta band (16 - 24 Hz), squaring the signal and averaging the samples over a 1-s epoch. The resulting 4 BP features were transformed with Fishers linear discriminant analysis (LDA) [9] into a control signal. Finally the computed control signal was used to control / modify the feedback (FB) and either visualized on the same PC as a bar (see Figure 1a) or sent to the VE as a steering input inside a virtual world (see Figure 1b and 1c) [5]. The complete biosignal analysis system consisted of an EEG amplifier (g.tec, Graz, Austria), a data acquisition card (National Instruments Corporation, Austin, USA) and a recording device running under WindowsXP (Microsoft Corporation, Redmond, USA) on a commercial desktop PC [10]. The BCI algorithms were implemented in MATLAB 6.5 and Simulink 5.0 (The MathWorks, Inc., Natick, USA) using rtsBCI [11] and the open source package BIOSIG [12]. Detailed information about the physiological background of motor imagery and ERD can be found elsewhere [4, 13], also about signal processing, feature extraction and the Graz-BCI [7, 10] and generally about various BCI systems [3, 14]. 2.2. Participants and