International Conference on Computer Systems and Technologies - CompSysTech’08 Emotion Recognition using Brain Activity Robert Horlings, Dragos Datcu, Leon J. M. Rothkrantz Abstract: Our project focused on recognizing emotion from human brain activity, measured by EEG signals. We have proposed a system to analyze EEG signals and classify them into 5 classes on two emotional dimensions, valence and arousal. This system was designed using prior knowledge from other research, and is meant to assess the quality of emotion recognition using EEG signals in practice. In order to perform this assessment, we have gathered a dataset with EEG signals. This was done by measuring EEG signals from people that were emotionally stimulated by pictures. This method enabled us to teach our system the relationship between the characteristics of the brain activity and the emotion. We found that the EEG signals contained enough information to separate five different classes on both the valence and arousal dimension. However, using a 3-fold cross validation method for training and testing, we reached classification rates of 32% for recognizing the valence dimension from EEG signals and 37% for the arousal dimension. Much better classification rates were achieved when using only the extreme values on both dimensions, the rates were 71% and 81%. Key words: EEG, emotion, classification, Brain Computing. INTRODUCTION In spite of the difficulty of precisely defining it, emotion is omnipresent and an important factor in human life. People’s moods heavily influence their way of communicating, but also their acting and productivity. Emotion also plays a crucial role in all-day communication. One can say a word like ”OK” in a happy way, but also with disappointment or sarcasm. In most communication this meaning is interpreted from the pitch of the voice or from non-verbal communication. Other emotions are in general only expressed by body language, like boredom. A large part of communication is done nowadays by computer or other electronic devices. This interaction is a lot different from the way human beings interact. Most of the communication between human beings involves non-verbal signs, and the social aspect of this communication is important. Humans also tend to include this social aspect when communicating with computers [14]. This interaction with or through a computer could be improved when the computer could recognize the user’s emotion. A lot of research is already done after recognizing emotions by computers. For example, research has been done to make computers recognize emotion from speech [7], facial expressions [12] or a fusion of both methods [4]. Measuring emotion from brain activity is a relatively new method. Electroencephalography (EEG) is a relatively easy and cheap method to measure this brain activity. It has been shown that emotional markers are present in EEG signals. Using these signals also has the advantage over other methods that they can hardly be deceived by voluntary control and are available all the time, without needing any further action of the user. The disadvantage of using these signals is that the user has to wear some measurement equipment, which can be quite demanding. Possible applications of this technique are many. First of all, the simple task of recognizing someone’s emotion automatically could assist therapists and psychologists in doing their job. Other applications can be found in the field of human machine interaction, human communication through computers and in assisting disabled people with communicating emotion. Our project focuses on creating a program for emotion recognition from EEG signals in practice. This programs design is based on literature on this subject. Some design choices are based on the results of our own experiments. RELATED WORK Several works exist that are related to emotion recognition from brain activity. On the one hand, there is information from neuroscientists about how the brain processes emotion [3], how emotion can be modeled [8] and how to recognize emotion from EEG recordings - II.1-1 -International Conference on Computer Systems and Technologies - CompSysTech’08 [1,16,17]. On the other hand there have been several research papers from researchers that have created programs to validate those theories by creating systems to recognize emotion [55,6]. The literature shows a lot of information about emotions, but also a lot of inconsistency. There is no agreement on the functioning of the brain, how to model emotion or how emotion can be measured. This fact makes it hard to build our program upon this knowledge. Fortunately, researchers that have tried to implement the theories report that in spite of this inconsistency emotion can be recognized from brain activity to some extent. Choppin uses neural networks to classify the EEG signals online, and achieved a correct classification rate for new unseen samples of about 64%, when using three emotion classes [6]. Another experiment of Chanel et. al. tried to recognize only the arousal dimension of emotion from EEG and other physiological measures [5]. Classification rates were around 60% when using two classes and 50% when using three classes. Most research uses a dimensional model of emotions. This model divides all emotions into two dimensions, valence (positive-negative) and arousal (calm-exciting). Emotions are then thought to be a point in the two-dimensional plane of valence and arousal. Many researchers state that different emotions can be found best in EEG recordings by looking at the difference in activity of both hemispheres [16]. Two different theories are both supported by evidence. The right hemisphere hypothesis says that the right hemisphere is mostly involved in processing emotion, and that this lateralization is most apparent with negative emotions. The valence asymmetry hypothesis poses that the involvement of both hemispheres depends on the valence of the emotion. The right hemisphere is dominant with negative emotions whereas the left hemisphere