Slide 1In the last 20 yearsSlide 3Slide 4Dimensional approach. (Russell, 1980, 2003)Image from Russell 1997Distinctive vs. Dimensional approach of emotionSlide 8Slide 9Slide 10Slide 11Scherer’s typology of affective statesScherer’s typologyWhy Emotion Detection from Speech or Text?Hard Questions in Emotion RecognitionSlide 16But…. Different Valence/ Same ActivationAccuracy of facial versus vocal cues to emotion (Scherer 2001)Background: The Brunswikian Lens ModelSlide 20Emotional communicationImplications for HMIExtroversion in Brunswikian LensData and tasks for Emotion DetectionFour quick case studiesSlide 26EPSaT ExamplesDetecting EPSaT EmotionsLiscombe et al. FeaturesGlobal Pitch StatisticsGlobal Pitch StatisticsLiscombe et al. FeaturesLiscombe et al. ExperimentExample 2 - Ang 2002Data AnnotationData LabelingEmotion SamplesEmotion Class DistributionProsodic ModelProsodic FeaturesProsodic Features (cont.)Features (cont.)Language Model FeaturesResults: Human and MachineResults (cont.)Predictors of Annoyed/FrustratedEffect of Class DefinitionAng et al ‘02 ConclusionsExample 3: “How May I Help YouSM” (HMIHY)User Emotion DistributionEmotion LabelsSlide 52Pitch/Energy/Speaking Rate : ChangeSlide 54Pitch/Energy/Speaking Rate : No ChangeHMIHY FeaturesLexical FeaturesProsodic FeaturesContextual FeaturesHMIHY ExperimentIntelligent Tutoring Spoken Dialogue SystemSlide 62Task 1Liscombe et al: Uncertainty in ITSpokeSlide 65Slide 66Liscombe et al: ITSpoke ExperimentSome summaries re: Prosodic featuresJuslin and Laukka metastudySlide 70Slide 71DiscussionDan JurafskyLecture 6: EmotionCS 424P/ LINGUIST 287Extracting Social Meaning and SentimentIn the last 20 yearsA huge body of research on emotionJust one quick pointer: Ekman: basic emotions:Ekman’s 6 basic emotionsSurprise, happiness, anger, fear, disgust, sadnessDisgustAngerFearHappinessSurpriseSadnessSlide from Harinder AujlaDimensional approach. (Russell, 1980, 2003)Arousal High arousal, High arousal, Displeasure (e.g., anger) High pleasure (e.g., excitement) ValenceLow arousal, Low arousal, Displeasure (e.g., sadness) High pleasure (e.g., relaxation) Slide from Julia Braverman6Image from Russell 1997valence-+arousal-Image fromRussell, 1997Distinctive vs. Dimensional approach of emotionDistinctiveEmotions are units.Limited number of basic emotions.Basic emotions are innate and universalMethodology advantageUseful in analyzing traits of personality.DimensionalEmotions are dimensions.Limited # of labels but unlimited number of emotions.Emotions are culturally learned.Methodological advantage:Easier to obtain reliable measures.Slide from Julia BravermanFour Theoretical Approaches to Emotion: 1. Darwinian (natural selection)Darwin (1872) The Expression of Emotion in Man and Animals. Ekman, Izard, PlutchikFunction: Emotions evolve to help humans surviveSame in everyone and similar in related speciesSimilar display for Big 6+ (happiness, sadness, fear, disgust, anger, surprise)  ‘basic’ emotionsSimilar understanding of emotion across culturesextended from Julia Hirschberg’s slides discussing Cornelius 2000The particulars of fear may differ, but "the brain systems involved in mediating the function are the same in different species" (LeDoux, 1996)Four Theoretical Approaches to Emotion: 2. Jamesian: Emotion is experienceWilliam James 1884. What is an emotion? Perception of bodily changes  emotion“we feel sorry because we cry… afraid because we tremble"’ “our feeling of the … changes as they occur IS the emotion"The body makes automatic responses to environment that help us survive Our experience of these reponses consitutes emotion.Thus each emotion accompanied by unique pattern of bodily responsesStepper and Strack 1993: emotions follow facial expressions or posture.Botox studies: Havas, D. A., Glenberg, A. M., Gutowski, K. A., Lucarelli, M. J., & Davidson, R. J. (2010). Cosmetic use of botulinum toxin-A affects processing of emotional language. Psychological Science, 21, 895-900.Hennenlotter, A., Dresel, C., Castrop, F., Ceballos Baumann, A. O., Wohlschlager, A. M., Haslinger, B. (2008). The link between facial feedback and neural activity within central circuitries of emotion - New insights from botulinum toxin-induced denervation of frown muscles. Cerebral Cortex, June 17.extended from Julia Hirschberg’s slides discussing Cornelius 2000Four Theoretical Approaches to Emotion: 3. Cognitive: AppraisalAn emotion is produced by appraising (extracting) particular elements of the situation. (Scherer)Fear: produced by the appraisal of an event or situation as obstructive to one’s central needs and goals, requiring urgent action, being difficult to control through human agency, and lack of sufficient power or coping potential to deal with the situation. Anger: difference: entails much higher evaluation of controllability and available coping potentialSmith and Ellsworth's (1985): Guilt: appraising a situation as unpleasant, as being one's own responsibility, but as requiring little effort.Adapted from Cornelius 2000Four Theoretical Approaches to Emotion: 4. Social ConstructivismEmotions are cultural products (Averill)Explains gender and social group differencesanger is elicited by the appraisal that one has been wronged intentionally and unjustifiably by another person. Based on a moral judgmentdon’t get angry if you yank my arm accidentallyor if you are a doctor and do it to reset a boneonly if you do it on purposeAdapted from Cornelius 2000Scherer’s typology of affective statesEmotion: relatively brief eposide of synchronized response of all or most organismic subsysteems in responseto the evaluation of an extyernalor internal event as being of major significanceangry, sad, joyful, fearful, ashamed, proud, desparateMood: diffuse affect state, most pronounced as change in subjective feeling, of low intensity but relatively long duration, often without apparent causecheerful, gloomy, irritable, listless, depressed, buoyantInterpersonal stance: affective stance taken toward another person in a specific interaction, coloring the interpersonal exchange in that situationdistant, cold, warm, supportive, contemptuousAttitudes: relatively unduring, afectively color beleifes, preference,s predispotiions