Deceptive Speech CS4706 Julia Hirschberg 1 Everyday Lies Ordinary people tell an average of 2 lies per day Your new hair cut looks great I m sorry I missed your talk but many variants In many cultures white lies are more acceptable than the truth Likelihood of being caught is low Rewards also low but outweigh consequences of being caught Not so easy to detect 2 Serious Lies Lies where Risks high Rewards high Emotional consequences more apparent Are these lies easier to detect By humans By machines 3 Outline Research on Deception Possible cues to deception Current approaches Our corpus based study of deceptive speech Approach Corpus collection paradigm Features extracted Experiments and results Human perception studies Future research 4 A Definition of Deception Deliberate choice to mislead Without prior notification To gain some advantage or to avoid some penalty Not Self deception delusion pathological behavior Theater Falsehoods due to ignorance error 5 Who Studies Deception Students of human behavior especially psychologists Law enforcement and military personnel Corporate security officers Social services workers Mental health professionals 6 Why is Lying Difficult for Most of Us Hypotheses Our cognitive load is increased when lying because Must keep story straight Must remember what we ve said and what we haven t said Our fear of detection is increased if We believe our target is hard to fool We believe our target is suspicious Stakes are high serious rewards and or punishments Makes it hard for us to control indicators of deception Does this make deception easy to detect 7 Cues to Deception Current Hypotheses Body posture and gestures Burgoon et al 94 Complete shifts in posture touching one s face Microexpressions Ekman 76 Frank 03 Fleeting traces of fear elation Biometric factors Horvath 73 Increased blood pressure perspiration respiration Odor Changes in brain activity Variation in what is said and how Adams 96 Pennebaker et al 01 Streeter et al 77 8 Spoken Cues to Deception DePaulo et al 03 Liars less forthcoming Talking time Details Presses lips Liars less compelling Plausibility Logical Structure Discrepant ambivalent Verbal vocal involvement Illustrators Verbal vocal immediacy Verbal vocal uncertainty Chin raise Word phrase repetitions Liars less positive pleasant Cooperative Negative complaining Facial pleasantness Liars more tense Nervous tense overall Vocal tension F0 Pupil dilation Fidgeting Fewer ordinary imperfections Spontaneous corrections Admitted lack of memory Peripheral details 9 Current Approaches to Deception Detection Training humans John Reid Associates Behavioral Analysis Interview and Interrogation Laboratory studies Production and Perception Automatic methods Polygraph Voice Stress Analysis Microtremors 8 12Hz Nemesysco and the Love Detector No evidence that any of these work but publishing this can be dangerous Anders Eriksson and Francisco La Cerda 10 Little objective experimentally verified study of spoken cues to deception which can be extracted automatically 11 Outline Research on Deception Possible cues to deception Current approaches Our corpus based study of deceptive speech Approach Corpus collection paradigm Features extracted Experiments and results Human perception studies Future research 12 Corpus Based Approach to Deception Detection Goal Identify a set of acoustic prosodic and lexical features that distinguish between deceptive and non deceptive speech as well or better than human judges Method Record a new corpus of deceptive non deceptive speech Extract acoustic prosodic and lexical features based on previous literature and our work in emotional speech and speaker id Use statistical Machine Learning techniques to train models to classify deceptive vs non deceptive speech 13 Major Obstacles Corpus based approaches require large amounts of training data difficult to obtain for deception Differences between real world and laboratory lies Motivation and potential consequences Recording conditions Identifying ground truth Ethical issues Privacy Subject rights and Institutional Review Boards 14 Columbia SRI Colorado Deception Corpus CSC Deceptive and non deceptive speech Within subject 32 adult native speakers 25 50m interviews Design Subjects told goal was to find people similar to the 25 top entrepreneurs of America Given tests in 6 categories e g knowledge of food and wine survival skills NYC geography civics music e g What should you do if you are bitten by a poisonous snake out in the wilderness Sing Casta Diva What are the 3 branches of government 15 Questions manipulated so scores always differed from a fake entrepreneur target in 4 6 categories Subjects then told real goal was to compare those who actually possess knowledge and ability vs those who can talk a good game Subjects given another chance at 100 lottery if they could convince an interviewer they matched target completely Recorded interviews Interviewer asks about overall performance on each test with follow up questions e g How did you do on the survival skills test Subjects also indicate whether each statement T or F by pressing pedals hidden from interviewer 16 17 The Data 15 2 hrs of interviews 7 hrs subject speech Lexically transcribed automatically aligned Truth conditions aligned with transcripts Global Local Segmentations Local Truth Local Lie Words 31 200 47 188 Slash units 5709 3782 Prosodic phrases 11 612 7108 Turns 2230 1573 250 features Acoustic prosodic features extracted from ASR transcripts Lexical and subject dependent features extracted from orthographic transcripts 18 Acoustic Prosodic Features Duration features Phone Vowel Syllable Durations Normalized by Phone Vowel Means Speaker Speaking rate features vowels time Pause features cf Benus et al 06 Speech to pause ratio number of long pauses Maximum pause length Energy features RMS energy Pitch features Pitch stylization Sonmez et al 98 Model of F0 to estimate speaker range Pitch ranges slopes locations of interest Spectral tilt features 20 Lexical Features Presence and of filled pauses Is this a question A question following a question Presence of pronouns by person case and number A specific denial Presence and of cue phrases Presence of self repairs Presence of contractions Presence of positive negative emotion words Verb tense Presence of yes no not negative contractions Presence of absolutely really Presence of hedges Complexity syls words Number of repeated words Punctuation type Length of unit