Spoken Cues to Deception CS 4706 What is Deception Defining Deception Deliberate choice to mislead a target without prior notification To gain some advantage or to avoid some penalty Not Self deception delusion Theater Falsehoods due to ignorance error Pathological behavior NB people typically tell at least 2 lies per day Who Studies Deception Language and cognition Law enforcement practitioners Police Military Jurisprudence Intelligence agencies Social services workers SSA Housing Authority Business security officers Mental health professionals Political consultants Why is it hard to deceive Increase in cognitive load if Fabrication means keeping story straight Concealment means remembering what is omitted Fear of detection if Target believed to be hard to fool Target believed to be suspicious Stakes are high serious rewards and or punishments Hard to control indicators of emotion deception So deception detection may be possible Potential Cues cf DePaulo 03 Body posture and gestures Burgoon et al 94 Complete shifts in posture touching one s face Microexpressions Ekman 76 Frank 03 Fleeting traces of fear elation Biometric factors Horvath 73 Increased blood pressure perspiration respiration Variation in what is said and how Adams 96 Pennebaker et al 01 Streeter et al 77 Contractions lack of pronominalization disfluencies slower response mumbled words increased or decreased pitch range less coherent microtremors Potential Cues to Deception DePaulo et al 03 Liars less forthcoming Talking time Details Presses lips Liars less compelling Plausibility Logical Structure Discrepant ambivalent Verbal vocal involvement Illustrators Verbal vocal immediacy Verbal vocal uncertainty Chin raise Word phrase repetitions Liars less positive pleasant Cooperative Negative complaining Facial pleasantness Liars more tense Nervous tense overall Vocal tension F0 Pupil dilation Fidgeting Fewer ordinary imperfections Spontaneous corrections Admitted lack of memory Peripheral details Potential Spoken Cues to Deception DePaulo et al 03 Liars less forthcoming Talking time Details Presses lips Liars less compelling Plausibility Logical Structure Discrepant ambivalent Verbal vocal involvement Illustrators Verbal vocal immediacy Verbal vocal uncertainty Chin raise Word phrase repetitions Liars less positive pleasant Cooperative Negative complaining Facial pleasantness Liars more tense Nervous tense overall Vocal tension F0 Pupil dilation Fidgeting Fewer ordinary imperfections Spontaneous corrections Admitted lack of memory Peripheral details Previous Approaches to Deception Detection John Reid Associates Behavioral Analysis Interview and Interrogation Polygraph http antipolygraph org The Polygraph and Lie Detection N A P 2003 Voice Stress Analysis Microtremors 8 12Hz No real evidence Nemesysco and the Love Detector Newer Techniques for Automatic Analysis Most previous deception studies focus on Visual or biometric behaviors A few hand coded or perception based cues Our goal Identify a set of acoustic prosodic and lexical features that distinguish between deceptive and non deceptive speech As well or better than human judges Using automatic feature extraction Using Machine Learning techniques to identify bestperforming features and create automatic predictors Our Approach Record a new corpus of deceptive non deceptive speech and transcribe it Use automatic speech recognition ASR technology to perform forced alignment on transcripts Extract acoustic prosodic and lexical features based on previous literature and our work in emotional speech and speaker id Use statistical Machine Learning techniques to train models to distinguish deceptive from nondeceptive speech Rule induction Ripper CART trees SVMs Major Obstacles Corpus based approaches require large amounts of training data ironically difficult for deception Differences between real world and laboratory lies Motivation and consequences Recording conditions Assessment of ground truth Ethical issues Privacy Subject rights and Institutional Review Boards Columbia SRI Colorado Deception Corpus CSC Deceptive and non deceptive speech Within subject 32 adult native speakers 25 50m interviews Design Subjects told goal was to find people similar to the 25 top entrepreneurs of America Given tests in 6 categories e g knowledge of food and wine survival skills NYC geography civics music e g What should you do if you are bitten by a poisonous snake out in the wilderness Sing Casta Diva What are the 3 branches of government Questions manipulated so scores always differed from a fake entrepreneur target in 4 6 categories Subjects then told real goal was to compare those who actually possess knowledge and ability vs those who can talk a good game Subjects given another chance at 100 lottery if they could convince an interviewer they match target completely Recorded interviews Interviewer asks about overall performance on each test with follow up questions e g How did you do on the survival skills test Subjects also indicate whether each statement T or F by pressing pedals hidden from interviewer The Data 15 2 hrs of interviews 7 hrs subject speech Lexically transcribed automatically aligned Truth conditions aligned with transcripts Global Local Segmentations Local Truth Local Lie Words 31 200 47 188 Slash units 5709 3782 Prosodic phrases 11 612 7108 Turns 2230 1573 250 features Acoustic prosodic features extracted from ASR transcripts Lexical and subject dependent features extracted from orthographic transcripts Acoustic Prosodic Features Duration features Phone Vowel Syllable Durations Normalized by Phone Vowel Means Speaker Speaking rate features vowels time Pause features cf Benus et al 06 Speech to pause ratio number of long pauses Maximum pause length Energy features RMS energy Pitch features Pitch stylization Sonmez et al 98 Model of F0 to estimate speaker range Pitch ranges slopes locations of interest Spectral tilt features Lexical Features Presence and of filled pauses Is this a question A question following a question Presence of pronouns by person case and number A specific denial Presence and of cue phrases Presence of self repairs Presence of contractions Presence of positive negative emotion words Verb tense Presence of yes no not negative contractions Presence of absolutely really Presence of hedges Complexity syls words Number of repeated words Punctuation type Length of unit in sec and words words unit length of laughs of audible breaths of other speaker