Speech Processing 15-492/18-492Human Speech ProcessingPhonetics and PhonologyThe vocal tractFrom meat to voiceBlow air through lungsBlow air through lungsVibrate larynxVibrate larynxVocal tract shape defines resonanceVocal tract shape defines resonanceObstructions modify soundObstructions modify soundTongue, teeth, lips, velum (nasal passage)Tongue, teeth, lips, velum (nasal passage)The earFrom sound to brain wavesSound wavesSound wavesVibrate ear drumVibrate ear drumCause fluid in cochlear to vibrateCause fluid in cochlear to vibrateSpiral cochlearSpiral cochlearVibrate hairs inside cochlearVibrate hairs inside cochlearDifferent frequencies vibrate different hairsDifferent frequencies vibrate different hairsConverts time domain to frequency Converts time domain to frequency domainSdomainSFrom grunts to meaningGrunts and vocalizationGrunts and vocalizationLots of variation availableLots of variation available(continuous systems (continuous systems ––not discrete)not discrete)Noises become distinct, recognizableNoises become distinct, recognizableGrow into languages, dialects and idiolectsGrow into languages, dialects and idiolectsWhat are the fundamental units?What are the fundamental units?Articulatory MovementsElectromagnetic ArticulographPhonemesDefined as fundamental units of speechDefined as fundamental units of speechIf you change it, it (can) change the meaningIf you change it, it (can) change the meaning“pat” to “bat”“pat” to “bat”“pat” to ““pat” to “pampam””Vowel Space• One or two banded frequencies (formants)English (US) VowelsfOOlfOOlUWUWfUllfUllUHUHtOYtOY, , OYsterOYsterOYOYlOnelOne, , nOsenOseOWOWbEAtbEAt, , shEEpshEEpIYIYbItbIt, , shIpshIpIHIHgAtegAte, , EIghtEIghtEYEYmakERmakER, , sEARchsEARchERERgEtgEt, , fEAtherfEAtherEHEHhIdehIde, , bUYbUYAYAYAbout, About, cAnoecAnoeAXAXhOWhOW, , sOUthsOUthAWAWlAWnlAWn, , mAllmAllAOAObUtbUt, , hUshhUshAHAHfAtfAt, , bAdbAdAEAEwAshingtonwAshingtonAAAAEnglish ConsonantsStops: P, B, T, D, K, GStops: P, B, T, D, K, GFricatives: F, V, HH, S, Z, SH, ZHFricatives: F, V, HH, S, Z, SH, ZHAffricatives: CH, JHAffricatives: CH, JHNasals: N, M, NGNasals: N, M, NGGlides: L, R, Y, WGlides: L, R, Y, WNote: voiced Note: voiced vsvsunvoiced:unvoiced:P P vsvsB, F B, F vsvsVVNumber of Phonemes in LanguageUS English: 43US English: 43UK English: 44UK English: 44Japanese: 25Japanese: 25Hindi: 81Hindi: 81Numbers aren’t definite thoughNumbers aren’t definite thoughDepends on who you ask,Depends on who you ask,And what you want it forAnd what you want it forNot all variation is PhoneticPhonology: linguistically discrete unitsPhonology: linguistically discrete unitsMay be a number of different ways to say themMay be a number of different ways to say them/r/ trill (Scottish or Spanish) /r/ trill (Scottish or Spanish) vsvsUS wayUS wayPhonetics Phonetics vsvsPhonemicsPhonemicsPhonetics: discrete unitsPhonetics: discrete unitsPhonemics: all soundsPhonemics: all sounds/t/ in US English: becomes “flap”/t/ in US English: becomes “flap”“water” / w “water” / w aoaot t erer//“water” / w “water” / w aoaodxdxerer//Dialect and IdiolectVariation within language (and speakers)Variation within language (and speakers)PhoneticPhonetic“Don” “Don” vsvs“Dawn”, “Cot” “Dawn”, “Cot” vsvs“Caught”“Caught”R deletion (R deletion (HaavaadHaavaadvsvsHarvard)Harvard)Word choice:Word choice:Y’all, Y’all, YinsYinsPoliteness levelsPoliteness levelsNot all languages use the same setAsperatedAsperatedstops (Korean, Hindi)stops (Korean, Hindi)P P vsvsPHPHEnglish uses both, but doesn’t careEnglish uses both, but doesn’t carePot Pot vsvssPotsPot(place hand over mouth)(place hand over mouth)LL--R in Japanese not phonologicalR in Japanese not phonologicalUS English dialects:US English dialects:Mary, Merry, MarryMary, Merry, MarryScottish English Scottish English vsvsUS EnglishUS EnglishNo distinction between “pull” and “pool”No distinction between “pull” and “pool”Distinction between: “for” and “four”Distinction between: “for” and “four”Different language dimensionsVowel lengthVowel lengthBit Bit vsvsbeatbeatJapanese: Japanese: shujinshujin(husband) (husband) vsvsshuujinshuujin(prisoner)(prisoner)TonesTonesF0 (tune) used phoneticallyF0 (tune) used phoneticallyChinese, Thai, BurmeseChinese, Thai, BurmeseClicksClicksXhosaXhosaCo-articulationVoicing actually doesn’t always stopVoicing actually doesn’t always stop“have honey”, “impossible”“have honey”, “impossible”Nasalized voices, lip rounding Nasalized voices, lip rounding “min” “min” vsvs“bit”, “sow” “bit”, “sow” vsvs“see”“see”Lexical stress:Lexical stress:EMphasisEMphasis, , emPHAsisemPHAsisPROjectPROject, , proJECTproJECTReduction, contractionReduction, contraction“A boy is riding a bike”“A boy is riding a bike”“I want to go to Disneyland.”“I want to go to Disneyland.”“I will go tomorrow”“I will go tomorrow”ProsodyIntonationIntonationTuneTuneDurationDurationHow long/short of each phonemeHow long/short of each phonemePhrasingPhrasingWhere the breaks areWhere the breaks areIntonation (F0)Rate of vibration during voiced speechRate of vibration during voiced speechMales: 80Males: 80--140 times a second140 times a secondFemales: 130Females: 130--220 times a second220 times a secondChildren: 180Children: 180--320 times a second320 times a secondUsed for:Used for:EmphasisEmphasisStyle: questions, statements, confidence etcStyle: questions, statements, confidence etcIntonation ContourIntonation InformationLarge pitch range (female)Large pitch range (female)AuthoritiveAuthoritivesince goes down at the endsince goes down at the endNews readerNews readerEmphasis for Finance H*Emphasis for Finance H*Final has a raise Final has a raise ––more information to