From Sounds to LanguageWho studies speech sounds?How do we represent speech sounds?Limits of OrthographyPhonetic Symbol SetsSlide 6Sound CategoriesArticulatory Phonetics: How do people produce speech?Vocal fold vibrationArticulators in actionHow do we capture articulatory data?Classes of SoundsConsonants: Place of ArticulationPlaces of articulationConsonants: Manner of ArticulationSlide 16Slide 17VowelsAmerican English vowel space[iy] vs. [uw][ae] vs. [aa]Acoustic landmarksA Problem: CoarticulationIPA consonantsIPA vowelsRepresentations for SoundsNext ClassFrom Sounds to LanguageCS 4706Julia HirschbergWho studies speech sounds?•Phoneticians: –What distinctive sounds do particular languages have?–How are they produced?•Phonologists:–What is the underlying theory of speech sound?–What explains how particular sounds vary in context?•Acoustic phoneticians, speech engineers, speech pathologists, lexicographers, singers,…How do we represent speech sounds?•Regular orthography•Special-purpose symbol sets•Abstract sound classes based upon sound similarities–What sounds are shared by languages X and Y?–What sounds are unique to particular languages? Or at least rare?–E.g. for language identificationLimits of Orthography•A single letter may have many different acoustic realizations, e.g., in Englisho comb, tomb, bomb oo blood, food, goodc court, center, cheese s reason, surreal, shy•A single sound may have different orthographic correspondences[i] sea, see, scene, receive, thief [s] cereal, same, miss[u] true, few, choose, lieu, do [ay] prime, buy, rhyme, lie•Orthography not a good choicePhonetic Symbol Sets•International Phonetic Alphabet (IPA)–Single character for each sound–Represents all sounds of the world’s languages•ARPAbet, TIMIT, …–Multiple characters for sounds but ASCII–English specific, so new symbol sets for each new language to be representedFigures 4.1 and 4.2:Jurafsky & Martin (2000),pages 94-95.Sound Categories•Phone: Basic speech sound–A minimal sound difference between two words (e.g. too, zoo)–Not every human sound is phonetic, e.g.•Sniffs, laughs, coughs,…•Phoneme: Class of speech sounds–Phoneme may include several phones (e.g. the /t/ in butter, trip, tip, but)•Allophone: set of phonetic variants of a phoneme (e.g. a flapped t is an allophone of /t/)Articulatory Phonetics: How do people produce speech?•General process:–Air expelled from lungs through windpipe (trachea) leaving via mouth (mostly) and nose (nasals) (e.g. [m], [n])–Air passing thru trachea goes thru ‘voice box’ (larynx), which contains vocal cords (vocal folds) – space between them is glottis–When vocal folds vibrate, we get voiced sounds (e.g. [v]); o.w. voiceless (e.g. [f])•The articulatory organsVocal fold vibration[UCLA Phonetics Lab demo]Articulators in action“Why did Ken set the soggy net on top of his deck?”(Sample from the Queen’s University / ATR Labs X-ray Film Database)How do we capture articulatory data?•X-ray/pellet film archive•X-Ray Microbeam Database–Sample output•Electroglottography•Electromagnetic articulography (EMMA)–3 transmitters on helmet produce alternating magnetic fields at different frequencies, forming equilateral triangle–Creates alternating current in 5-15 sensors to calculate sensor positions via XY coordinates–Sample outputClasses of Sounds•Consonants and vowels:–Consonants: •Restriction/blockage of air flow•Voiced or voiceless–Vowels:•Generally voiced, less restriction–Semivowels: [w], [y]Consonants: Place of Articulation•What is the point of maximum restriction?–Labial: bilabial [b], [p]; labiodental [v], [f]–Dental: [], [] thief vs. them–Alveolar: [t], [d], [s], [z]–Palatal: [], [t] shrimp vs. chimp–Velar: [k], [g]–Glottal: [?] glottal stopPlaces of articulationhttp://www.chass.utoronto.ca/~danhall/phonetics/sammy.htmllabialdentalalveolarpost-alveolar/palatalvelaruvularpharyngeallaryngeal/glottalConsonants: Manner of Articulation•How is the airflow restricted?–Stop: [p],[t],[g],…•Airflow completely blocked (closure), then released (release)•Aka plosive–Nasal: air is released thru nose [m],[ng],…–Fricative: [s],[z], [f] air forced thru narrow channel–Affricates [t] begin as stops and end as fricatives–Approximant: [w],[y]•2 articulators come close but don’t restrict much•Between vowels and consonants•Lateral: [l]–Tap or flap: [ ]PLACE OF ARTICULATIONbilabiallabio-dentalinter-dentalalveolar palatal velar glottalstop p b t d k g qfric. f v th dh s z sh zh haffric.ch jhnasal m n ngapprox w l/r yflap dxMANNER OF ARTICULATIONVOICING:voiceless voicedVowels•Vowel height–How high is the tongue? high or low vowel–Where is its highest point? front or back vowel•How rounded are the lips?•Mono vs. diphthong, e.g. [ei]–1 vowel sound or 2?American English vowel spaceFRONT BACKHIGHLOWeyowawoyayiyihehaeaaaouwuhahaxix ux[iy] vs. [uw](From a lecture given by Rochelle Newman)[ae] vs. [aa](From a lecture given by Rochelle Newman)Acoustic landmarks“Patricia and Patsy and Sally”[p] [t] [p] [t][p] [t][l][sh] [s] [s][n] [n][ix][ix] [ih][ih] [ax] [ae] [iy] [iy][ae]A Problem: Coarticulation•Same phone produced differently depending on phonetic context•Occurs when articulations overlap as articulators are moving in different timing patterns to produce different adjacent sounds–Eight vs. Eighth •Place of articulation moves forward as /t/ is dentalized–Met vs. Men•Vowel is nasalizedIPA consonants(Distributed by the International Phonetics Association.)IPA vowels(Distributed by the International Phonetics Association.)Representations for Sounds•Now we have ways to represent the sounds of a language (IPA, Arpabet…) and to classify similar sounds–Automatic speech recognition–Speech synthesis–Speech pathology, language id, speaker id•But…how can we recognize different sounds automatically?–Acoustic analysis and toolsNext Class•Acoustics of speech production (J&M 7.4, *Johnson