Word Pronunciation Julia Hirschberg CS 4706 01 14 2019 1 Today Motivation Challenges for automatic word pronunciation Standard methods Innovative solutions 01 14 2019 2 TTS demos AT T Cepstral SNL Robot Repair 01 14 2019 3 Motivation Intelligibility Naturalness Applications to language learning Unlimited vocabulary Type a word or phrase and hear it spoken in your target language To imitate To learn to recognize Speech therapy 01 14 2019 4 Word Pronunciation What determines how a word is pronounced History Language Origin Dictionaries shoe ME shoo phoenix Gr mole attaches resume Part of speech use close dove multiply coax Morphology ferryboat ferryboats Popemobile pope mobile 01 14 2019 5 Letter2Sound Rules Define correspondences between orthography and phonemic representation e g i C e ai rise Else i ih rip Deals with any input 01 14 2019 6 Problems with Rule Based Systems Must be built by hand Many exceptions e g i C e ai matches magazine crevice beige Proper names Nice Rise Symbols and abbreviations c evalu8 cu tsp Assigning lexical stress is also tricky Rules to decide primary stress Monotone monotony monolingual montage Solutions More complex rules Consult exceptions dictionary first But how do we handle morphological 01 14 2019 variation E g Rise s hat 7 Dictionary based Approaches Rely on very large dictionary with orthography and pronunciation for each word Typically created by hand or by expansion of online pronouncing dictionary 01 14 2019 8 Problems Redundancy of representation Cat cats cat s cats Out of vocabulary OOV items Proper names covering all U K surnames requires 5 000 000 entries New words Technical terms liposuction anova bernaise Foreign borrowings frappe ciao louche 01 14 2019 9 Solutions Even larger dictionary Morphological preprocessing before dictionary look up Fall back to Letter2Sound rules if no dictionary hit 01 14 2019 10 Major Challenges for TTS Disambiguating homographs bass bass Pronouncing new words New names in the news New words iPad Kindle Expanding abbreviations and pronouncing acronyms correctly 01 14 2019 11 Homograph Disambiguation by Decision List Classifiers Yarowsky 97 E g bass bass nice Nice live live desert desert lead lead 01 14 2019 12 Decision List for Bass Bass Can be treated as a case statement Learning Decision Lists Restrict lists to rules that test a single feature Evaluate each possible test and rank them based on how well they classify Order the top N tests as the decision list Yarowsky s Metric On a binary homonymy distinction used the following metric to rank the tests P Sense 1 Feature log P Sense 2 Feature This gives about 95 on bass Semi supervised Extensions Bootstrap from decision list built from small amount of labeled data Classify unlabeled data Create decision list from new labeled corpus 01 14 2019 16 Bootstrapping Phonetic Lexicons Maskey et al 04 For some languages online pronouncing lexicons exist but for Low Resource Languages e g Nepali How to minimize effort in creating lexicons Approach Given a native speaker and a large amount of online text in the language Native speaker builds small lexicon by hand for seed set of N most common words in text e g is izh the dhax 01 14 2019 17 Automatically derive L2S rules from lexicon e g is ih zh the dh ax Loop Choose the next N most common words from the corpus and use lexicon L2S rules to predict pronunciations e g telephone telaxfown He hax Rise rihzhax Assign a confidence score to each prediction by comparing each word to all words in lexicon If is ihzh in lexicon and no other orthographically similar words are pronounced differently then new rule his hihzh scores high 01 14 2019 18 For low confidence pronunciations Active Learning step Inspect and calculate error rate Hand correct errors and add all to lexicon Iterate from Loop until performance stabilizes Results English 94 success on test set after 23 iterations 16K entry lexicon Performance comparable to CMUDict and 1 7 the size German 90 accuracy after 13 iterations 28K lexicon Nepali 94 6 accuracy after 16 iterations 5K lexicon 01 14 2019 19 Pronouncing OOV Words Techniques for handling OOVs Inferring country of origin Takashita Leroy Kirov Lima Infiniti Pronunciation by analogy Analog dialog Risible visible Proper names Alifano Califano AT T Cepstral 01 14 2019 20 Improving Pronunciation Dictionary Coverage Fackrell and Skut 04 Idea Many proper names have more than one spelling e g More Moore Smith Smythe Homophones different spelling same pronunciation 1 Identify spelling alternations that are pronunciationneutral in an existing lexicon I e homophones right write Wright Produce rewrite rules e g wr r ight ite 2 Find a fuzzy mapping between new OOV words and words already in the lexicon using these rules e g Wrothman 01 14 2019 21 Pros Cons 01 14 2019 22 Deriving Pronunciations from the Web Ghoshal et al 09 Extract candidate orthography pronunciation pairs ad hoc and IPA E g bruschetta pronounced broo SKETuh Validate the candidates how likely are these pairs to represent a word and its pronunciation Normalize ad hoc and IPA pronunciations 01 14 2019 23 Pros Cons 01 14 2019 24 Pronunciation Evaluation How would you evaluate the pronunciation module of a TTS system 01 14 2019 25 Next Class Readings Download the ToBI cardinal examples see http www1 cs columbia edu agus tobi for instructions You will first need to download WaveSurfer http www speech kth se wavesurfer Then download the cardinal examples http www1 cs columbia edu agus tobi cardinals manual php Follow the installation directions carefully Listen to the cardinal examples for the following and see if you can hear the differences H H L H H H L H H H H H 01 14 2019 26