Speech Processing 15-492/18-492Speech TranslationSpeech TranslationThree part systemsThree part systemsASR ASR --> Translation > Translation --> TTS> TTSSystem configurationsSystem configurationsOne way One way ––phrasalphrasalOne way One way ––broadcast/lecturebroadcast/lecture1.5 way 1.5 way ––phrasal with limited answersphrasal with limited answersTwo way Two way ––full two wayfull two wayMachine Translation TechnologiesPhrasalPhrasalPhrase to phrase look upPhrase to phrase look upTemplate:Template:Template fillers, fixed translationTemplate fillers, fixed translationInterlinguaInterlinguaTranslation into meaning representationTranslation into meaning representationStatistical Machine TranslationStatistical Machine TranslationFrom large collect of parallel textFrom large collect of parallel textClassification base translationClassification base translationIdentify classes and deal directly with themIdentify classes and deal directly with themChoices in TranslationChoose any two …Choose any two …High accuracyHigh accuracyLarge vocabularyLarge vocabularyFully automaticFully automaticSpeech Speech vsvsTextTextSpeech less clear than textSpeech less clear than textLess speech to train fromLess speech to train fromNeeds to be realNeeds to be real--time (probably)time (probably)Simple TranslationPhrase to PhrasePhrase to PhraseGreetingsGreetingsDo you need medical attention?Do you need medical attention?Relatively easy to build, but limited useRelatively easy to build, but limited useTemplate translationsTemplate translationsThe next train leaves at TIME from gate The next train leaves at TIME from gate GATEGATEform PLACEform PLACELimited but still usefulLimited but still usefulInterlinguaTranslate sentences into standard formTranslate sentences into standard formGenerate sentences from standard formGenerate sentences from standard formPROS:PROS:Can do multiple languages easilyCan do multiple languages easilyCan be very accurateCan be very accurateCONSCONSDesigning universal interlingua is very hardDesigning universal interlingua is very hardDoesn’t do well when out of domainDoesn’t do well when out of domainStatistical Machine TranslationBuild probabilistic models from parallel textBuild probabilistic models from parallel textParallel text often available fromParallel text often available fromBilingual organizationsBilingual organizationsGovernments, UNGovernments, UNRelatively easy to collect Relatively easy to collect Requires translators rather than MT expertsRequires translators rather than MT expertsLearning from Parallel TextLearning from Parallel TextStatistical Machine TranslationPROSPROSData collection doesn’t require MT expertsData collection doesn’t require MT expertsData drivenData drivenDegrades gracefully when out of domainDegrades gracefully when out of domainCONSCONSNeeds all language pairsNeeds all language pairsNeeds good/lots of dataNeeds good/lots of dataHard to fix specific errorsHard to fix specific errorsSPEECH TranslationSpeech isn’t textSpeech isn’t textDifferent style, hard to find lots of Different style, hard to find lots of exaplesexaplesSpeech isn’t fluentSpeech isn’t fluentFalse starts, hesitations, ungrammaticalFalse starts, hesitations, ungrammaticalASR never makes errors ASR never makes errors ☺☺One Way: Broadcast One speaker One speaker Lecturer: can modify language modelLecturer: can modify language modelMultiple speakersMultiple speakersMay be repeat speakers (News Anchor)May be repeat speakers (News Anchor)May had other noises: music etcMay had other noises: music etc(TV programs)(TV programs)Doesn’t need to be real time (maybe)Doesn’t need to be real time (maybe)Two Way: DialogUsers can detect own errors and correctUsers can detect own errors and correctNeeds to be real timeNeeds to be real timeOne user may be much more familiarOne user may be much more familiarHow do you teach the other userHow do you teach the other userTypically domain directedTypically domain directedSpeech Technology IssuesASR:ASR:DisfluenciesDisfluencies, dialects, speaking style, dialects, speaking styleUnfamiliarity with systemUnfamiliarity with systemTTS:TTS:MT output isn’t always fluentMT output isn’t always fluentTTS says it anywayTTS says it anywayCan be hard to understandCan be hard to understandSpeech Technology IssuesSpoken not Written LanguagesSpoken not Written LanguagesArabic Arabic vsvsArabic DialectsArabic DialectsMixture of languagesMixture of languagesPoliteness levelsPoliteness levelsGender in speechGender in