Speech Processing 15-492/18-492Spoken Dialog SystemsTree based dialogsVoiceXMLState-based DialogsSimple stateSimple state--based dialog systemsbased dialog systemsGet NameGet NameGet Account numberGet Account numberGet PinGet PinPresent balancePresent balanceGo back to start or exitGo back to start or exitState-based DialogsGet Name:Get Name:What is your name?What is your name?ASR NameASR NameMay be correct (in the database)May be correct (in the database)May be unknown (not in database)May be unknown (not in database)May not be name (What do I say?/Help/Repeat)May not be name (What do I say?/Help/Repeat)Should you echo the recognized name?Should you echo the recognized name?Confirmation (or not)Confirmation (or not)State-based dialogGet nameGet nameCheck in databaseCheck in databaseAsk again if notAsk again if notDeal with helpDeal with helpGet account numberGet account numberCheck in database (with name)Check in database (with name)Confirm account number and nameConfirm account number and nameFor securityFor securityState-based InteractionTrees can get very largeTrees can get very largeUser can get lost easilyUser can get lost easilyYou want to minimize the number of turnsYou want to minimize the number of turnsFaster throughput means more callsFaster throughput means more callsFaster throughput means happier customerFaster throughput means happier customerThe level of helpFirst time users *need* a successful callFirst time users *need* a successful callOtherwise, they wont call backOtherwise, they wont call backHaving very helpful prompts is goodHaving very helpful prompts is goodAt start, gets annoying quicklyAt start, gets annoying quicklyDesigning prompts is a craftDesigning prompts is a craftWhat should say that is understoodWhat should say that is understoodHow much should you tailor it to the userHow much should you tailor it to the userVoiceXMLA W3C standard for voice browsingA W3C standard for voice browsingXML based “programming” language for XML based “programming” language for speechspeechOutput synthesized (and recorded) speechOutput synthesized (and recorded) speechRecognition of speech and DTMFRecognition of speech and DTMFRecording of spoken inputRecording of spoken inputTelephony featuresTelephony featuresVoiceXMLASRASRFrom Grammars (JSGF)From Grammars (JSGF)From triFrom tri--gramsgramsFrom “Domain Managers” From “Domain Managers” Credit card numbersCredit card numbersCity, StatsCity, StatsVoiceXMLTTSTTS<<ssmlssml> markup> markupChoice of voiceChoice of voiceChoice of languageChoice of languageChoice of how to pronounce thingsChoice of how to pronounce thingsSpecify breaks, timing emphasisSpecify breaks, timing emphasisStructure<<vxmlvxmlversion="1.0">version="1.0"><meta name="author" content="John Doe"/><meta name="author" content="John Doe"/><<varvarname="hi" name="hi" exprexpr="'Hello World!'"/>="'Hello World!'"/><form><form><block><block><value <value exprexpr="hi"/>="hi"/><<gotogotonext="#next="#say_goodbyesay_goodbye"/>"/></block></block></form></form><form id="<form id="say_goodbyesay_goodbye">"><block><block>Goodbye!Goodbye!</block></block></form></form></</vxmlvxml>>Basic Tags<form id=“<form id=“xxxxxxxx”>”><<gotogotonext=“#xxx”>next=“#xxx”><field> gather info from user through <field> gather info from user through speech or DTMFspeech or DTMF<record> <record> recordrecorddata userdata user<<subdialogsubdialog> performs some sub dialog> performs some sub dialog<field> tag<form id=“<form id=“getBusNumbergetBusNumber”>”><field name=“<field name=“BusNumberBusNumber”>”><prompt>Which bus line do you want?</prompt><prompt>Which bus line do you want?</prompt><grammar <grammar srcsrc=“grams/=“grams/bus.grambus.gram”>”><help> Please say you desired bus number, e.g. <help> Please say you desired bus number, e.g. 61C</help>61C</help></field></field></form></form>Flow of ControlGotoGoto<<gotogotonext=“#next=“#GetBusNumberGetBusNumber>><<gotogotonext=“next=“Trains.vxmlTrains.vxml”>”><if <if condcond=“=“BusNumberBusNumber== ‘501”>== ‘501”><prompt> Sorry that bus no longer runs</prompt><prompt> Sorry that bus no longer runs</prompt><<elseifelseifcondcond=“=“BusNumberBusNumber== ’56U”>== ’56U”><prompt> Sorry it’ll be a long wait </prompt><prompt> Sorry it’ll be a long wait </prompt><else /><else /><prompt> One will be along shortly </prompt><prompt> One will be along shortly </prompt></if></if>Variables<<varvarname=“var1” name=“var1” exprexpr=“hello”>=“hello”><prompt I just wanted to say <value <prompt I just wanted to say <value exprexpr=“var1”> </prompt>=“var1”> </prompt><assign name=“var1” <assign name=“var1” exprexpr=“goodbye”>=“goodbye”>Recognition GrammarsSpeech Recognition Grammar SpecificationSpeech Recognition Grammar Specification(SRGS)(SRGS)Augmented BNFAugmented BNF$order = I would like a $drink$order = I would like a $drink$drink = coke | $drink = coke | pepsipepsi| | mountain_dewmountain_dewVoiceXML BrowsersCompatibilityCompatibilityNot as compatible as one would likeNot as compatible as one would like<objects> can be different (but useful)<objects> can be different (but useful)City, State recognizersCity, State recognizersECMAscriptECMAscript((JavascriptJavascript))Beyond VoiceXML(in VoiceXML)Mixing html/Mixing html/cgicgiscripts in scripts in VoiceXMLVoiceXMLUse Use phpphpto generate to generate VoiceXMLVoiceXMLfilesfilesUse Use urlsurls(with ?...) to calculate/get data(with ?...) to calculate/get datahttp://weather.com?zip=“15213http://weather.com?zip=“15213””Use Use urlsurlsto get waveformsto get waveformshttp://tts.com?text=“Hellohttp://tts.com?text=“HelloWorld”World”VoiceXML futureNN--gram grammar Markup Languagegram grammar Markup LanguageMany browsers hove own extensionsMany browsers hove own extensionsPronunciation Lexicon Markup LanguagePronunciation Lexicon Markup LanguageA way to add new items to the lexiconA way to add new items to