Speechbuilder TutorialSpeaker Independent; Domain DependentSpeechbuilderGalaxy’s ComponentsSpeechbuilder APIGrammarSpeechbuilder’s GrammarWhat does a rule look like?Example after reductionDomain XML exampleSlide 11What happens to domain XMLImportant stuffMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction SpeechbuilderTutorialMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 2Speaker Independent;Domain DependentWhat is a domain?a vocabulary (words)sentencesHow to define words?English spelling and pronunciation How to define sentences?GrammarMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 3SpeechbuilderGalaxy is the speech recognition systemSpeechbuilder is a tool to develop a domain for galaxyReal speech recognizers take a lot of work and detailed knowledge of all the components.Speechbuilder is great for prototypingMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 4HubSpeechSynthesisSpeechSynthesisLanguageGenerationLanguageGenerationDialogueManagementDialogueManagementContextResolutionContextResolutionLanguageProcessingLanguageProcessingSpeechRecog.SpeechRecog.DatabaseServerDatabaseServerAudioAudioGalaxy’s ComponentsHubSpeechSynthesisSpeechSynthesisLanguageGenerationLanguageGenerationDialogueManagementDialogueManagementContextResolutionContextResolutionLanguageProcessingLanguageProcessingSpeechRecog.SpeechRecog.DatabaseServerDatabaseServerAudioAudioHttpSpeechBuilderServerApplication(Python, Java, ...)Application(cgi)Frame RelayServerTCP SocketMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 5Speechbuilder APIGalaxy Frame RelayPython classgalaxy.server.ServerApplicationPython classgalaxy.frame.Framegalaxy.server.Server methods:Constructor(machine,port,ID)connect()processMessage(blocking)disconnect()galaxy.frame.Frame methods:getAction()getAttribute(attr_name)getText()toString()PythonAPITCPSocketGalaxy meaning representation provided through frame relayApplications connect via TCP socketsAPI provided in Python, Java, PerlGalaxy Frame RelayPython classgalaxy.server.ServerApplicationPython classgalaxy.frame.Framegalaxy.server.Server methods:Constructor(machine,port,ID)connect()processMessage(blocking)disconnect()galaxy.frame.Frame methods:getAction()getAttribute(attr_name)getText()toString()PythonAPITCPSocketMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 6GrammarWhat is a grammar?a set of terminalsA, B, ...a set of rules or productions<nt-1> == B | <nt-2> A<nt-2> == <nt-1> | NULLa sample sentence: B A A A nt-1 --> nt-2 A --> nt-1 A --> nt-2 A A --> nt-1 A A ... Can you explain this to Grandma? would probably use examplesMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 7Speechbuilder’s GrammarAttributes think of them as: terminalsactually, a non-terminal that goes to a terminalFor exampleA set of terminals: lights, microwave, toaster, vcr, tvThese are all “objects”So, “object” would be an attribute Another exampledining room, living room, kitchen“room” is the attributeMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 8What does a rule look like?Speechbuilder calls them “actions”No complicated productionsEach action is an example sentenceSentence containsan “action” terminal zero or more attributesoptional wordsE.g. Turn on the lights“lights” is an example of an “object” attribute“on” is an example of an “onoff” attribute“turn” is an “action”MIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 9Example after reductionAll sentences for action turn What gets sent to applicationMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 10Domain XML example<class name="object" type="Key"> <entry>(television | tv) {television}</entry> <entry>lights</entry> <entry>microwave</entry> <entry>toaster</entry> <entry>v c r {VCR}</entry></class>MIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 11Domain XML example<class name="onoff" type="Key"> <entry>lit {on}</entry> <entry>off</entry> <entry>on</entry></class><class name="turn" type="Action"> <entry>[can you] [please] turn all the lights off</entry> <entry>[can you] [please] turn off all the lights</entry> <entry>[can you] [please] turn off the (living room lights | lights in the living room)</entry> <entry>[can you] [please] turn the (living room lights | lights in the living room) off</entry></class><class name="status" type="Action"> <entry>([can you] [please] tell me | do you know) (what | which) lights are on</entry> <entry>([can you] [please] tell me | do you know) if the (lights in the kitchen | kitchen lights) are on</entry> <entry>(is | are) the (dining room television | tv in the living room) On or Off</entry> <entry>(is | are) the (dining room television | tv in the living room) on</entry></class><class name="good_bye" type="Action"> <entry>good bye</entry> <entry>later</entry></class><class name="room" type="Key"> <entry>dining room</entry> <entry>kitchen</entry> <entry>living room</entry></class>MIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 12What happens to domain XML Compile the domaincheck for errorsCan look at reduced sentencesDON’T click run (it will not work)Can download xml (if you want)Will start galaxy on ocha.csail.mit.edu/usr/sls/Galaxy/users/rudolph/DOMAIN.houseusing command oxclass.cmd yes yes yesstartup Galaudio and python on ipaqMIT 6.893; SMA 5508 Spring 2004 Larry Rudolph Lecture Introduction 13Important stufhttp://ocha.csail.mit.edu/SpeechBuilder/SpeechBuilder.cgiipkg’sgalaudiodoes end of sentence detection (and a little more)sends waveform to galaxyreceives waveform from galaxypython classes for galaxy and xmluse pydoc to get documentation on theseneed to register with frame-relay to get xmlto modify domain (advanced)modify xml of domain, compile, and
View Full Document
Unlocking...