4/17/2011 1Spoken Dialogue Systems: Managing InteractionJulia HirschbergCS 47064/17/2011 2Outline• ‘Rules’ of Human-Human Conversation– Turn-taking– Speech Acts– Grounding• Dialogue Management in SDS– Types of Dialogue Management – Varieties of Initiative•VoiceXML4/17/2011 2Speech and Language Processing -- Jurafsky and Martin4/17/2011 3Turn-taking• Dialogue is characterized by turn-taking.–A:–B:–A:–B:–…• Resource allocation problem• How do speakers know when to take the floor?– Total amount of overlap relatively small (5% - Levinson 1983)– But there is very little pause– Must be a way to know who should talk and when4/17/2011 3Speech and Language Processing -- Jurafsky and Martin4/17/2011 4Turn-taking rules• At each transition-relevance place (TRP) of each turn:– a) If during this turn the current speaker has selected B as the next speaker, then B must speak next.– b) If the current speaker does not select the next speaker, any other speaker may take the next turn.– c) If no one else takes the next turn, the current speaker may take the next turn.4/17/2011 4Speech and Language Processing -- Jurafsky and Martin4/17/2011 5Implications of Subrule a• For some utterances, current speaker selects next speaker– Adjacency pairs• Question/answer• Greeting/greeting• Compliment/downplayer• Request/grant• Silence between 2 parts of adjacency pair is different than silence after– A: Is there something bothering you or not?– (1.0)– A: Yes or no?– (1.5)–A: Eh?–B: No.4/17/2011 5Speech and Language Processing -- Jurafsky and Martin4/17/2011 6Speech Acts• Austin (1962): An utterance is a kind of action• Clear case: performatives– I name this ship the Titanic– I second that motion– I bet you five dollars it will snow tomorrow• Performative verbs (name, second, bet…)• Austin’s idea: not just these verbs4/17/2011 6Speech and Language Processing -- Jurafsky and Martin4/17/2011 7Each utterance is 3 acts• Locutionary act: the utterance of a sentence with a particular meaning• Illocutionary act: the act of asking, answering, promising, etc., in uttering a sentence.• Perlocutionary act: the (often intentional) production of certain effects upon the thoughts, feelings, or actions of addressee in uttering a sentence.4/17/2011 7Speech and Language Processing -- Jurafsky and Martin4/17/2011 8Locutionary vs. Illocutionary vs. Perlocutionary• “You can’t do that!”• Illocutionary force:–Protest• Perlocutionary force:– Intent to annoy addressee– Intent to stop addressee from doing something4/17/2011 8Speech and Language Processing -- Jurafsky and Martin4/17/2011 9Illocutionary Acts• How many are there?• What are they?• How do we decide?4/17/2011 9Speech and Language Processing -- Jurafsky and Martin4/17/2011 10Some Ideas from Searle (1975): Speech Acts• Assertives: Commitments by the speaker to something’s being the case– suggesting, putting forward, swearing, boasting, concluding• Directives: Attempts by the speaker to get the addressee to do something– asking, ordering, requesting, inviting, advising, begging• Commissives: Commitments by the speaker to some future course of action– promising, planning, vowing, betting, opposing• Expressives: Expressions of the psychological state of the speaker about a state of affairs – thanking, apologizing, welcoming, deploring• Declarations: Utterances by the speaker that themselves bring about a different state of the world– I resign; You’re fired; I now pronounce you…)4/17/2011 10Speech and Language Processing -- Jurafsky and Martin4/17/2011 11Grounding• Assumption: Dialogue is a collective act performed by speaker (S) and hearer (H)• Common ground: set of things mutually believed by both speaker and hearer• S and H need to achieve common ground to achieve successful communication, so H must ground or acknowledge S’s utterance• Clark (1996):– Principle of closure. Agents performing an action require evidence, sufficient for current purposes, that they have succeeded in performing it– True in HCI as well (Norman,1988)– Need to know whether an action succeeded or failed4/17/2011 11Speech and Language Processing -- Jurafsky and Martin4/17/2011 12Clark and Schaefer: Types of Grounding• Continued attention: B continues attending to A• Relevant next contribution: B starts in on next relevant contribution• Acknowledgement: B nods or says continuer like uh-huh, yeah, assessment (great!)• Demonstration: B demonstrates understanding A by paraphrasing or reformulating A’s contribution, or by collaboratively completing A’s utterance• Display: B displays verbatim all or part of A’s presentation4/17/2011 12Speech and Language Processing -- Jurafsky and Martin4/17/2011 13A human-human conversation4/17/2011 13Speech and Language Processing -- Jurafsky and Martin4/17/2011 14Grounding examples• Display:– C: I need to travel in May–A: And, what day in May did you want to travel?• Acknowledgement– C: He wants to fly from Boston–A: mm-hmm– C: to Baltimore Washington International– [Mm-hmm (usually transcribed “uh-huh”) is a backchannel, continuer, or acknowledgement token]4/17/2011 14Speech and Language Processing -- Jurafsky and Martin4/17/2011 15• Acknowledgement + next relevant contribution–And, what day in May did you want to travel?–Andyou’re flying into what city?–Andwhat time would you like to leave?• The and indicates to the client that agent has successfully understood answer to the last question.4/17/2011 15Speech and Language Processing -- Jurafsky and Martin4/17/2011 16Grounding negative responsesFrom Cohen et al. (2004)• System: Did you want to review some more of your personal profile?•Caller: No.• System: Okay, what’s next?• System: Did you want to review some more of your personal profile?•Caller: No.• System: What’s next?4/17/2011 16Speech and Language Processing -- Jurafsky and Martin4/17/2011 17Grounding and Dialogue Systems• Grounding is not just a useful fact about humans• Key to designing a good conversational agent•Why?4/17/2011 17Speech and Language Processing -- Jurafsky and Martin4/17/2011 18Grounding and Dialogue Systems• Grounding is not just a tidbit about humans• Is key to design of conversational agent•Why?– HCI researchers find users of speech-based interfaces are confused when system doesn’t give them an