View Full Document

Response-Based Confidence Annotation for Spoken Dialogue Systems



View the full content.
View Full Document
View Full Document

9 views

Unformatted text preview:

Response Based Confidence Annotation for Spoken Dialogue Systems Alexander Gruenstein Spoken Language Systems Group M I T Computer Science and Artificial Intelligence Laboratory 32 Vassar St Cambridge MA 02139 USA alexgru csail mit edu Abstract Spoken and multimodal dialogue systems typically make use of confidence scores to choose among or reject a speech recognizer s Nbest hypotheses for a particular utterance We argue that it is beneficial to instead choose among a list of candidate system responses We propose a novel method in which a confidence score for each response is derived from a classifier trained on acoustic and lexical features emitted by the recognizer as well as features culled from the generation of the candidate response itself Our responsebased method yields statistically significant improvements in F measure over a baseline in which hypotheses are chosen based on recognition confidence scores only 1 Introduction The fundamental task for any spoken dialogue system is to determine how to respond at any given time to a user s utterance The challenge of understanding and correctly responding to a user s natural language utterance is formidable even when the words have been perfectly transcribed However dialogue system designers face a greater challenge because the speech recognition hypotheses which serve as input to the natural language understanding components of a system are often quite errorful indeed it is not uncommon to find word error rates of 20 30 for many dialogue systems under development in research labs Such high error rates often arise due to the use of out of vocabulary words noise and the increasingly large vocabularies of more capable sys tems which try to allow for greater naturalness and variation in user input Traditionally dialogue systems have relied on confidence scores assigned by the speech recognizer to detect speech recognition errors In a typical setup the dialogue system will choose to either accept that is attempt to



Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view Response-Based Confidence Annotation for Spoken Dialogue Systems and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Response-Based Confidence Annotation for Spoken Dialogue Systems and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?