View Full Document

3 views

Unformatted text preview:

USER COME BACK Maxine Eskenazi Language Technologies Institute CMU June 16 1999 What is the goal Carnegie Mellon Assessment differs when the application becomes real Then you can based it on the volume of calls As long as we do not make real reservations is user satisfaction valid Users can point to places hotspots where they had a hard time What do they remember that would make them call or not call again We would want the system with the least hotspots March 31 2004 2 System hotspots Carnegie Mellon What do you remember about the last system you called It kept repeating the information about one hotel Why was it irritating I had to repeat myself The system asked the same thing over and over I told it Pittsburgh and then it asked where I wanted to go It wasn t talking about the same thing I was March 31 2004 3 Measures Carnegie Mellon label turns by subgoal departure time hotel location sr no syst repeats ur no user repeats T no turns us user then syst said same thing sn syst subgoal user subgoal repeats R sr ur 2T 2 per goal I had to repeat myself The system asked the same thing over and over shadows S us T 1 per goal I told it Pittsburgh and then it asked where I wanted to go system user N sn T It wasn t talking about the same thing I was March 31 2004 4 System Assessment Carnegie Mellon Compare semi automatic measures to what users say Panel of 9 callers 9 different scenarios 3x3 different difficulty levels easy 1 leg med 2 legs hard 3 legs each caller did 1 easy 1 medium 1 hard March 31 2004 5 Repetitions questions Carnegie Mellon Tell me where in the dialogue Something didn t go as you thought it should You had to change what you wanted You wanted to give up You had no idea what to do next March 31 2004 6 Comparative results Carnegie Mellon For 20 dialogues user label agreement highest number of turns 65 highest percent of repeats R 85 highest percent of system shadows S 60 highest percent of system user N 55 S seems to refine the information that is in R



Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view USER, COME BACK!! and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view USER, COME BACK!! and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?