03/28/07 15-494 Cognitive Robotics 1Human-Robot Interaction15-494 Cognitive RoboticsDavid S. Touretzky &Ethan Tira-ThompsonCarnegie MellonSpring 200703/28/07 15-494 Cognitive Robotics 2Human-Robot Interaction Topics●Awareness of humans–Person tracking–Face detection; gaze tracking–Face recognition–Human's “perspective” considerations●Gesture recognition–pointing–hand motions●Social interaction–Gaze as indicator of attention–Facial expressions (e.g., Kismet)–Sound effects (R2D2, AIBO) vs. speech–Use of displays (Looking Glass project)03/28/07 15-494 Cognitive Robotics 3Awareness 1: Person Tracking●Be aware of human presence–Follow a human (robot assistant)–Avoid the humans–Interact with humans (museum tour guide robots)●Use skin color; look for legs (rangefinder); etc.03/28/07 15-494 Cognitive Robotics 4Awareness 2: Face Detection●Rowley, Baluja, and Kanade (1998) used a neural net:(movie)03/28/07 15-494 Cognitive Robotics 5Awareness 3: Gaze Tracking●What is the human looking at?●Gaze has high social significance among primates.●For robots, hard to measure gaze at a distance.03/28/07 15-494 Cognitive Robotics 6Awareness 4: Face Recognition●Which human is this?●Lots of work in this area now for security applications.●Sony's AIBO, QRIO haveface recognition modules.03/28/07 15-494 Cognitive Robotics 7Awareness 5: The Human's Perspective●What can the human see from his present location?●Trafton et al.: “Give me the wrench.”●Robot sees two wrenches, but knows that the human can only see one.03/28/07 15-494 Cognitive Robotics 8Gesture Recognition●Pointing–Point at objects to designate them to the robot–Point in a direction, or towards a goal location●Hand gestures–“Come here” / “Come closer” / “Back off”–“Stop”–“Put that there”03/28/07 15-494 Cognitive Robotics 9Social Interaction●Do robots need heads?●What are heads used for?–Indicate focus of attention by gazedirection–Gestures such as nodding agreement–Anthropomorphism makes robots more acceptable to humans●Headless robots are creepy.DARPA Little Dog03/28/07 15-494 Cognitive Robotics 10Facial Expressions: Kismet●Cynthia Breazeal, ca. 1999-2000http://www.ai.mit.edu/projects/sociable/facial-expression.html03/28/07 15-494 Cognitive Robotics 11Kismet Social Interactions(see movies)03/28/07 15-494 Cognitive Robotics 12Communicating with Humans●Should robots talk?–R2D2 used sound effects to convey emotion–AIBO and Kismet do likewise●Use of canned messages:–“Excuse me, you're blocking my path.”–Roboceptionist: “How may I help you?”●Will people expect to be able to talk back?–Voice recognition gets harder when the robot is noisy.●Use of lights to communicate status, mood.03/28/07 15-494 Cognitive Robotics 13Communication via aDetached Display●AIBO's Magic Looking Glass (Kirtane & Libby, 2005)●Question: how can you use a robot-controlled flat-panel display to mediate human-robot interactions?03/28/07 15-494 Cognitive Robotics 14Looking Glass Applications●Display instructions for a game.●Keep score.●Display a landmark the AIBO can use for navigation.●Display AIBO's view of the world.●Serve as a backdrop for a dramatic presentation:–Display background scenery–Display objects the robot is talking about–Display another agent the robot can interact with03/28/07 15-494 Cognitive Robotics 15Display as Input Device●User points at display to indicate their choice.03/28/07 15-494 Cognitive Robotics 16La Traviata03/28/07 15-494 Cognitive Robotics 17Virtual Violetta(movie)03/28/07 15-494 Cognitive Robotics 18A Visual Joke●At the end of the performance, the user's picture is inserted into an audience shot.03/28/07 15-494 Cognitive Robotics 19How Looking Glass Worksupload image and HTML filesJavaclientrenderer03/28/07 15-494 Cognitive Robotics 20Looking Glass Example#include "Behaviors/BehaviorBase.h"#include "Wireless/LGmixin.h"class LGdemo : public BehaviorBase, public LGmixin {public: LGdemo() : BehaviorBase("LGdemo"), LGmixin() {} void DoStart() { BehaviorBase::DoStart(); erouter->addListener(this,EventBase::textmsgEGID); } void processEvent(const EventBase &) { LGmixin::displayHtmlText("<html><body>Hello world!</body></html>"); }};03/28/07 15-494 Cognitive Robotics 21Looking Glass Functions●uploadFile(string filename)●displayHtmlFile(string remoteFilename)●displayImageFile(string remoteFilename)●displayHtmlText(string text)●uploadCameraImage(string remoteFilename)●uploadSketch(Sketch<uchar> sketch, string