Embodied Speech and Facial Expression Avatar Critical Design Review Dan Harbin Evan Zoss Jaclyn Tech Brent Sicking March 10 2004 Problem Background Needs Statement There has been a great deal of research trying to understand how a person interacts with a computer Messages of the face help illustrate verbal communication by revealing what the expresser is feeling or trying to convey The ability to generate animated facial expressions together with speech is important to many diverse application areas A deaf person could use an animated face as a lipreading system An autistic child could be positively affected from a robotic face in terms of social interaction language development and learning through structure and repetition Goals and Objectives The overall goal of this project is to create a robotic face capable of displaying human emotion accompanied with speech Goals and Objectives Reverse engineer Yano s motors and sensors so we are able to move them to any desired position Develop a GUI that allows the user to move each motor in both directions to a desired position Research the psychology behind the use of facial expressions to convey emotion and mimic these facial expressions with the Yano face Develop a GUI that allows the user to select and display real human facial expressions Add a microphone input in which we will develop software to mimic speech based on a measure of intensity Incorporate facial expressions with speech input to complete the avatar Yano Control System User Interface Flow Chart Note In the future a speech dialog box will be added to the menu User Interface Class Diagram MenuDlg ManualControlDlg CYanoDlg SpeechControlDlg YanoEngine Motor 3 instances CComm ManualControlDlg cpp Class SetProgressValues CheckEndSwitches YanoEngine CalibrateMotor int YanoEngine adjustMotors int destX int destY int destZ Goes back to menu CYanoDlg cpp Class YanoEngine conveyEmotion int destX int destY int destZ YanoEngine adjustMotors int destX int destY int destZ Goes back to menu Command Examples Open Mouth PC5PS6D200PS5PS6 r Close Mouth PS5PC6D200PS5PS6 r Raise Cheeks PS3PC4D200PS3PS4 r Lower Cheeks PC3PS4D200PS3PS4 r Open Mouth and Lower Cheeks PC5PS6PC3PS4D200PS3PS4PS5PS6 r SV203 Microcontroller Circuit Description Motor Control Port Input Port AD1 SV6 AD5 Power SV1 Gnd Vcc Serial Port SV203 Microcontroller Functional Description Receives command through the serial port Set or Clear the appropriate Motor Control Pin s Read an analogue voltage off of the desired Input Pin s Transmit a value representing the voltage back up the serial line SV203 Microcontroller Interface Description Serial Port ASCII text commands are sent to the board via the serial port to tell it what to do Values from the input pins are also sent back to the computer via the serial port List of commands we use SVxM0 initialize pin x to use digital logic PSx set pin x high PCx clear pin x to low Dn delay for n milliseconds before next command PC1PC3PC5D300PS1PS3PS5 typical motor control command ADy read the voltage of input pin y transmit up serial port Motor Control Port sends the logic controls for the motors to the Yano I O Board When a pin is set high with PSx it is set to 6V PCx will set it to 0V We use six pins SV1 through SV6 A D Input Port receives the status of Yano s switches from the Yano I O Board We use 5 pins AD1 through AD5 Each pin will have 6V on it if it s switch is open and near 0V if it is closed The SV203 converts these voltages to the numbers 0 255 for 0V 6V SV203 Microcontroller Logic Description Yano I O Board Switch Circuit Yano SV203 Microcontroller Circuit Description Yano I O Board Functional Description Receives logic controls for the motors from SV203 Converts them into powered control for Yano s motors Reads in status of Yano s switches open or closed Converts this to a voltage 6V for open 0V for closed and sends back to SV203 Yano I O Board Interface Description Motor Control Input the logic input for the H Bridges that determines motor direction and movement They are paired off 2 pins per H Bridge 1 Bridge per motor Mouth SV5 and SV6 Cheeks SV3 and SV4 Eyes SV1 and SV2 Motor Outputs 3 two pin ports one for each motor each pin will have either Vcc or Gnd If both pins are Vcc default state there is no potential between them and the motor will not turn If one pin drops to Gnd the motor will turn one way vice versa for the other pin Sensor Inputs these ports connect directly to Yano s switches Each motor has two limit switches to determine when it runs far enough in each direction Sensor Outputs the interface back to the SV203 that has 5 pins each of which are set to 6V for open switch and 0V for closed switch They are paired off according to which motor they are the limit switches for Mouth AD3 and AD4 Cheeks AD1 and AD2 Eyes AD5 Yano I O Board Logic Description Yano Circuit Description Yano Functional Description Yano has 3 motors powered by the Yano I O Board One for each the mouth one for the cheeks and one to control the eyelids eyebrows and ears When the mouth and cheek motors reach their endpoints ie fully open or fully closed they close a switch to indicate that limit is reached These switches are read by the Yano I O Board Yano Interface Description Yano s interfaces are the motor controls and the switch feedbacks The wires are coded as follows Motors Red Black Eyes SV1 SV2 Green Black Cheeks SV3 SV4 White Black Mouth SV5 SV6 Sensors Red Green Brown Mouth Gnd AD4 AD3 Gray Yellow Pink Cheeks Gnd AD2 AD1 Green Yellow Red Brown Eyes Vcc AD5 Gnd Gnd Yano Logic Description Tasks and Accomplishments Completed Disassemble Yano Reverse engineer motors and end switches Create control circuit Simple computer interface for motor control In Progress Motor calibration Reverse engineer eye motor IR switch To Do Facial expressions Sound analysis software Complete GUI Validation and Testing Procedures Calibration Test Calibrate the motors then run the motors to its limits and back to see if it stays calibrated Expression Test Change from any one expression to any other expression and the face should show the desired expression each time Speech Test Using a sample sound file make sure Yano produces the right mouth movements for the differences in sound volume consistently and accurately Validation and Testing Procedures Complete Project Validation When we are successfully able to calibrate Yano s motors and change between various facial expressions as well as produce mouth