Embodied Speech and Facial Expression Avatar Implementation Notes Written by Dan Harbin – Brent Sicking – Jaclyn Tech – Evan Zoss- 2 -Table of Contents Project Overview........................................................................................................................... 4 Problem Background and Needs Statement................................................................................ 4 Project Accomplishments ........................................................................................................... 4 System Design Overview .............................................................................................................. 5 System Block Diagram – The Four Parts ................................................................................... 5 System Block Diagram – The Interfaces.................................................................................... 6 Complete System Specifications ..................................................................................................7 Part 1: The Computer................................................................................................................. 7 Part 2: The SV203 Microcontroller ......................................................................................... 18 Part 3: The Yano I/O Board..................................................................................................... 22 Part 4: Yano ............................................................................................................................. 26 Appendices................................................................................................................................... 28 Appendix A: Software Class and Method Outline .................................................................. 28 Appendix B: Hardware Logic Flow......................................................................................... 33 Appendix C: Product Datasheets............................................................................................. 33 Appendix D: Bibliography....................................................................................................... 35- 3 -Table of Figures Figure 1: System Block Diagram................................................................................................... 5 Figure 2: The Computer................................................................................................................. 7 Figure 3: Yano Control Center ...................................................................................................... 8 Figure 4: Parameterized Facial Expressions Screen...................................................................... 9 Figure 5: Manual Motor Control Screen...................................................................................... 10 Figure 6: Sound Processing Screen.............................................................................................. 11 Figure 7: Original Waveform....................................................................................................... 13 Figure 8: Power Waveform.......................................................................................................... 14 Figure 9: Power Waveform Regions............................................................................................ 15Figure 10: The SV203 Microcontroller ....................................................................................... 18 Figure 11: SV203 Pin-out............................................................................................................ 21 Figure 12: Yano I/O Board.......................................................................................................... 22 Figure 13: Yano I/O Circuitry...................................................................................................... 22 Figure 14: Sensor Input Switch ................................................................................................... 23 Figure 15: Yano I/O Board Pin-out ............................................................................................. 25Figure 16: Yano ........................................................................................................................... 26- 4 -Project Overview Problem Background and Needs Statement There has been a great deal of research done in the area of human-robot interaction to understand how a person interacts with a computer. Nonverbal communication, such as facial expression, provides us with characteristics about the expresser and emotion that is otherwise lost in flat speech alone. The ability to generate animated facial expressions together with speech is greatly important to many diverse application areas. For example, a deaf person could use an animated face as a lip-reading system. An autistic child could be positively affected from a robotic face in terms of social interaction, language development, and learning through structure and repetition. Project Accomplishments Our team delivered a computer-controlled robotic head taken from a Yano toy. Our final product contains all the necessary hardware, software, and user interface tools that are needed for its control.The overall goal of this project was to create a robotic face capable of displaying human emotion accompanied with speech. Our final product accomplishes the following objectives: 1. We reverse-engineered Yano’s motors and sensors so we are able to move them to any desired position. 2. We developed a Windows-based graphical user interface that allows the user to move each motor in both directions to a desired position. 3. We researched the psychology behind the use of facial expressions to convey emotion and mimicked these facial expressions with Yano’s face. 4. We developed a graphical user interface that allows the user to select and display real human facial expressions. 5. We developed software to mimic speech with Yano’s mouth based on a measure of intensity from various wave files.- 5 -System Design Overview Figure 1: System Block Diagram The following is a general, functional description of each part and interface in our system. The details for specific software description, circuits, and pin-outs of the parts and the timing and logic of the interfaces are given later in the section