Facial Tracking and AnimationProblem BackgroundGoals and ObjectivesSystem FunctionalitySystem DescriptionData AcquisitionPoint Initialization and TrackingPoint TransformPoint Transform: DemoFAP Generation and Audio ProcessingEnvironmental and Health ConsiderationsSocial, Political and Ethical ConsiderationsEconomics and SustainabilityThanks For the ContributionsMay 10, 2004 Facial Tracking and AnimationFacial Tracking and AnimationTodd BeloteBryan HarrisDavid BrownBrad BusseMay 10, 2004 Facial Tracking and AnimationProblem Background•Speech driven facial animation•Correlate captured facial movements to audio patterns–Capture facial movements–Analyze corresponding audioMay 10, 2004 Facial Tracking and AnimationGoals and Objectives•Develop an inexpensive, robust real-time system to track facial motion and process corresponding audio. The system must:–Cost around $1000–Run on a personal computer–Allow for long periods of data acquisition–Handle head movements–Recover from point occlusion–Output only necessary informationMay 10, 2004 Facial Tracking and AnimationSystem Functionality•Capture Live Data•Record Video•Capture From Video•Generate FAP from output file•Process audio of .wav fileMay 10, 2004 Facial Tracking and AnimationSystem DescriptionDATA ACQUISITIONFAP GENERATIONPOINT INITIALIZATIONPOINT TRACKINGAUDIO PROCESSING•Top level system organization–Illustrates data flow–Functional block divisionMay 10, 2004 Facial Tracking and AnimationData AcquisitionVideo Data – 320x240 at 30 fps (bmp)Audio Data – 16 bit at 16kHz (wav)CAMERAMICROPHONEAVI MOVIE FILEEHEHAUDIO PROCESSINGVIDEO PROCESSINGFRAMEGRABBERSW TIMERCAPTURE CARDWAV FILEMay 10, 2004 Facial Tracking and AnimationPoint Initialization and Tracking•Process frame data–Locate all areas in the frame which constitute a point•Identify points–Initialization – algorithm to determine identity–Tracking – algorithm to match current frame points with previous location of identified pointsMay 10, 2004 Facial Tracking and AnimationPoint TransformApproach: Criminisi et al.•Maps any arbitrary quadrilateral onto any other•This can account for all six degrees of freedom as well as perspective distortion, greatly simplifying the computation required to reorient the face•When using an orientation square that encompasses most of the face, this algorithm can be made as accurate as necessaryMay 10, 2004 Facial Tracking and AnimationPoint Transform: DemoActual Camera Data Re-oriented Processed DataMay 10, 2004 Facial Tracking and AnimationFAP Generation and Audio Processing•FAP Generation–Process text file of point movement–Output FAE Engine compliant file of point displacements•Audio Processing–LPC–MFCC–Pitch–PowerMay 10, 2004 Facial Tracking and AnimationEnvironmental and Health Considerations•All hardware is off the shelf•No harm from infrared light•No harm from other products–Eg. Reflective markersMay 10, 2004 Facial Tracking and AnimationSocial, Political and Ethical Considerations•Provide low cost audiovisual capture–Increase research in field by removing cost barrier–Further advances•Eg. Phone for the deaf•No Ethical Issues•No Political affectsMay 10, 2004 Facial Tracking and AnimationEconomics and Sustainability•No economies of scale due to narrow scope•IBM PupilCAM is hard to locate and therefore sustainability with current hardware is issue–Other cameras could provide the same functionMay 10, 2004 Facial Tracking and AnimationThanks For the
View Full Document
Unlocking...