Facial Tracking and AnimationProblem BackgroundProblem StatementDesign ObjectivesExisting SolutionsMicrophone Array, Camera, and Capture CardIBM Pupil CamDesign Constraints and FeasibilityAlternative SolutionsDesign ValidationSocietal, Safety and Environmental AnalysisManagementSchedulingScheduling – Gant ChartPert ChartFacial Tracking and AnimationProject ProposalComputer System DesignSpring 2004Todd Belote David BrownBrad Busse Bryan HarrisProblem Background•Speech Driven Facial Animation (PRISM)–Facial animation from processed speech•Previous Research (Jablonski & Zavala)–Low cost facial motion and speech processing using facial markers–An infrared camera is used with infrared reflectors to pick up facial markers–Microphone array for audio–Winnov capture card (640x480 at 30 fps)•Areas of Desired Improvement–There is no current recovery method for point loss–Feature points skewed in perspective create playback artifacts–There are only 22 feature points, which cannot fully describe a face–Initialization requires mouse-clicking the markers on the first frame–The current algorithm is costlyProblem Statement•Design a low cost computer system which can decode and analyze audio/video data in real-time and save the resulting analysis to disk.Design Objectives•Analyze video at 30 fps; generate an FAP(Facial Animation Parameter) file.•Continue with current audio analysis.•All processing done in real-time.Existing Solutions•Lin et al, from National Taiwan University use a system with mirrors and camera to determine (x,y,z) coordinates of feature points•Essa et al, from MIT Media Lab use computer recognition to analyze video in non real-time settingMicrophone Array, Camera, and Capture Card•Audio sample rates range from 8 to 48 kHz•Video sample rate will be 30 fps, and can be captured at 640x480 resolution•Capture card brings audio and video into computer in synchronyIBM Pupil Cam•Camera designed to identify human pupils by emmitting infrared light using LEDs•We will use infrared reflective markers to track facial movementsDesign Constraints and Feasibility•Cost and Speed–The system must run in real-time•Portability–Hardware specific system•Quicktime would make a more portable system•Usability–Point initialization–Freedom of movement for each user–Recovery from point occlusionAlternative Solutions•Leave system as-is•Use mirrors to find (x,y,z) coordinates•Quicktime libraries•Individual facial templatesDesign Validation•Can generate FAP file in real-time•Lost points can be recovered•User can rotate her head without data loss•Audio is analyzed in real-time•Automatic point initializationSocietal, Safety and Environmental Analysis•Primary use as a research tool•Low bandwidth face to face phones using texture maps•Low bandwidth phone support for the deaf•Audio-visual data recordingManagement•Todd Belote – Data Aquisition•David Brown – Marker Initialization•Brad Busse – Marker Tracking Algorithms•Bryan Harris – Facial RelationshipsScheduling•Mondays – 4:20 to 6:30 PM–With Steve Ortiz•Wednesdays – 5:30 to 7:00 PM–With Steve Ortiz and Marco Zavala•Steve Ortiz – Project Advisor•Marco Zavala – Previous Project OwnerScheduling – Gant ChartPert
View Full Document
Unlocking...