PowerPoint PresentationOVERVIEWMOTIVATIONComponents of the recognition systemSlide 5SNAKE MODELMODEL BASED 3-D FACE TRACKINGSlide 8PROBLEM DESCRIPTION(Tracking 1)PROBLEM DESCRIPTION(Tracking 2)PROBLEM DESCRIPTION (Recognition)Slide 12ASSUMPTIONSSlide 14Slide 15Slide 16Slide 17STEREO LEARNINGFACE TRACKINGUNIVERSAL FACE TRACKERSlide 21Slide 22Slide 23RECOGNIZING FACIAL EXPRESSIONS THROUGH TRACKINGSalih Burak GokturkOVERVIEW•MOTIVATION•PREVIOUS WORK•PROBLEM DESCRIPTION•THEORY OF TRACKING•TRACKING VIDEOS•THEORY OF SVMMOTIVATIONWhat does expression recognition mean?Guessing the meaning of facial deformations.What are the possible applications ?- Any interactive scenerio.- Video conferencingWhy do we want to use 3-D information ?- 2-D system is very dependent on the view yet computationally simple.Components of the recognition systemAnalysis -Face Tracking Intelligence-Support Vector Machine ClassifierShape ParametersOVERVIEW•MOTIVATION•PREVIOUS WORK•PROBLEM DESCRIPTION•THEORY OF TRACKING•TRACKING VIDEOS•THEORY OF SVMSNAKE MODEL•Introduced by Kass and Witkin.•Energy Minimization Problem: •Used by Waters and Terzopoulos for tracking.•Snakes are fit to important regions, and tracked from one view to the other.MODEL BASED 3-D FACE TRACKING•DeCarlo and Metaxas, ’96, very accurate tracking.•Eisert and Girod, ’98, used in video-conferencing application, with efficient and accurate compression.•Gokturk et. al., ’00, brings a data driven approach where the face model is learnt from stereo tracking. •Model based approaches.•Track all the points together with n-dimensional freedom on the shape. •Based on Lukas-Tomasi-Kanade optical flow tracker.OVERVIEW•MOTIVATION•PREVIOUS WORK•PROBLEM DESCRIPTION•THEORY OF TRACKING•TRACKING VIDEOS•THEORY OF SVMPROBLEM DESCRIPTION(Tracking 1)?PROBLEM DESCRIPTION(Tracking 2)X(t)I(x(t))I(t+1)TIME t+1?X(t+1)PROBLEM DESCRIPTION (Recognition)X(t)[ Rigid, Open Mouth, Smile]?[ Rigid, Open Mouth, Smile]TrainingData ClassifierTestingNew Data OutputOVERVIEW•MOTIVATION•PREVIOUS WORK•PROBLEM DESCRIPTION•THEORY OF TRACKING•TRACKING VIDEOS•THEORY OF SVMASSUMPTIONS• Cameras are calibrated. • The person should move slow unless the camera is fast enough for motion capture.• The mesh is initialized to the first image.• The user performs the expressions known to the computerPiiiXXX10 p - degrees of freedom- shape is learnt from stereo learning in our caseStereo TrackingDataMonocular TrackingLearn ShapeIl(xi(t))Time t:Il(xi(t+1))Time t+1:? tyxIvuII - For robustness, u and v are estimated using a neighbourhood around the point. LUKAS TOMASI KANADE OPTICAL FLOW TRACKERLUKAS TOMASI KANADE OPTICAL FLOW TRACKER EXTENDED TO 3DX(t)I(x(t))I(t+1)TIME t+1?X(t+1) tyxIvuII PiiiXXX10),,(TRXddTdRdvdTvdRvdudTudRuvuJ   tyxIddTdRJII OVERVIEW•MOTIVATION•PREVIOUS WORK•PROBLEM DESCRIPTION•THEORY OF TRACKING•TRACKING VIDEOS•THEORY OF SVMSTEREO LEARNINGFACE TRACKINGPiiiXXX10- The deformation space of a particular individual is learnt- That particular individual is tracked using a mono camera- Tracked parameters : , R, T.UNIVERSAL FACE TRACKER-The deformation space of a subset of people learnt- The shapes are aligned and PCA is applied on this setOVERVIEW•MOTIVATION•PREVIOUS WORK•PROBLEM DESCRIPTION•THEORY OF TRACKING•TRACKING VIDEOS•THEORY OF SVMSupport Vector Machines (SVM)- Best discriminating hyperplane between two class of objects- Distinguish the vectors that carry the relevant information (support vectors)- if nonlinear data, map the data to high dimensional domain, then apply the SVM.TrainingData ClassifierTestingNew Data OutputExpected Contributions- Show that 3-D model based tracking is suitable for further applications. - Support vector machine is a suitable classifier for expression recognitionWhat needs to be done- Choose the appropriate data input for SVM classifier. Using  vector might not help. Create more intelligent vectors in that case.- Choose an appropriate kernel (transformation function) for SVM. - Apply SVM in a one to many fashion.- Combination of intelligent features and SVM should give robust