Seminar on Vision and Learning University of California San Diego September 20 2001 Learning and Recognizing Human Dynamics in Video Sequences Christoph Bregler Presented by Anand D Subramaniam Electrical and Computer Engineering Dept University of California San Diego Learning and Vision Seminar Anand D Subramaniam Outline Gait Recognition The Layering Approach Layer One Input Image Sequence Optical Flow Layer Two Coherence Blob Hypothesis EM Clustering Layer Three Simple Dynamical Categories Kalman Filters Layer Four Complex Movement Sequences Hidden Markov Models Model training Simulation results Learning and Vision Seminar Anand D Subramaniam Gait Recognition Running Walking Learning and Vision Seminar Skipping Anand D Subramaniam The Layering Approach Layer 4 Layer 3 Layer 2 Layer 1 Learning and Vision Seminar Anand D Subramaniam Input Image Sequence Layer 1 Feature vector comprises of optical flow color value and pixel value Optical Flow equation I t x y v x y I t t x y 0 Affine Motion Model xs1 1 ys1 2 d x v x y xs ys d 2 1 2 2 y Affine Warp Learning and Vision Seminar s1 2 d x 1 s1 1 S s2 1 1 s2 2 d y Anand D Subramaniam Learning and Vision Seminar Anand D Subramaniam Expectation Maximization Algorithm EM is an iterative algorithm which computes locally optimal solutions to certain cost functions EM simplifies a complex cost function into a bunch of easily solvable cost functions by introducing a missing parameter Missing data is the Indicator Function Learning and Vision Seminar Si y Anand D Subramaniam Expectation Maximization Algorithm EM iterates between two steps E Step Estimate the conditional mean estimate of the missing parameter given the previous estimate of model parameters and the observations M Step Re estimate the model parameters given the soft clustering done by the E Step EM is numerically stable with the likelihood non decreasing with every iteration EM converges to a local optima EM has linear convergence Learning and Vision Seminar Anand D Subramaniam Density Estimation using EM Gaussian mixture models can be used to model any given probability density function to arbitrary accuracy by using sufficient number of clusters curve fitting using Gaussian kernels For a given number of clusters the EM tries to minimize the Kullback Leibler divergence measure between the arbitrary pdf and the class of Gaussian mixture models with the given number of clusters 0 7 0 6 0 5 0 4 0 3 0 2 0 1 0 0 200 Learning and Vision Seminar 400 600 800 1000 1200 Anand D Subramaniam Coherence Blob Hypotheses 2 Likelihood Equation Mixture Model Layer P I t P I t x y x y t x y K P I t x y x y t k t P x y k t P I t x y x y k t k 0 Missing Data Simplified Cost Functions S k t x y P S t x y k I t x y x y t k P x y k t P I t x y x y k t C1 S k t x y log k k x y C2 S k t x y log P x y k t k x y k x y C3 S k x y log P I t x y x y k t Learning and Vision Seminar Anand D Subramaniam EM Initialization We need to track the temporal variation of blob parameters in order to initialize the EM for a given frame Kalman filters Recursive EM using Conjugate priors Learning and Vision Seminar Anand D Subramaniam Learning and Vision Seminar Anand D Subramaniam All Roads Lead From Gauss 1809 since all our measurements and observations are nothing more than approximations to the truth the same must be true of all calculations resting upon them and the highest aim of all computations made concerning concrete phenomenon must be to approximate as nearly as practicable to the truth But this can be accomplished in no other way than by suitable combination of more observations than the number absolutely requisite for the determination of the unknown quantities This problem can only be properly undertaken when an approximate knowledge of the orbit has been already attained which is afterwards to be corrected so as to satisfy all the observations in the most accurate manner possible From Theory of the Motion of the Heavenly Bodies Moving about the Sun in Conic Sections Gauss 1809 Learning and Vision Seminar Anand D Subramaniam Estimation Basics Problem statement Observation Random variable X Given Target Random Variable Y Unknown Joint Probability Density f x y Given What is the best estimate yopt g x which minimizes the expected mean square error between yopt and y Answer Conditional Mean g x E Y X x Estimate g x can be potentially nonlinear and unavailable in closed form When X and Y are jointly Gaussian g x is linear What is the best linear estimate ylin Wx which minimizes the mean square error Learning and Vision Seminar Anand D Subramaniam Wiener Filter 1940 Y Ylin Span X Wiener Hopf Solution W RYX Rxx 1 Involves Matrix Inversion Applies only to stationary processes Not amenable for online recursive implementation Learning and Vision Seminar Anand D Subramaniam Kalman Filter STATE SPACE MODEL Process Model Measurement Model yk 1 Ak yk uk x k M k y k v k The estimate can be obtained recursively Can be applied to non stationary processes If measurement noise and process noise are white and Gaussian then the filter is optimal Minimum variance unbiased estimate In the general case the Kalman filter is the best linear estimator among all linear estimators Learning and Vision Seminar Anand D Subramaniam The Water Tank Problem dL r dt Lt 1 Lt rt 1 rt 1 rt Process Model Measurement Model Lt 1 1 1 Lt uk 1 r 0 1 r u t k 2 t 1 Lt Lt 1 0 vk rt uk vk are zero mean i i d Guassian Learning and Vision Seminar Anand D Subramaniam What does a Kalman filter do The Kalman filter propagates the conditional density in time 0 7 0 7 0 6 0 6 0 5 0 5 0 4 0 4 0 3 0 3 0 2 0 2 0 1 0 1 0 5 4 3 2 1 0 1 f y x1 2 3 4 0 5 5 4 3 2 1 0 1 2 3 4 5 f y x2 0 8 0 7 0 6 0 5 0 4 0 3 0 2 0 1 0 5 Learning and Vision Seminar 4 3 2 1 0 1 f y x1 x2 2 3 4 5 Anand D Subramaniam How does it do it The Kalman filter iterates between two steps Time Update Predict Project current state and covariance forward to the next time step that is compute the next a priori estimates Measurement Update Correct Update the a priori quantities using noisy measurements that is compute the a posteriori estimates y k y k K k xk M k x k Choose Kk to minimize error covariance Learning and Vision Seminar Anand D Subramaniam Applications GPS Satellite …
View Full Document
Unlocking...