Multivariate Gaussian Distribution Leon Gu CSD CMU Multivariate Gaussian p x 1 2 n 2 1 2 1 exp x T 1 x 2 I Moment Parameterization E X Cov X E X X T symmetric positive semi definite matrix I Mahalanobis distance 42 x T 1 x I Canonical Parameterization 1 p x exp a T x xT x 2 where 1 1 a 12 n log 2 log T I Tons of applications MoG FA PPCA Kalman Filter Multivariate Gaussian P X1 X2 P X1 X2 Joint Gaussian 1 2 11 12 21 22 P X2 Marginal Gaussian m 2 2 m 2 2 P X1 X2 x2 Conditional Gaussian 1 2 1 12 1 22 x2 2 1 2 11 12 1 22 21 Operations on Gaussian R V The linear transform of a gaussian r v is a guassian Remember that no matter how x is distributed E AX b AE X b Cov AX b ACov X AT this means that for gaussian distributed quantities X N AX b N A b A AT The sum of two independent gaussian r v is a gaussian Y X1 X2 X1 X2 Y 1 2 Y 1 2 The multiplication of two gaussian functions is another gaussian function although no longer normalized N a A N b B N c C where C A 1 B 1 1 c CA 1 a CB 1 b Maximum Likelihood Estimate of and Given a set of i i d data X x1 xN drawn from N x we want to estimate by MLE The log likelihood function is ln p X N 2 ln N 1 X T xn 2 n 1 1 xn const Taking its derivative w r t and setting it to zero we have N 1 X N n 1 xn Rewrite the log likelihood using trace trick ln p X N P ln 1 xn T 1 xn const N 2 2 n 1 N P ln 1 Trace 1 xn xn T N 2 2 n 1 N P N ln 1 Trace 1 xn xn T 2 2 n 1 1 Taking the derivative w r t and using 1 T A Tr AB A Tr BA B we obtain N 1 X N n 1 A T xn xn log A A T 2
View Full Document
Unlocking...