Bivariate and Multivariate Normal1BivariateNormalThe bivariate normal pdf is given byf(x, y)=12πσXσY√1−ρ2exp�−12(1−ρ2)��x−µXσX�2+�y −µYσY�2−2ρ�x−µXσX��y − µYσY���(1)It has 5 parameters µX,µY, σ2X, σ2Y, ρ.There is another nice way of expressing this pdf. LetQ =�σ2XρσXσYρσXσYσ2Y�Thenf(x, y)=12π�det(Q)exp�−12(x − µX,y−µY) Q−1(x − µX,y−µY)T�(2)(in the above expression the superscript T means the transpose of the rowvector.) In this formulation (equation 2) the matrix Q is a symmetric positivedefinite 2 by 2 matrix.This bivariate pdf can b e generalized to a multivariate normal pdf. Let xand µ be k dimensional row vectors. x will be the argument of a function andµ will be a (vector) parameter. Let Q be a k ×k positive definite matrix. Thenthe function f : Rk�→R given byf(x)=1(2π)k/2�det(Q)e−12(x−µ)Q−1(x−µ)Tis called the k dimensional multivariate normal probability density function.Equation (2) is of this form with k = 2.1Joint densityBivariate Normal 2Marginal distribution of a bivariate normal.Supp ose X, Y has the bivariate distribution given above.fX(x)=�∞−∞f(x, y)dy=�∞−∞12πσXσY�1 − ρ2exp�−12(1 − ρ2)��x − µXσX�2+�y −µYσY�2−2ρ�x − µXσX��y −µYσY���dy=12πσXσY�1 − ρ2exp�−12(1 − ρ2)�x − µXσX�2�×�∞−∞exp�−12(1 − ρ2)�−2ρ�x − µXσX��y −µYσY�+�y −µYσY�2��dyChange variables u =y −µYσYin this last integral gives�∞−∞exp�−12(1 − ρ2)�−2ρ�x − µXσX��y −µYσY�+�y −µYσY�2��dy= σY�∞−∞exp�−12(1 − ρ2)�−2ρ�x − µXσX�u + u2��du(next complete the square in u)= σY�∞−∞exp�−12(1 − ρ2)�u −�x − µXσX��2�du × exp�ρ22(1 − ρ2)�x − µXσX�2�= σY�2π(1 − ρ2)exp�ρ22(1 − ρ2)�x − µXσX�2�ThusfX(x)=12πσXσY�1 − ρ2exp�−12(1 − ρ2)�x − µXσX�2�×σY�2π(1 − ρ2)exp�ρ22(1 − ρ2)�x − µXσX�2�=1√2πσXexp�−1 − ρ22(1 − ρ2)�x − µXσX�2�=1√2πσXexp�−12�x − µXσX�2�Marginal density of XBivariate Normal 3which is the N(µX, σ2X) density function.By a symmetric argument we findfY(y)=�∞−∞f(x, y)dx=1√2πσYexp�−12�y −µYσY�2�which is the N(µY, σ2Y) density function.Note that in general f(x, y) �= fX(x)fY(y), so that X and Y are dependent.Next consider the case ρ = 0. In this case we have f (x, y)=fX(x)fY(y),so that X and Y are independent. Thus in the normal case we X and Y areindep e ndent if and only if ρ = 0.Later in the course we will seeρσXσY=Cov(X, Y )=E((X − µX)(Y − µY))Next the correlation of X with Y isCorr(X, Y )=Cov(X, Y )σXσY= ρso that ρ is the correlation of X with Y .Marginal density of YCovariance of X and YCorrelationBivariate Normal 42 Conditional distributions of a bivariate nor-mal.The conditional probability density function of Y given X = x is defined forall x ∈R since the marginal distribution of X has supportR. It is given byfY |X=x(y)=f(x, y)fX(x)=√2πσX2πσXσY�1 − ρ2exp�−12(1 − ρ2)��x − µXσX�2+�y −µYσY�2−2ρ�x − µXσX��y −µYσY��+12�x − µXσX�2�=1σY�2π(1 − ρ2)exp�−12(1 − ρ2)��y −µYσY�2+ ρ2�x − µXσX�2−2ρ�x − µXσX��y −µYσY���(now complete the square in y)=1σY�2π(1 − ρ2)exp�−12(1 − ρ2)��y −µYσY�− ρ�x − µXσX��2�=1σY�2π(1 − ρ2)exp�−12σ2Y(1 − ρ2)�y −µY− ρσYσX(x − µX)�2�which is the normal density with mean given by µY+ρσYσX(x − µX) and variancegiven by (1 − ρ2)σ2Y.In particular notice that the conditional mean of Y given X is a linearfunction of X.A second thing to notice is that in the case ρ = 0 this conditional densityis the N (µY, σ2Y) density, the same as the marginal pdf of Y . Thus in the casethat ρ = 0, we have indep endence of Y and X.Conditional distribution of Y given XBivariate Normal 53 Moments for Bivariate Normal DistributionsFirst we calculate the mean and variance for Z ∼ N(0, 1). Let φ be the standardnormal density.E(Z)=�∞−∞xφ(x)dx .Since the integral on the rhs is finite (for example using a comparison testfrom first year calculus), then by symmetry E(Z) = 0. The variance of Z isthen given byVar(Z)=E(Z2)=�∞−∞x2φ(x)dx=1√2π�∞−∞x2e−12x2dx=2√2π�∞0x2e−12x2dx=2√2π�∞02we−w√22w−1/2dw =2√π�∞0w3/2−1e−wdw=2√πΓ�32�=2√π12Γ�12�=1.If W ∼ N (µ, σ2), then we also have the representation W = µ + Zσ,whereZ ∼ N (0, 1). Then by the linearity properties of expectation we haveE(W )=E(µ + σZ)= µVar(Z) = Var(µ + σZ)= σ2Var(Z)= σ2Note these could also be calculated directly from the appropriate integrals. Thestudent should do so at home.Correlation for bivariate normalConsider the bivariate normal distribution (1) with parameters (0, 0, 1, 1, ρ).Supp ose that (X, Y ) has this bivariate normal distribution. Then X and Y bothhave the standard normal distribution as their marginal distributions. In classwe calculated the mean and variance of a N(µ, σ2) distribution.Bivariate Normal 6E(XY )=�∞−∞�∞−∞xy12π�1 − ρ2exp�−12(1 − ρ2)�x2− 2ρxy + y2��dydxWe can evaluate the inner integralA =�∞−∞y12π�1 − ρ2exp�−12(1 − ρ2)�x2− 2ρxy + y2��dy=�∞−∞y12π�1 − ρ2exp�−12(1 − ρ2)�y2− 2ρxy + ρ2x2��dy e−x2(1−ρ2)2(1−ρ2)=1√2π�∞−∞y1�2π(1 − ρ2)exp�−12(1 − ρ2)�(y − ρx)2��dy e−12x2=1√2πρxe−12x2since the integral is the mean of a N(ρx, (1 − ρ2)) distribution.Substituting this into the double (or iterated) integral we obtainE(XY )=�∞−∞1√2πxρxe−12x2dx= ρ�∞−∞1√2πx2e−12x2dx= ρVar(X)= ρsince marginally X ∼ N (0 , 1).Now suppose that (X, Y ) has the bivariate normal distribution (1) of thegeneral form. Consider the transformX1=X − µXσXY1=Y − µYσYThen (X1,Y1) has the bivariate normal distribution with parameters (0, 0, 1, 1, ρ).Thus E(X1Y1)=ρ. AlsoE(X1Y1)=E��X − µXσX��Y − µYσY��From the linearity property of expectation we then obtainCov(X, Y )=E[(X − µX)(Y − µY)]=E(X1Y1)σXσY= ρσXσYBivariate Normal 7Finally we then
View Full Document
Unlocking...