Oct 23 2007 LEC 8 ECON 140A 240A 1 Correlation and Analysis of Variance L Phillips I Introduction Pursuing the results from the ordinary least squares estimates of the linear model from the previous lecture we investigate correlation measures of goodness of fit and analysis of variance Then we turn to issues of central tendency and dispersion for the parameter estimates of the intercept and slope II Correlation Recall that in Lectures Four and Six we introduced the covariance between y and x as a measure of the relationship between these variables E y Ey x Ex Cov yx 1 The covariance depends on units of measurement so it is not a relative measure We can go back to the linear model Eq 4 from the previous lecture and see the relationship between the variance of y and the covariance of y and x yi a b xi ui 2 Taking expectations E y a b E x Eu 3 The mean of the error term E u is zero by assumption and we have seen that the sample mean of the estimated error is zero i e u i i n 0 Subtract Eq 3 from Eq 2 to write in deviation form y Ey b x Ex u 4 Multiply both sides by x Ex and take expectations If the residual u is independent of the explanatory variable which is another assumption of regression analysis then E y Ey x Ex b E x Ex 2 E x Ex u b Var x 5 Oct 23 2007 LEC 8 ECON 140A 240A 2 Correlation and Analysis of Variance L Phillips Solving for the slope b b Cov yx Var x 6 We can estimate the parameter b using the method of moments which substitutes the sample estimates for the population entities of Cov yx and Var x b yi i i yi n xi xi n xi i xi n xi i i xi n 7 This is the same formula as the ordinary least squares solution given by Eq 15 in the previous lecture To see this rewrite Eq 7 above as b yi y xi x i i xi x 2 8 and expanding b yi xi y xi x yi yx i i xi 2 2 x xi x 2 9 and taking summations yi xi y b i i xi x yi n yx xi 2 2 x i i i xi n x 2 10 and collecting terms 2 2 b yi xi n yx xi n x i 11 i and multiplying top and bottom by n n b n yi xi i i yi xi n xi 2 xi 2 i i i 12 the same as Eq 15 as promised We can use Eq 4 to pursue the relationship between variances of y x and u Square both sides of Eq 4 and take expectations Oct 23 2007 LEC 8 ECON 140A 240A 3 Correlation and Analysis of Variance L Phillips E y Ey 2 b2 E x Ex 2 2b E x Ex u E u2 13 Var y b2 Var x Var u 14 or since another assumption of least squares as discussed above is that the explanatory variable x is independent of the error u so their covariance is zero i e E x Ex u 0 Thus the total variance in y can be decomposed into two parts the variance explained by y s dependence on x b2 Var x called the signal and the unexplained variance Var u called the noise Combining Eq s 6 and 14 Var y Cov yx 2 Var x Var u 15 And dividing by the variance of y 1 Cov yx 2 Var xVar y Var u Var y 16 where 1 Var u Var y is one minus the ratio of the unexplained variance to the total variance i e 1 Var u Var y 1 unexplained variance total variance 17 total variance unexplained variance total variance explained variance total variance 18 19 and from Eq 16 this fraction of the total variance that is explained is explained variance total variance Cov yx 2 Var xVar y 20 Note the covariance squared divided by the variance of y and the variance of x cancels out the units of measurement leaving a relative measure called R2 the coefficient of determination This coefficient which measures the fraction of the variance in y explained by dependence on x is consequently a measure of goodness of fit Oct 23 2007 LEC 8 ECON 140A 240A 4 Correlation and Analysis of Variance L Phillips In bivariate regression of y on x the coefficient of determination R2 is just the square of the correlation coefficient r between y and x which is a relative unitless measure of the interdependence of y and x R2 Cov yx Var y Var x r 21 The sample correlation coefficient r can be estimated by the method of moments substituting sample estimates of sums of squares for the covariance and variances y i y x i x n 1 r i n 1 y i y 2 n 1 x i x 2 or y i y x i x r i y i y 2 x i x 2 22 The correlation coefficient ranges between minus one if the correlation is negative and perfect through zero for no correlation and up to one if the correlation is positive and perfect i e 1 r 1 In multivariate regression where y depends on two or more explanatory variables or regressors the estimated coefficient of determination R 2 can be calculated from the sum of squared residuals and the sum of squared deviations of y around its mean 2 u i 2 y y 2 i R 1 i i 23 III Analysis of Variance The results of a regression analysis can be summarized in a table of analysis of variance or ANOVA as depicted in Table 1 Oct 23 2007 LEC 8 ECON 140A 240A 5 Correlation and Analysis of Variance L Phillips Table 1 Table of Analysis of Variance Bivariate Regression Source of Variation Explained by x Sum of Squares b Unexplained 2 i xi x 2 Degrees of Freedom 1 u 2 n 2 n 1 i Total i Mean Square b 2 i xi x 2 1 u 2 n 2 i yi y 2 i yi y 2 n 1 A key to understanding ANOVA is that the total sum of squared deviations of the dependent variable y from its mean can be partitioned into the explained sum of squares and the unexplained sum of squares in a fashion parallel to how we partitioned the population variance To see this combine Eq 6 u i y i y i with Eq 10 y i a b x i both from the previous chapter to obtain u i y i a b x i y a b x 24 u i y i y b x i x 25 Squaring the observed residual and summing u i 2 i y i y 2 b 2 x i x 2 2 b x i x y …
View Full Document
Unlocking...