252mreg doc 1 22 07 Open this document in Outline view Roger Even Bove J MULTIPLE REGRESSION 1 Two explanatory variables a Model Let us assume that we have two independent variables so that Y j represents the jth observation on the dependent variable and X ij is the jth observation on independent variable i For example X 15 is the 5th observation on independent variable 1 and X 29 is the 9th observation on independent variable 2 We wish to estimate the coefficients 0 1 and 2 of the presumably true Yj may not be precisely on the regression j where j is a random variable usually assumed to regression line Y 0 1 X 1 2 X 2 Any actual point line so we write Y j 0 1 X 1 j 2 X 2 j be N 0 and is unknown but constant The line that we estimate will have the equation Y b0 b1 X 1 b2 X 2 Our prediction of X 1 j and X 2 j will be Y j and since Y j is unlikely to equal Y j exactly we call our error in estimating Y j e j Y j Y j so that Y j b0 b1 X 1 j b2 X 2 j e j Y for any specific b Solution After computing a set of six spare parts put them together in a set of Simplified Normal Equations X 1Y nX 1Y b1 X 12 nX 12 b2 X 1 X 2 nX 1 X 2 X 2Y nX 2 Y b X 1 1X 2 nX 1 X 2 b X 2 2 2 nX 22 and solve them as two equations in two unknowns for b1 and b2 and then get b0 by solving b0 Y b1 X 1 b2 X 2 c Example Recall our original example Y is the number of children actually born and X is the number of children wanted Add a new independent variable W a dummy variable indicating the education of the wife W 1 if she has finished college W 0 if she has not In the above equations X X 1 and W X 2 i Y X W 1 0 0 1 0 2 2 1 0 1 3 1 2 1 4 4 3 1 0 1 5 1 0 0 0 6 3 3 0 9 7 4 4 0 16 8 2 2 1 4 9 1 2 1 4 10 2 1 0 1 sum 19 16 4 40 2 X 2 W XW 1 0 1 0 0 0 0 1 1 0 4 0 0 2 0 0 0 0 2 2 0 6 0 2 2 3 0 9 16 4 2 2 40 XY WY Y 2 0 0 0 4 1 1 0 9 0 1 0 9 0 16 2 4 1 1 0 4 4 49 1 Copy sums n 10 Y 19 X 1 X 16 X W 4 XW 6 2 X 40 X W 4 X X X Y XY 40 X Y WY 4 and Y 49 Y X 16 The compute means Y 19 1 90 X X 1 60 X 12 2 2 2 2 1 and X 2 W 2 W Y n 2 nY 1 10 n 1 1 2 2 2 1 2 1 2 2 2 2 1X 2 10 4 0 40 10 49 10 1 9 2 12 90 SST SS Y 2 X Y nX Y 40 10 1 6 1 9 9 60 S X Y nX Y 4 10 0 4 1 9 3 60 S X nX 40 10 1 60 14 40 SS X nX 4 10 0 4 2 40 SS X 2 2 n Spare Parts 1 X 1Y 2 X 2Y X1 2 X2 nX 1 X 2 6 10 1 6 0 4 0 40 S X 1X 2 Note that SS Y SS X 1 and SS X 2 must be positive while the other sums can be either positive or negative Also note that df n k 1 10 2 1 7 k is the number of independent variables SST is used later Rewrite the Normal Equations to move the unknowns to the right X Y nX Y X X Y nX Y X 1 2 1 2 1 2 nX 12 b1 1X 2 X X nX X b b X nX b nX 1 X 2 Y b0 X 1b1 X 2 b 2 1 1 2 1 2 2 2 2 Eqn 1 2 2 2 Eqn 2 Eqn 3 Or S X 1Y SS X 1 b1 S X 1 X 2 b2 Eqn 1 S X 2Y S X 1 X 2 b1 SS X 2 b2 Eqn 2 Y b0 X 1b1 X 2 b2 Eqn 3 If we fill in the above spare parts we get 9 60 14 40b1 0 40b2 3 60 0 40b1 2 40b2 1 90 b0 1 60b1 0 40b2 Eqn 1 Eqn 2 Eqn 3 We solve the first two equations alone by multiplying one of them so that the coefficients of b1 or b2 are of equal value We then add or subtract the two equations to eliminate one of the variables In this case note that if we multiply equation 1 by 6 the coefficients of b2 in Equations 1 and 2 will be equal and opposite so that if we add them together b2 will be eliminated 2 57 60 86 40b1 3 60 54 00 0 40b1 2 40b2 86 00b1 b1 6 Eqn 1 Eqn 2 2 40b2 But if 86b1 54 00 then 54 0 62791 86 Now solve either Equation 1 or 2 for b2 If we pick Equation 1 we can write it as 0 40b2 9 60 14 40b1 We can solve this for b2 by dividing through by 0 40 so that b2 24 0 36 0b1 If we substitute in b1 0 62791 we find that b2 24 0 36 0 0 62791 1 3956 Finally rearrange Equation 3 to read b0 1 90 1 60b1 0 40b2 1 90 1 60 0 62791 0 40 1 3956 1 4536 Now that we have values of all the coefficients our regression equation Y b0 b1 X 1 b2 X 2 becomes Y 1 4536 0 6279 X 1 3956 X or Y 1 4536 0 6279 X 1 1 3956 X 2 e 1 2 2 Interpretation 3 Standard errors Y Recall that in the example in J1 SST 2 nY 2 SS Y 12 90 and that we had computed Spare Parts S X 1Y 9 60 S X 2Y 3 60 SS X 1 14 40 SS X 2 2 40 and S X 1 X 2 0 40 The explained or regression sum of squares is SSR b1 X Y nX Y b X 1 1 2 2Y nX 2 Y b1 S X 1Y b2 S X 2Y The error or residual sum of squares is SSE SST SSR k 2 is the number …
View Full Document
Unlocking...