Chapter 12Multiple Regression Analysis- The basic ideas are the same as in Chapters 3 & 11.- We have one response (dependent) variable, Y.- The response (Y) is a quantitative variable.- There are more than one predictors (independentvariables): X1, X2, …, Xp where p = number ofpredictors in the model.o The predictors can be: Quantitative (as before) Categorical (new) Interaction terms (product of predictors) Powers of predictors (e.g. 24X).- In this course we will concentrate ono Reading computer outputo Interpreting coefficientso Determining the order to interpret things.Chapter 12, Fall 2007Page 1 of 49Some ExamplesExample – 1: Suppose we want to predict temperaturefor different cities, based on their latitude andelevation.In this case, the response and the predictors areY = temperatureX1 = LatitudeX2 = ElevationPossible models areWith p = 2: 1 1 2 2y x xa b b e= + + + (Stiff surface) With p = 3: 1 1 2 2 3 1 2y x x x xa b b b e= + + + + (Twisted surface)Example – 2: We want to predict patients’ “well-being” from the dosage of medicine they take (mg.)using a quadratic model:21 2( )y x xa b b e= + + +Here X = Dosage of the active ingredient (in mg’s),and p = 2.Chapter 12, Fall 2007Page 2 of 49Example – 3: Suppose we want to predictY = the highway mileage of a car using X1= its city mileage and X2= its size (a categorical variable) where,20 if car is compactX1 if car is larger�=��The model we may use is1 2 2 2 3 1 2( )y x x x xa b b b e= + + + +Note that the last term 3 1 2( )x xbis for interaction whichallows for NON-parallel lines.Chapter 12, Fall 2007Page 3 of 49In general terms:The model:1 2 2 2 p py x x xa b b b e= + + + +LAssumptions:1) - ~ N(0, -) [ Error terms are iid normal withmean zero and constant standard deviation -]. 2) As a result of this, we have, Y ~ N(µY, -), forevery combination of x1, x2, …, xp. That is, theresponse (Y) has a normal distribution with meanµY (that depends on the values of the independentvariables, x’s) and a constant standard deviation,- (that does not depend on the values of X’s).We use data to find theFitted Equation orPrediction Equation1 2 2 2ˆp py a b x b x b x= + + +LChapter 12, Fall 2007Page 4 of 49ANOVA F-test: Overall test of “goodness” of the modelHo: -1 = -2 = -3 = … = -p = 0 NOTHING GOOD in modelHa: at least one of -’s ≠ 0 SOMETHING IS GOOD.Test Statistic: MSRegFMSE=P-Value from the tables of the F-distribution with df1 = p = degrees of freedom of MSReg df2 = n – p – 1 = degrees of freedom of MSE ANOVA for Multiple Regression ModelSource df SS MSE FRegression(Model)p SSRegSSRegMSRegp=MSRegFMSE=Residual(Error)n – p – 1 SSE1SSEMSEn p=- -Totaln – 1 SSTChapter 12, Fall 2007Page 5 of 49Testing for Individual  ’s:Computer output from Minitab:Regression Analysis Y vs. X1, X2, …, XpPredictor Coef SE Coef T P Constant a SE(a) a/SE(a) .X1b1SE(b1) b1 /SE(b1) .X2b2SE(b2) b2 /SE(b2) .MMMMMXpbpSE(bp) bp /SE(bp) .Estimatesof -iSE of theestimate of -iTest Statisticfor Ho: -i = 0vs. Ha: -i - 0p-value(2-sided)Look at the p-value for each - If p-value for -i is small, then Xi is good If p-value for -i is large, then the independentvariable Xi is NOT ADDING any information tothe model AFTER all other predictors are takeninto account.Chapter 12, Fall 2007Page 6 of 49Example - 1: Let Y = height of a person, X1 = Length of right arm, X2 = Length of left arm. 1. Suppose after collecting data we obtained anANOVA table with a small p-value. What does thatmean?2. What is the next step?3. Let’s say you carried out individual t-tests on eachof the slopes, -1 and -2 and found that the p-valuesfor both are large, what does that mean?4. Can you see a contradiction here? 5. When do we get such contradictory results?6. So, when do we have multicollinearity?Chapter 12, Fall 2007Page 7 of 49Example – 2: Suppose we are interesting in predictingthe GPA of students in college (CGPA) using 16different predictor variables. Data were collected froma random sample of 59 college students.1. What is the response variable in this problem?2. What are the values of n and p?3. What are Ho and Ha that you can test using theANOVA table?4. What is your decision, based on the followingANOVA table? What is your conclusion?Analysis of VarianceSource DF SS MS F PRegression 16 3.3135 0.2071 1.99 0.037Residual Error 42 4.3601 0.1038Total 58 7.67365. What is the next step? 6. When do you NOT take the next step?Chapter 12, Fall 2007Page 8 of 49Now look at the following output from Minitab:Regression Analysis: CGPA versus Height, Gender, ... The regression equation isCGPA = 0.53 + 0.0194 Height + 0.047 Gender – 0.00163 Haircut – 0.042 Job + 0.0004 Studytime – 0.375 Smokecig + 0.0488 Dated + 0.546 HSGPA+ 0.00315 HomeDist + 0.00069BrowseInternet– 0.00128 WatchTV – 0.0117 Exercise + 0.0140 ReadNwsP + 0.039 Vegan – 0.0139 PoliticalDeg – 0.0801 PoliticalAff7. Can you make any decisions based on the above? Why or why not?Chapter 12, Fall 2007Page 9 of 498. The following is another part of the Minitab output.Which predictor(s) is/are “good?”Predictor Coef SE Coef T PConstant 0.532 1.496 0.36 0.724Height 0.01942 0.01637 1.19 0.242Gender 0.0468 0.1429 0.33 0.745Haircut – 0.001633 0.001697 –0.96 0.341Job – 0.0418 0.1024 –0.41 0.685Studytime 0.00043 0.01921 0.02 0.982Smokecig – 0.3746 0.2249 –1.67 0.103Dated 0.04881 0.07111 0.69 0.496HSGPA 0.5457 0.1776 3.07 0.004HomeDist 0.003147 0.003400 0.93 0.360BrowseInternet 0.000689 0.001163 0.59 0.557WatchTV –0.0012840 0.0009710 –1.32 0.193Exercise –0.011657 0.005934 –1.96 0.056ReadNewsP 0.01395 0.02272 0.61 0.543Vegan 0.0392 0.1578 0.25 0.805PoliticalDegree –0.01390 0.03185 –0.44 0.665PoliticalAff –0.08006 0.07741 –1.03 0.307S = 0.322198 R-Sq = 43.2% R-Sq(adj) = 21.5%Chapter 12, Fall 2007Page 10 of 499. The following is the last part of the output. What does it tell us?Unusual ObservationsObs Height CGPA Fit SE Fit Residual St Resid 28 67.0 2.9800 3.5898 0.2442 –0.6098 –2.90R 40 65.0 3.9300 3.3458 0.2176 0.5842 2.46R 59 62.0 2.5000 3.4718 0.1352 –0.9718 –3.32RR denotes an observation with a large standardized residual.Although the individual t-tests indicate that the GPAof the student in high school (HSGPA) and Exercisehave coefficients (-i)