MulticollinearityMulticollinearityExample #1Slide 4What is effect on regression analyses if predictors are perfectly uncorrelated?Regress Y on X1Regress Y on X2Regress Y on X1 and X2Regress Y on X2 and X1If predictors are perfectly uncorrelated, then…Slide 11Same effects for “real data” with nearly uncorrelated predictors?Regress BP on StressRegress BP on BSARegress BP on BSA and StressRegress BP on Stress and BSAIf predictors are nearly uncorrelated, then…What happens if the predictor variables are highly correlated?Regress BP on WeightSlide 20Regress BP on BSA and WeightRegress BP on Weight and BSAEffect #1 of multicollinearityEven correlated predictors not in the model can have an impact!Effect #2 of multicollinearityEffect #3 of multicollinearityWhat is the effect on estimating mean or predicting new response?Effect #4 of multicollinearity on estimating mean or predicting YWhat is effect on tests of individual slopes?Slide 30Slide 31Effect #5 of multicollinearity on slope testsSummary commentsSummary comments (cont’d)Diagnosing multicollinearityMulticollinearityMulticollinearity•Multicollinearity (or intercorrelation) exists when at least some of the predictor variables are correlated among themselves.•In observational studies, multicollinearity happens more often than not.•So, we need to understand the effects of multicollinearity on regression analyses.Example #112011053.2547.7597.32589.3752.1251.8758.2754.42572.565.512011076.2530.7553.2547.7597.32589.3752.1251.8758.2754.42572.565.576.2530.75BPAgeWeightBSADurationPulseStressn = 20 hypertensive individualsp-1 = 6 predictor variablesExample #1 BP Age Weight BSA Duration PulseAge 0.659Weight 0.950 0.407BSA 0.866 0.378 0.875Duration 0.293 0.344 0.201 0.131Pulse 0.721 0.619 0.659 0.465 0.402Stress 0.164 0.368 0.034 0.018 0.312 0.506Blood pressure (BP) is the response.What is effect on regression analyses if predictors are perfectly uncorrelated? x1 x2 y 2 5 52 2 5 43 2 7 49 2 7 46 4 5 50 4 5 48 4 7 44 4 7 43Pearson correlation of x1 and x2 = 0.000The regression equation is y = 48.8 - 0.63 x1Predictor Coef SE Coef T PConstant 48.750 4.025 12.11 0.000x1 -0.625 1.273 -0.49 0.641Analysis of VarianceSource DF SS MS F PRegression 1 3.13 3.13 0.24 0.641Error 6 77.75 12.96Total 7 80.88Regress Y on X1Regress Y on X2The regression equation is y = 55.1 - 1.38 x2Predictor Coef SE Coef T PConstant 55.125 7.119 7.74 0.000x2 -1.375 1.170 -1.17 0.285Analysis of VarianceSource DF SS MS F PRegression 1 15.13 15.13 1.38 0.285Error 6 65.75 10.96Total 7 80.88Regress Y on X1 and X2The regression equation is y = 57.0 - 0.63 x1 - 1.38 x2Predictor Coef SE Coef T PConstant 57.000 8.486 6.72 0.001x1 -0.625 1.251 -0.50 0.639x2 -1.375 1.251 -1.10 0.322Analysis of VarianceSource DF SS MS F PRegression 2 18.25 9.13 0.73 0.528Error 5 62.63 12.53Total 7 80.88Source DF Seq SSx1 1 3.13x2 1 15.13Regress Y on X2 and X1The regression equation is y = 57.0 - 1.38 x2 - 0.63 x1Predictor Coef SE Coef T PConstant 57.000 8.486 6.72 0.001x2 -1.375 1.251 -1.10 0.322x1 -0.625 1.251 -0.50 0.639Analysis of VarianceSource DF SS MS F PRegression 2 18.25 9.13 0.73 0.528Error 5 62.63 12.53Total 7 80.88Source DF Seq SSx2 1 15.13x1 1 3.13If predictors are perfectly uncorrelated, then…•You get the same slope estimates regardless of the first-order regression model used.•That is, the effect on the response ascribed to a predictor doesn’t depend on the other predictors in the model.If predictors are perfectly uncorrelated, then…•The sum of squares SSR(X1) is the same as the sequential sum of squares SSR(X1|X2).•The sum of squares SSR(X2) is the same as the sequential sum of squares SSR(X2|X1).•That is, the marginal contribution of one predictor variable in reducing the error sum of squares doesn’t depend on the other predictors in the model.Same effects for “real data” with nearly uncorrelated predictors? BP Age Weight BSA Duration PulseAge 0.659Weight 0.950 0.407BSA 0.866 0.378 0.875Duration 0.293 0.344 0.201 0.131Pulse 0.721 0.619 0.659 0.465 0.402Stress 0.164 0.368 0.034 0.018 0.312 0.506Regress BP on StressThe regression equation isBP = 113 + 0.0240 StressPredictor Coef SE Coef T PConstant 112.720 2.193 51.39 0.000Stress 0.02399 0.03404 0.70 0.490S = 5.502 R-Sq = 2.7% R-Sq(adj) = 0.0%Analysis of VarianceSource DF SS MS F PRegression 1 15.04 15.04 0.50 0.490Error 18 544.96 30.28Total 19 560.00Regress BP on BSAThe regression equation isBP = 45.2 + 34.4 BSAPredictor Coef SE Coef T PConstant 45.183 9.392 4.81 0.000BSA 34.443 4.690 7.34 0.000S = 2.790 R-Sq = 75.0% R-Sq(adj) = 73.6%Analysis of VarianceSource DF SS MS F PRegression 1 419.86 419.86 53.93 0.000Error 18 140.14 7.79Total 19 560.00Regress BP on BSA and StressThe regression equation is BP = 44.2 + 34.3 BSA + 0.0217 StressPredictor Coef SE Coef T PConstant 44.245 9.261 4.78 0.000BSA 34.334 4.611 7.45 0.000Stress 0.02166 0.01697 1.28 0.219Analysis of VarianceSource DF SS MS F PRegression 2 432.12 216.06 28.72 0.000Error 17 127.88 7.52Total 19 560.00Source DF Seq SSBSA 1
View Full Document
Unlocking...