Multiple Regression Models: Some Details & Surprises• Review of raw & standardized models• Differences between r, b & β• Bivariate & Multivariate patterns• Suppressor Variables• Colinearity•MR Surprises:– Multivariate power– Null Washout– Extreme colinearity• Missing Dataraw score regression y’ = b1x1+ b2x2 + b3x3+ aeach b• represents the unique and independent contribution of that predictor to the model• for a quantitative predictor tells the expected direction and amount of change in the criterion for a 1-unit change in that predictor, while holding the value of all the other predictors constant • for a binary predictor (with unit coding -- 0,1 or 1,2, etc.), tells direction and amount of group mean difference on the criterion variable, while holding the value of all the other predictors constant a• the expected value of the criterion if all predictors have a valueof 0 standard score regression Zy’= βZx1+ βZx2+ βZx3each β• for a quantitative predictor the expected Z-score change in the criterion for a 1-Z-unit change in that predictor, holding the values of all the other predictors constant • for a binary predictor, tells size/direction of group mean difference on criterion variable in Z-units, holding all other variable values constantAs for the standardized bivariate regression model there is no “a”or “constant” because the mean of Zy’ always = Zy = 0The most common reason to refer to standardized weights is when you (or the reader) is unfamiliar with the scale of the criterion. A second reason is to promote comparability of the relative contribution of the various predictors (but see the important caveat to this discussed below!!!).Different kinds of correlations & regression weightsr -- simple correlationtells the direction and strength of the linear relationship between two variables (r = β for bivariate models)b -- raw regression weight from a bivariate modeltells the expected change (direction and amount) in the criterion for a 1-unit increase in the predictorβ -- standardized regression weight from a bivariate modeltells the expected change (direction and amount) in the criterion in Z-score units for a 1-Z-score unit increase in that predictorbi-- raw regression weight from a multivariate model tells the expected change (direction and amount) in the criterionfor a 1-unit increase in that predictor, holding the value of all theother predictors constantβi-- standardized regression weight from a multivariate modeltells the expected change (direction and amount) in the criterion in Z-score units for a 1-Z-score unit change in that predictor, holding the value of all the other predictors constantWhat influences the size of bivariate r, b & β ?????r -- bivariate correlation range = -1.00 to +1.00 -- strength of linear relationship with the criterion-- sampling “problems” (e.g., range restriction)b -- raw-score regression weights range = -∞ to ∞-- strength of linear relationship with the criterion-- scale differences between & criterion-- sampling “problems” (e.g., range restriction)β -- standardized regression weights range = -1.00 to +1.00-- strength of linear relationship with the criterion-- sampling “problems” (e.g., range restriction)What influences the size of multivariate bi& βib (raw-score regression weights range = -∞ to ∞-- strength of linear relationship with the criterion-- collinearity with the other predictors-- scale differences between predictor and criterion-- sampling “problems” (e.g., range restriction)β -- standardized regression weights range = -1.00 to +1.00-- strength of relationship with the criterion-- collinearity with the other predictors-- sampling “problems” (e.g., range restriction)Difficulties of determining “more important contributors”-- b is not very helpful - scale differences produce b differences -- β works better, but influenced by sampling variability and measurement influences (range restriction)Only interpret “very large” β differences as evidence that one predictor is “more important” than anotheryx1x2x3Venn diagrams representing r, b and R2ry,x1ry,x2ry,x3yx1x2x3Remember that the b of each predictor represents the part of that predictor shared with the criterion that is not shared with any other predictor -- the unique contribution of that predictor to the modelbx1 & βx1bx2& βx2bx3& βx2yx1x2x3Remember R2is the total variance shared between the model (all of the predictors) and the criterion (not just the accumulation of the parts uniquely attributable to each predictor).R2= + + +Bivariate vs. Multivariate Analyses & InterpretationsWe usually perform both bivariate and multivariate analyses with the same set of predictors. Why?Because they address different questions• correlations ask whether variables each have a relationship with the criterion• bivariate regressions add information about the details ofthat relationship (how much change in Y for how muchchange in that X)• multivariate regressions tell whether variables have a unique contribution to a particular model (and if so, how much change in Y for how much change in that X after holding all the other Xs constant)So, it is important to understand the different outcomes possible when performing both bivariate and multivariate analyses with the same set of predictors.Simple correlation with the criterion- 0 +Multiple regression weight+ 0 -Non-contributing –probably because colinearity with one or more other predictorsNon-contributing –probably because colinearity with one or more other predictorsNon-contributing –probably because of weak relationship with the criterionBivariate relationship and multivariate contribution (to this model) have same sign“Suppressor variable”– no bivariate relationship but contributes (to this model) “Suppressor variable”– no bivariate relationship but contributes (to this model) “Suppressor variable” –bivariate relationship & multivariate contribution (to this model) have different signs“Suppressor variable” –bivariate relationship & multivariate contribution (to this model) have different signsThere are 5 patterns of bivariate/multivariate relationshipBivariate relationship and multivariate contribution (to this model) have same signHere’s a depiction of the two