VariableCharacteristics of the Individual Regression ParametersMultiple CorrelationSemi-Partial CorrelationPartial CorrelationSignificance Tests of sr and prAnalytic Strategies for MRCSimultaneous1CORRELATION AND REGRESSIONWITH MULTIPLE VARIABLES2Bivariate Associations-Bivariate correlation and regression examine the association between two variables (X & Y)-Bivariate associations may provide a deceptive view of the actual association between X & Y-X & Y’s associations with other variables are not removed from the bivariate association between X and Y3The Deceptive Lens of Bivariate Assc.Bivariate Associations b/w Salary, PhD, Publications, Sex andCitationsVariable Salary PhD Pubs Sex CitationsSalary - - - - -PhD .62 - - - -Pubs .46 .68 - - -Sex -.26 -.15 .05 - -Citations .51 .46 .30 -.01 -Note. Sex is coded such that 0= male and 1 = female.-Bivar r suggest that salary -increases with PhD, Pubs, Citations-is less for females than males-However, PhD, Pubs, and Citations are correlated-Some of the assc. Between salary and Pubs may really be due to Citationsand PhD (and vice versa).4Goal of MRC…Focusing the Lens-To accurately assess the X and Y assc.we need to remove theinfluence of ALL variables associated with X and Y-When X, Y, and Z are analyzed in a MRC the effect of Z is removed from the association of X & Y-MRC can might also be deceptive if there are other variablesassociated with X and Y (in addition to Z) that are not included in the analysis.-Note that the problem of shared association with unmeasured variables is a problem of the research method.5IVs from a pure experiment are uncorrelated with other factors.Today’s Lecture-Multiple regression-characteristics of the overall model-characteristics of the regression parameters-Multiple correlation -semi-partial correlations-partial correlations-Analytic strategies of MRC-simultaneous-hierarchical-step-wise6Multiple Regression-Provides a linear model linking Y to multiple predictors -unconfounds the effect of each predictor on Y from the effect of the other predictors21212112ˆXBXBAYYYY -Betas are “partial betas”21YBis the effect of X1 on Y controlling for X212YBis the effect of X2 on Y controlling for X1-Y-intercept (12YA) is the predicted value of Y when X1 and X2 are zero (point where regression line crosses the Y-axis).-Simplified notation of multiple regression equation:72211ˆXBXBAY Academic Salary Example-Bivariate regressions: Salary = 21106 + 566PubsSalary =22976 + 1918Citations-Pubs & citations r=.30-Multiple regressionSalary = 20285 + 418Pubs +1536Citations (note, partial B does not have to be smaller than bivariate B…e.g. rCP= -, rYC= + & rYP= +)8Least Square Formulas12121221211XYYYYrrrrB22121212121XYYYYrrrrB21212112XBXBYAYYY -Formulas are for a 2-predictor model-Become more complicated as more predictors are addedMultiple Regression in SAS9proc reg;model salary = pubs citations;run;-See output for the partialled BsSalary = 20285 + 418Pubs +1536CitationsCharacteristics of the Overall Model10-Sum of squares-Coefficient of multiple determination (R2)-F-test of the overall model-Adjusted R2-Standard error of the estimate (SEestimate)Sum of Squares-Top of output has ANOVA table for the ModelSum of MeanSource DF Squares Square F pModel 2 202061502 101030751 3.41 0.0672Error 12 355458823 29621569Total 14 55752032511SStotal = SSmodel + SSerror-SStotal = total variability in Salary-SSmodel = amount of variability in Salary related to Pubs & Citations-SSerror = amount of variability in Salary NOT related to Pubs & Citation(2)ˆ( YYerror)Coefficient Of Multiple Determination (R2)-Proportion of variability in Y explained by the modeltotalmodel2SSSSR12avc-Salary example (salary = pubs citations):3602.557520325202061502SSSStotalmodel2R (R-square on SAS output)-Pubs and citations together account for 36% of variability in Salary-Multiple R2 is larger than either bivariate r2 21.2SPr 26.2SCr (unless 1 predictor is unrelated to DV)-Bivariate r2 do not sum to R2 (.21 + .26 - .36)-partialled from each r2 shared variability among the predictorsF-Test Of The Model-F(2, 12) = 3.41, p = .0672 in ANOVA is a test of the model-Comparison of 2 models:13Salary = intercept Pubs Citations (full model)Salary = intercept (restricted model)-does model with pubs and citations fit better (i.e. less error) than a model without pubs and citationsFFFRFRdfEdfdfEEF)()(ER and EF = the SSerror of the full and restricted models dfR and dfF represent the degrees of freedom (error) of the full and restricted models.Run Both Models in SASproc reg;model salary = pubs citations ;model salary = ;run;1441.312355458823)1214()355458823557520325()()(FFFRFRdfEdfdfEEF-F(2, 12) = 3.41, p = .0672 indicates that the full model does not fit better than the restricted model (but n = 15)F-test (model comparison) in Terms of R241.312)3624.1(1214)03624(.)1()(22Re2FFullFRstrictedFulldfRdfdfRRF-By changing the restricted relative to full model we can test other hypotheses.15Does a model with publications and citations fit better than a model with only publications?Adjusted R2Shrunken R2 or 2ˆ(omega hat squared) or 2~Radjusts for the positive bias of R2.-The sample R2 is an estimate of the degree of shared variability in the population (-2). But it tends to overestimate due to sampling error. When there are no associations in the16population, sampling error may produce associations in the sample.-Salary example (2~Ris Adj r-sqr in output)R2 = .36 and 2~R = .26Standard Error Of The Estimate (SEestimate)-average amount of error when predicting Y from the multiple predictor variables.1)ˆ( SE2estimateknYY, where k = number of predictor variables -Listed as Root MSE in SAS (i.e., MSE)17-Salary exampleSEestimate - 5443When predicting Salary from publications and citations we’ll be in error, on average, by $5443. Characteristics of the IndividualRegression ParametersSalary = 20285 + 418Pubs +1536CitationsKeep in mind that the B’s are sample estimates-Standard Error of the Partial Betas (SEB)-Significance test of the Partial Betas18-Standardized
View Full Document