MASON PSYC 612 - Lecture 4: Multiple Regression and Correlation

Unformatted text preview:

PSCY 612, SPRING 2008Lecture 4: Multiple Regression and CorrelationLecture Date: 2/13/2008Contents1 Preliminary iClicker Questions 22 Part I: Multiple Regression (70 minutes; 5 minute break) 22.1 Purpose: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 Objectives: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.3 The purpose of multiple regression . . . . . . . . . . . . . . . 32.4 Parameter estimation in multiple regression . . . . . . . . . . 62.5 Hypothesis testing in multiple regression . . . . . . . . . . . . 82.6 Complexities of MRC . . . . . . . . . . . . . . . . . . . . . . . 92.6.1 Standardized (β) versus unstandardized parameters (b) 92.6.2 Multicollinearity . . . . . . . . . . . . . . . . . . . . . 102.6.3 Partitioning variance . . . . . . . . . . . . . . . . . . . 102.6.4 Mixed variables . . . . . . . . . . . . . . . . . . . . . . 113 Part II: Multiple Regression Details (50 minutes; 5 minutebreak) 113.1 Shared variance . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2 Sums of Squares (SS) Types . . . . . . . . . . . . . . . . . . . 123.2.1 Type I: Hierarchical partitioning - o rdered or sequential 143.2.2 Type II: Partial hierarchical - non-ordered but hierar-chical . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2.3 Type III: Simultaneous - non-ordered . . . . . . . . . . 183.2.4 Others . . . . . . . . . . . . . . . . . . . . . . . . . . . 1914 Part III: Advanced Material (30 minutes) 204.1 Appendix A: Introduction to Linear Algebra . . . . . . . . . . 204.1.1 Matrix Terminology . . . . . . . . . . . . . . . . . . . 224.1.2 Matrix Manipulation . . . . . . . . . . . . . . . . . . . 234.1.3 Back to our example . . . . . . . . . . . . . . . . . . . 234.1.4 Why understanding linear algebra is helpful . . . . . . 251 Preliminary iClicker Questions•Have you read all the assigned reading for today?•Are my notes helpful?•What would improve my notes?•What is the correlation between the two va riables on the board?• What is the slope and intercept for those two va riables?•Is the relationship significant (t(3)95%:crit= 3.18)?2 Part I: Multiple Regression (70 minutes; 5minute break)2.1 Purpose:To introduce yo u to multiple regression through numerical and graphicalexamples2.2 Objectives:1. Explain the purpose and advantage of multiple regression over bivariateregression2. Discuss the details of multiple regression - parameter estimation3. Discuss the details of multiple regression - hypothesis testing4. Expand discussion of main effects to include interactions5. Introduce complexities of multiple regression22.3 The purpose of multiple regressionBiva riate regression serves a purpose in social science. Unfortunately, realityis more complicated than simple bivariate relationships. Multiple regressionhelps us model the complexity by adding more predictors. Recall that theassumptions of r egression specifically addressed errors of omission and com-mission; the former being more deadly than the lat t er. The f ollowing pointshighlight the purpose of multiple regression and, hopefully, help you easilydifferentiate bivariate regression f r om multiple regression.• more predictor sp ecificationY = X1b1+ X2b2+ · · · + Xnbn+ y0• main effects and interactionsY = X1b1+ X2b2+ X1∗ X2b12 + y0• rectilinear and curvilinear relationshipsY = X1b1+ X21b2+ y03−2 0 1 2−2 0 2 4 6 8Rectilinearxy−2 0 1 2−2 0 2 4 6 8CurvilinearxyThe rectilinear relationship results in the following regression equation:ˆY = 0.82 ∗ x + 0.93whereas the curvilinear relationship results in this regression equation:ˆY = 0.82 ∗ x + 1.08 ∗ x2− 0.01Note how the regression parameters change just by specifying the expo-nential?• graphical presentation gets more difficultThe previous graphs depicting bivariate relationships were easy to pro-duce. When we introduce more predictors, we must increase the numberof dimensions. I am only capable o f 3 dimensions; others may be ableto envision hyper-planes and other complexities but I cannot. Thus, ourgraphical presentations of multiple regression r esults shift from observed4predictors and a dependent variable to the predicted and observed resultsfor the dependent variable. We will also use different plots that help usdiagnose potential problems with these plots.−30 −20 −10 0 10 20 30−30 −20 −10 0 10 20 30Y−observedY−predictedThe plot above represents the following results:Estimate Std. Error t value Pr(>|t|)(Intercept) −1.1131 0.6914 −1.61 0.1109x1 0.6424 0.6821 0.94 0.3487x2 0.5105 0.4410 1.16 0.2500x3 1.1209 0.2879 3.89 0.0002x1:x2 0.2927 0.4138 0.71 0.4811x1:x3 0.1187 0.2327 0.51 0.6113x2:x3 0.0743 0.1043 0.71 0.4779x1:x2:x3 −0.0386 0.0788 −0.49 0.6254Notice how these results are far more complicated than could be repre-5sented by a plot with just two axes.2.4 Parameter estimation in multiple regressionWhen we compute regression para meters with multiple predictors, the stepsget a bit more complicated than the simple bivariate case. The complica-tions arise from what is formally known as multicollinearity - predictors arecorrelated. We shall return to this complication shortly. For now, we justneed to acknowledge the fact that our predictors in MRC may be correlated.Factors or “independent variables” in ANOVA ar e unrelated by definition.The same cannot be said in MRC. We account for the inter-correlations be-tween our predictors in parameter estimation by computing parameters thatcapture only the unique variance shared between our dependent variable andour predictor. Given a simple two-predictor MRC model expressed as:Y = X1b1+ X2b2+ Y0The parameter estimation procedure follows a general approach for twopredictors:1. estimate unique variance among the predictors (u)(a) run regression between predictorsThe following equations make this rather clear but I intend to gothrough each piece during lecture.X1= bX2+ a + u1andX2= bX1+ a + u2The terms u1and u2are the residuals from the two equationsabove and represent the typical residuals we discussed previouslyand expressed as:u = X −ˆXexcept in this case, the residuals are between predictors.6(b) estimate unique variance of the dependent variable for each pre-dictor that is not accounted for by the other predictors (v)Y = bX2+ a + v1andY = bX1+ a + v2The terms v1and v2are the r esiduals from the two equations aboveand, again, t hey represent the typical residuals we discussed withthe bivariate residuals.v = Y


View Full Document

MASON PSYC 612 - Lecture 4: Multiple Regression and Correlation

Download Lecture 4: Multiple Regression and Correlation
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Lecture 4: Multiple Regression and Correlation and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Lecture 4: Multiple Regression and Correlation 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?