Handout Linear Regression Consuelo Arbona Ph.D.EPSY 8334 University of Houston Definition: Multiple linear regressions allow one to examine the individual and collective contribution of more than one independent variable (or predictor variable) on a dependent variable (also called criterion variable). For example, regression could be used to examine the contribution of gender, ethnic identity and self esteem to depression scores. Typically, one would say depression scores are regressed onto gender, ethnicity, self esteem and ethnic identity scores. Regression assumes that the relations of the independent and dependent variables are linear (positive or negative).Regression line – best line that describes the relation of one or more dependent variables ( X1 X2 X3) to an independent variable Y: Y = b1 X1 + b2 X2 + b3 X3 + b4 X4 + a (constant) (bs are the beta coefficients, or weight of each variable – the larger the b the stronger the unique association of the predictor variable to the criterion. Characteristics: Both criterion and predictor variables in regression analyses should be continuous. Categorical variables can be included as predictors only if they have two levels (e.g. gender: 1 = F; 2 =M). Categorical variables with more than two levels (e.g. ethnicity: White, Black and Hispanic) need to be dummy-coded. One way to do it is to create two dummy variables that use one group- lets say Whites - as the reference group: Var 1 Hisp where the coding goes Hisp = 1 and W =2 and B = 2 ; Var 2 Black where Black =1 and W=2 and H = 2--- so we code three ethnic groups in terms of two variables Hisp with two values 1= Hisp and 2 for every one else and Black where 1 stands for Blacks and 2 stands for every one else. Both Hisp (coded 1,2)and Black (coded 1,2) are entered as variables to account for three ethnic groups.Predictor variables should be highly related to the criterion variable and the correlation of the predictor variables among themselves should be low (to aovid multicollinearity problems). Three types of linear regression analyses: Simultaneous, Stepwise, and HierarchicalSimultaneous – all predictors are entered at once in the equationStepwise – Computer use an algorithm to decide which predictors variables to enter. For example inforward stepwise regression, the variable with the highest correlation with the dependent variable is entered first, followed by the variable that has the highest correlation with the dependent variable once controlling for the fist variable entered and so on. Stepwise methods (there is also backward stepwise) should used only for prediction and with large sample sizes (about 40 cases per predictor);it is problematic because results tend to be very sample specific. Hierarchical – researcher chooses order that variables are entered in the equation, decided according to (1) causal priority – variables presumed to cause other predictor variables are entered first (e.g. parental SES and offspring’s academic attainment), (2) research relevance/theory- those variables that have been studied before are entered first; variables that theory predict should antecede other variables are entered first (3) main effect variables are always entered before interaction effect variables. EPSY 8334 - Linear Regression Page 1/4Sample size – Depends on the power one wants, the expected value of R2 and the number of predictors. Some authors give the following formula to calculate minimum sample size for linear regression (not considering interaction effects): SS = 50 + 8k (k = # of predictors)Interpretation of Regression Table:R2 Equals the proportion of variance in the criterion variable accounted for or “explained” by the linear combination of the predictor variables. (Comparing values R2 from different studies is tricky because determining the relative magnitude of the R2 requires taking into consideration range of scores in independent variables, number of independent variables in the analyses and sample size for each study.) Δ R2In hierarchical regression, ΔR2 equals the proportion of variance accounted for by an predictor variable (or a collection of predictor variables) entered in one step over and above the proportion of variance accounted by all the predictor variables entered in the previous steps in the equation (in other words, the incremental value of R2 ). (B) Non-standardized Beta Partial regression coefficients (Column labeled B in the RegressionTables refers to the bs in the equation: Y = b1 X1 + b2 X2 + b3 X3 + b4 X4 + a). The Betasindicate how much Y (the value of the criterion variable) will change for a unit of change in the predictor variable when all the other variables are controlled for. Each B is expressed in the metric of each variable; therefore they cannot be compared with each other β Standardized Beta (or Beta coefficients) are the Beta values standardized (expressed in terms of standard deviation units), so within one regression analyses they can be compared to each other (However, without a test of significance one cannot determine if the differences observed in Beta coefficients in a regression output are statistically significant or not)Example It is known that depression tends to be higher among women and those with low self esteem than among men and those with high self-esteem. It is also known that among Hispanics ethnic identity achievement is positively related to self esteem. It is hypothesized that among Hispanics ethnic identity is negatively related to depression. In a study authors examined with Hispanic participants:1. to what extent ethnic identity achievement adds to gender and self esteem in predicting depression 2. to what extent the relation of ethnic identity to depression is moderated by self esteem .Using scores on the Beck Depression Inventory, Phinney’s. Ethnic Identity measure, and the Rosenberg self-esteem inventory, they conducted a hierarchical regression analyses. In the first step the two known variables (from previous research were entered). In step 2 the variable that had not been examined in relation to depression - ethnic identity- was entered.EPSY 8334 - Linear Regression Page 2/4Moderation analyses is used to examine to what extent the strength and direction of the relation of two predictor variables (ethnic identity and depression) is different for people who vary in terms of a third variable (self-esteem). This