G89 2247 Lecture 2 Regression as paths and covariance structure Alternative saturated path models Using matrix notation to write linear models Multivariate Expectations Mediation G89 2247 Lect 2 1 Question Does exposure to childhood foster care X lead to adverse outcomes Y Example of purported causal model X Y B 1 Y B 0 B 1X e Regression approach e B0 and B1 can be estimated using OLS Estimates depend on sample standard deviations of Y and X sample means and covariance between Y and X B1 SXY S2X B0 MY B1MX Correlation rXY SXY SXSY can be used to estimate the variance of the residual e V e S2e S2Y 1 r2XY S2Y S2XY S2X G89 2247 Lect 2 2 A Covariance Structure Approach If we have data on Y and X we can compute a covariance matrix 2 SY S S XY SYX 2 SX This estimates the population covariance structure Y2 XY Y2 YX 2 2 X B 1 X 2 X Y can itself be expressed as B21 2X 2e Three statistics in the sample covariance matrix are available to estimate three population parameters G89 2247 Lect 2 3 Covariance Structure Approach Continued A structural model that has the same number of parameters as unique elements in the covariance matrix is saturated Saturated models always fit the sample covariance matrix 2 B12 X e2 2 B 1 X B1 2 X 2 X G89 2247 Lect 2 4 Another saturated model Two explanatory variables The first model is likely not to yield an unbiased estimate of foster care because of selection factors Isolation failure Suppose we have a measure of family disorganization Z that is known to have an independent effect on Y and also to be related to who is assigned to foster care X X XZ Z Y e G89 2247 Lect 2 5 Covariance Structure Expression The model Y b0 b1X b2Z e If we assume E X E Z E Y 0 and V X V Z V Y 1 then b0 0 and s are standardized The parameters can be expressed YZ YX XZ YX YZ XZ 1 2 2 2 1 XZ 1 XZ When sample correlations are substituted these expressions give the OLS estimates of the regression coefficients G89 2247 Lect 2 6 Covariance Structure 2 Explanatory Variables In the standardized case the covariance structure is 1 XY ZY XY 1 XZ ZY 1 XZ 1 XZ 2 1 2 XZ 1 1 XZ 1 Each correlation is accounted by two components one direct and one indirect There are three regression parameters and three covariances G89 2247 Lect 2 7 The more general covariance matrix for two IV multiple regression If we do not assume variances of unity the regression model implies 1 Y V X D 1 XZ 2 Z 2 XZ 1 Y2 X 1 XZ 2 Y Z 2 XZ 1 Y 1 XZ D 1 X2 Z XZ Y Z2 G89 2247 Lect 2 8 More Math Review for SEM Matrix notation is useful Y Y V X 0 Z 0 0 X 0 0 1 0 1 XZ 2 Z 2 XZ 1 Y2 X 1 XZ 2 Y Z 2 XZ 1 Y 1 XZ 2 2 XZ 1 Y XZ 1 1 XZ X2 Z XZ Y 0 0 0 X 0 2 Z G89 2247 Lect 2 9 0 0 Z A Matrix Derivation of OLS Regression OLS regression estimates make the sum of squared residuals as small as possible If Model is Y X B e Then we choose B so that e e is minimized The minimum will occur when the residual vector is orthogonal to the regression plane In that case X e 0 G89 2247 Lect 2 10 When will X e 0 When e is the residual from an OLS fit X e 0 X Y X B X Y X X B X X B X Y X X 1 1 X X B X X X Y 1 B X X X Y G89 2247 Lect 2 11 Multivariate Expectations There are simple multivariate generalizations of the expectation facts E X k E X k x k E k X k E X k x V X k V X x2 V k X k2 V X k2 x2 Let XT X1 X2 X3 X4 T and let k be scalar value E k X k E X k E X k 1 E X k 1 k 1 G89 2247 Lect 2 12 Multivariate Expectations In the multivariate case Var X is a matrix V X E X X T 12 13 14 2 21 2 23 24 2 31 32 3 34 2 41 42 43 4 2 1 G89 2247 Lect 2 13 Multivariate Expectations The multivariate generalizations of V X k V X x2 V k X k2 V X k2 x2 Are Var X k 1 Var k X k2 Let cT c1 c2 c3 c4 cT X is a linear combination of the X s Var cT X cT c This is a scalar value If this positive for all values of c then is positive definite G89 2247 Lect 2 14 Semi Partial Regression Adjustment The multiple regression coefficients are estimated taking all variables into account The model assumes that for fixed X Z has an effect of magnitude Z Sometimes people say controlling for X The model explicitly notes that Z has two kinds of association with Y A direct association through Z X fixed An indirect association through X magnitude X XZ G89 2247 Lect 2 15 Pondering Model 1 Simple Multiple Regression X Y XZ Z e The semi partial regression coefficients are often different from the bivariate correlations Adjustment effects Suppression effects Randomization makes XZ 0 in probability G89 2247 Lect 2 16 Mathematically Equivalent Saturated Models Two variations of the first model suggest that the correlation between X and Z can itself be represented structurally X Y eY eX Z eZ Z X eY Y G89 2247 Lect 2 17 Representation of Covariance Matrix Both models imply the same correlation structure 1 XY ZY XY 1 XZ ZY 1 XZ 1 3 2 1 1 2 3 1 3 1 The interpretation however is very different G89 2247 Lect 2 18 Model 2 X leads to Z and Y X Y eY Z eZ X is assumed to be causally prior to Z The association between X and Z is due to X effects Z partially mediates the overall effect of X on Y X has a direct effect 1 on Y X has an indirect effect on Y through Z Part of the bivariate association between Z and Y is spurious due to common cause X G89 2247 Lect 2 19 Model 3 Z leads to X and Y eX Z X Y eY Z is assumed to be causally prior to X The association between X and Z is due to Z effects X partially mediates the overall effect of Z on Y Z has a direct effect 2 on Y Z has an indirect effect on Y through X …