G89 2247 Lecture 4 Panel Designs Panel Path Analyses Examples Lord s Paradox Extensions G89 2247 Lecture 4 1 A Panel Design In many cases we have only two time points a baseline and a followup Between the two time points some treatment growth or other change is assumed to happen The two time points allow certain causal features of the relationships to be examined Temporal ordering of assumed cause to effect Establishing the contiguity of cause to effect Ruling out alternative causes such as selection effects G89 2247 Lecture 4 2 A Panel Analysis Path Regression methods are often proposed for panel analyses X1 Y1 b2 e2 b1 Y2 Y 2 b0 b1 Y 1 b2 X e 2 G89 2247 Lecture 4 3 Features of the Panel Path Analysis The effect of X b2 is a semi partial effect that takes into account the level Y1 Holding constant Y1 what is the effect of X b2 is sometimes called regressed change Restate Y2 b0 b1Y1 b2X e as Y2 b1Y1 b0 b2X e2 When b1 is close to 1 then simple change and regressed change analyses give similar answers G89 2247 Lecture 4 4 An example Suppose we want to show that the bar exam group has a different change in anxiety than the comparison group using time 1 and time 4 data POMS Anxiety Four weeks of Anxiety 2 5 2 1 5 1 0 5 0 WEEK1 WEEK2 WEEK3 WEEK4 First we regress Anxiety week4 on Group and Anxiety week1 G89 2247 Lecture 4 5 Example continued We proceed with a two step approach first with no adjustment for week1 baseline and then with an adjustment Sample 0 Examinees 1 Comparison Coefficients for Model Predicting Anxiety Week4 Model B S E Beta t 1 Constant 1 979 0 093 21 343 SAMPLE 0 489 0 132 0 307 3 715 2 Constant 1 027 0 121 8 516 SAMPLE 0 772 0 105 0 484 7 375 WEEK1 0 796 0 081 0 642 9 773 When week 1 is adjusted the group difference increases The initial tendency for the comparison group to be more anxious is statistically adjusted G89 2247 Lecture 4 6 In Path Terms r 28 X1 Y1 772 796 e2 Y2 G89 2247 Lecture 4 7 Example continued As Rogosa and others note the results vary if we adjust for a different baseline Coefficients for Week4 Anxiety as Outcome Model B S E Beta t 1 Constant 1 979 0 093 21 343 SAMPLE 0 489 0 132 0 307 3 715 2 Constant 0 584 0 098 5 946 SAMPLE 0 346 0 075 0 217 4 619 WEEK3 0 842 0 050 0 791 16 827 The one week lag produces an effect which is less than a half the effect of the three week lag G89 2247 Lecture 4 8 Interpreting Statistical Adjustment for Baseline The panel analysis allows X1 and Y1 to be correlated If X1 is a treatment this correlation may reflect some selection effect Including Y1 in the equation greatly strengthens the apparent causal claims regarding X1 Y2 compared to a simple crosssectional analysis The statistical adjustment for Y1 may not be enough to eliminate selection effects If Y1 is measured with error it will tend to underadjust If Y1 does not perfectly describe the selection bias the analysis also will underadjust E g Cohen s premature covariate G89 2247 Lecture 4 9 Regression Estimates If X and the Y measures are standardized then the estimate for b2 is 2 X 21 1 X b2 2 1 1 X If however Y1 is measured with error such that its reliability is R1 1 then b2 2 X 21 1 X RY 1 1 12X RY 1 G89 2247 Lecture 4 10 A numerical example of underadjustment X X Y1 Y2 Y1 Y2 1 0 6 1 0 5 0 7 1 Standardized coefficient for effect of X on Y b2 0 125 Assuming Measurement error for Y1 R 0 5 b2star 0 354 G89 2247 Lecture 4 11 Unreliability is not the only basis for underadjustment Consider Lord s Paradox Lord 1967 May Wt Weight change in two dorms Sept May Is there an effect of food service September Wt G89 2247 Lecture 4 12 The two perspectives The person who analyses raw change finds no difference On the average the two dorms have no weight gain The person who uses regressed change finds that a dorm effect Holding constant September weight persons from one dorm are likely to be heavier than persons in the other dorm G89 2247 Lecture 4 13 The Paradox Explained The regressed change analysis focuses on May Weight holding constant September weight Suppose we found that women were more likely to be in Dorm B and men in Dorm A When we compare a man and a woman who are the same weight in September we expect the man to gain weight and the woman to lose weight Even though it is reliable and valid September weight is not a perfect proxy for selection effects G89 2247 Lecture 4 14 Combining Raw Change and Regression Suppose we defined D Y2 Y1 to be raw change D will be negatively correlated with level of Y 1 Higher scores on Y1 are more likely to go down and lower scores are more likely to go up Consider adjusting for Y1 D a0 a1Y1 a2X e Y2 Y1 a0 a1Y1 a2X e This implies Y2 a0 1 a1 Y1 a2X e G89 2247 Lecture 4 15 Return to Example Let D41 difference between Anx 4 and Anx 1 Coefficients for Model Fitting Difference between T4 and T1 Model B S E Beta t 1 Constant 0 783 0 072 10 826 SAMPLE 0 845 0 103 0 581 8 235 2 Constant 1 027 0 121 8 516 SAMPLE 0 772 0 105 0 531 7 375 WEEK1 0 204 0 081 0 180 2 502 The group difference in this case is larger before adjustment but the unstandardized adjusted results are exactly what we had before G89 2247 Lecture 4 16 The Closer Time Point Now consider the anxiety difference between Times 4 and 3 Coefficients for Model Fitting Difference between T4 and T3 Model B S E Beta t 1 Constant 0 321 0 054 5 927 SAMPLE 0 320 0 077 0 339 4 151 2 Constant 0 584 0 098 5 946 SAMPLE 0 346 0 075 0 367 4 619 WEEK3 0 158 0 050 0 251 3 161 In this case the raw effect is diminished by adjustment Note that while the unstandardized coefficient does not change the standardized value does change G89 2247 Lecture 4 17 Extensions of Path Approach X1 Y1 X2 Y2 X3 Y3 G89 2247 Lecture 4 X4 Y4 18