Power Analysis for Correlational Studies• Remember that both “power” and “stability” are important!• Useful types of power analyses– simple correlations– correlation differences between populations (groups, etc.)– differences between correlated correlations– multiple correlation models– differences between nested multiple correlation models– semi-partial and partial correlations– differences between non-nested multiple correlation models– differences between multiple regression models for different groups – Differences between multiple regression models for different criteriaWe know from even rudimentary treatments of statistical power analysis that there are four attributes of a statistical test that drive the issue of selecting the sample size needed a particular analysis…• acceptable Type I error rate (chance of a “false alarm”)• acceptable Type II error rate (chance of a “miss”)• size of the effect being tested for (.1=small, .3=med, .5=large)• sample size for that analysisWe also know that power is not the only basis for determining “N”The stability/variability of each r in the correlation matrix is related to NStd of r = 1 / √ (N-3), so …N=50 r +/- .146 N=100 r +/- .101 N=200 r +/- .07 N=300 r +/- .058 N=500 r +/- .045 N=1000 r +/- .031Power Analysis for Simple CorrelationOn the following page is a copy of the power analysis table from the first portion of the course. Some practice...Post hocI found r (22) = .30, p < .05, what’s the chance I made aType II error ??N = Power = Chance Type II errorA priori#1 I expect my correlation will be about .25, & want power = .90sample size should be = #2 Expect correlations of .30, .45, and .20 from my threepredictors & want power = .80sample size should be = 24.30 .70160191, based on lowest r = .20Power analysis for correlation differences between populations• the Bad News• this is a very weak test -- requires roughly 2x the N to test for a particular r-r value than to test for a comparable r-value• the Good News• the test is commonly used, well-understood and tables have been constructed for our enjoyment (from Cohen, 1988)Important! Decide if you are comparing r or |r| valuesr1 -r2.10 .20 .30 .40 .50 .60 .70 .80Power.25 333 86 40 24 16 12 10 8.50 771 195 88 51 34 24 19 15.70 1237 333 140 89 52 37 28 22.80 1573 395 177 101 66 47 35 28.90 2602 653 292 165 107 75 56 44all values for α = .05 Values are “S” which is total sample sizePower Analysis for Comparing “Correlated Correlations”It takes much more power to test the H0: about correlations differences than to test the H0: about each r = .00• Most discussions of power analysis don’t include this model• Some sources suggest using the tables designed for comparing correlations across populations (Fisher’s Z-test)• Other sources suggest using twice the sample size one would use if looking for r = the expected r-difference (works out to about the same thing as above suggestion)• Each of these depends upon having a good estimate of both correlations, so that the estimate of the correlation difference is reasonably accurate• It can be informative to consider the necessary sample sizes for differences in the estimates of each correlationHere’s an example …Suppose you want to compare the correlations of GREQ and GREA with graduate school performance.Based on a review of the literature, you expect that…• GREQ and grad performance will correlate about .4• GREA and grad performance will correlate about .6• so you would use the value of r-r = .20 …• and get the estimated necessary sample size of N = 395To consider how important are the estimates of r…• if the correlations were .35 and .65, then with r-r = .30, N= 177• if the correlations were .45 and .55, the with r-r=.10, N= 1573Power Analysis for comparing nested multiple regression models (R²Δ)…The good news is that this process is almost the same as was the power analysis for R². Now we need the power of …R²L-R²S/ kL-ksR²L-R²SN - kl-1 F = -------------------------- = --------------- * ------------1 - R²L/ N - kl- 1 1 - R²LkL-ksWhich, once again, corresponds to: significance test = effect size * sample sizethe notation we’ll use is … R2Y-A,B -R2Y-A -- testing the contribution of the “B” set of variablesUsing the power tables (post hoc) for R²Δ (comparing nested models) requires that we have four values:a = the p-value we want to use (usually .05)w = # predictors different between the two models)u = # predictors associated with the smaller model v = df associated with F-test error term (N - u - w - 1)f² = (effect size estimate) = (R²L-R²S) / (1 - R²L)λ = f² * ( u + v + 1) , wherePost Hoc E.g., N = 65, R²L(k=5) = .35, R²S (k=3) = .15a = .05 w = 2 u = 3 v = 65 - 2 - 3 - 1 = 59f² = .35 - .15 / 1 - .35 = .3077 λ = .3077 * (3 + 59 + 1) = 19.4Go to table -- a = .05 & u = 3 λ = 20power about .97 v = 60 .97 a priori power analyses for nested model comparisons are probably most easily done using the “what if “ approachI expect that my 4-predictor model will account for about 12% of the variance in the criterion and that including an additional 3 variables will increase the R² to about .18 -- what sample size should I use ???a = .05 w = 3 u = 4 f² = (RL²-RS²) / (1 - R²) = (.18 - .12) / (1 - .18) = .073“what if..” N = 28 N = 68 N = 128 N = 208 (∞)v = (N - u - w - 1) = 20 60 120 200 (∞)λ = f² * ( u + v + 1) = 1.83 4.75 9.13 15.0Using the table…power = < .15 about .37 about .64 about .89If we were looking for power of .80, we might then try N = 158so v = 150, λ = 11.3 power = about .77 (I’d go with N = 180 or so)Power Analysis for Semi-partial CorrelationsA semi-partial correlation can be obtained from the difference between two multiple regression models…rY(,A.B)= √ R²Y.AB-R²Y.Bor ……the correlation between Y & A, controlling A for B, is the square root of the unique contribution of A to the A-B modelSo,we could perform power analyses for semi-partial correlations using the same process we use for a nested model comparison. Now we need the power of