POWER SAMPLE SIZE PSY 201 power 1 Professor Greg Francis the probability of rejecting H0 when it really is false we know that having a larger sample size gives us more power and makes it easier to reject H0 Lecture 32 we want tests with lots of power Power power depends on Introduction to Statistics in Psychology How to spend your money wisely 1 the directional nature of Ha onetailed versus two tailed test 2 The level of significance 3 The sample size n suggests that we should use really big sample sizes all the time but sometimes data is difficult to come by patients we do not want to waste resources how do we select a sample size that will give us the proper power 4 The effect size ES like a magnifying glass more powerful tests resolve smaller ES s 2 3 SAMPLE SIZE FORMULA HUH choosing a sample size to match a specified power requires knowledge of for a one tailed test 1 The level of significance 2 The power of the test 1 3 The population error variance 2 4 The effect size ES need to know values before starting experiment estimate 2 from other sources there is a formula used to calculate the appropriate sample size n n 2 z z ES 2 where 2 2 population error variance z critical value of the test statistic in the sampling distribution association with H0 for a given n 2 z z ES 2 z is the z score for which area is to the left in the Ha sampling distribution for a right hand tail test z is the z score for which area is to the right in the H0 sampling distribution for a right hand tail test if converted to raw scores these should equal each other z standard score in the sampling distribution associated with Ha corresponding to z for a given power ES effect size 4 5 2 6 EXAMPLE EXAMPLE TWO TAILED TEST SAT data from the standard normal distribution table we find that z 0 842 z 1 645 so 2 2 z z n 2 ES split across two tails H0 455 Ha 465 suppose I set 0 05 one tailed test 0 20 4 1 ratio Power 0 80 100 known from other sources ES 10 what is considered important 100 2 0 842 1 645 2 618 52 n 10 2 n 619 0 1 0 08 0 06 still pick a particular Ha e g H0 455 Ha 465 ES 10 power in left tail of Ha sampling distribution is negligible so consider only right side means the situation is just like for a one tailed test but we use a smaller value 2 2 z z 2 n ES 2 0 04 0 02 0 430 440 450 460 470 480 490 500 Sample Mean 7 8 9 EXAMPLE EXAMPLE EXAMPLE 2 0 20 4 1 ratio Power 0 80 100 n ES 10 from the standard normal distribution table we find that z 2 1 96 thus for a one tailed test we need a sample size of 619 to detect a 10 point effect size 2 z z 2 n ES 2 0 05 two tailed test 100 2 0 842 1 96 2 785 12 10 2 n 786 note we could have set ES 10 and would have looked at the left tail to determine power would get the same sample size due to symmetry 0 1 z 0 842 0 08 0 06 0 1 0 04 0 08 0 02 0 06 when confused draw a picture 0 430 0 04 for a two tailed test we need a sample size of 786 to detect a 10 point effect size 440 450 460 470 480 490 500 Sample Mean 0 02 0 430 440 450 460 470 480 490 500 Sample Mean 10 11 12 STANDARD DEVIATION a problem with this approach to identifying sample size is that we rarely know the population standard deviation two approaches STANDARDIZED EFFECT SIZE STANDARDIZED EFFECT SIZE define ES d as the standardized effect size for example I want to know how large my sample size should be with a standardized effect size d 0 20 0 05 and power 0 80 0 20 1 estimate the standard deviation from other research if is estimated by s we would use ES d s 2 express the effect size in terms of standard deviation units z scores from the standard normal distribution table I find that one tailed test z 0 842 z z d2 n 2 two tailed test n 2 z z 2 d2 z 1 645 so with a one tailed test I need 2 z z n 2 d 0 842 1 645 2 n 154 63 155 0 04 13 14 15 STANDARDIZED EFFECT SIZE TWO SAMPLE CASE EXAMPLE same approach suppose I want to know the sample sizes needed to detect a standardized effect size of d 0 5 with 0 01 and 0 04 power 0 96 with a two tailed test for the two sample case from the standard normal distribution table I also find that z 2 1 96 so with a two tailed test I need 2 z z 2 n d2 0 842 1 96 2 n 196 28 197 0 04 but n applies to each group formulas are just double for the one sample case for one tailed test 2 2 z z n d2 for two tailed test n 2 z z 2 d2 draw a graph to show direction of differences 0 4 0 3 2 0 2 0 1 0 6 4 2 0 Sample Mean 16 17 18 2 4 6 EXAMPLE EXAMPLE EFFECT SIZE I find from the standard normal distribution table that when we take the sample size into account we can calculate the standard error for the standardized scores 1 1 sX 1 X 2 s2 n1 n2 and for our standardized scores we have s2 1 0 we get 1 1 0 1154 sX 1 X 2 1 0 150 150 all of this presupposes that we have a desired effect size z 2 2 5758 z 1 7507 2 2 z z 2 d2 2 1 7507 2 5758 2 n 149 7 150 0 25 in each group n a difficult issue to address depends upon use of the data requires experience with the situation standardized units do not really help because importance varies across disciplines 3 2 1 0 6 4 2 0 2 4 6 Sample Mean 19 20 21 BENEFITS CONCLUSIONS NEXT TIME when you know how big of an effect is an important one power and sample size Review for Exam III effect size then standardized effect size After Exam III significance versus importance Running multiple hypothesis tests you can specify the properties of the test and power 1 to check for that effect notice if the effect does not exist the test will not make it look as if it is there except with probability Some problems Error is sneaky statistics is not simply a numbers …