Introduction to Statistics in Psychology HYPOTHESIS TESTING Professor Greg Francis suppose I want to test whether a population mean is different from a value Lecture 31 H0 455 Power Ha 455 PSY 201 Will your research project work ERRORS State of nature Decision made H0 true H0 false Reject H0 Type I error Correct decision Do not reject H0 Correct decision Type II error We have mostly focused on avoiding Type I errors small now further suppose that the population mean really is different say it is 465 the probability of making a Type II error is probability of rejecting H0 when it is false correct decision is called the power of the test 1 2 3 DIRECTION LEVEL OF SIGNIFICANCE one tailed tests are more powerful Power 0 33 0 67 for 0 1 0 5319 Power 0 4681 FACTORS AFFECTING POWER 1 the directional nature of Ha onetailed versus two tailed test 2 The level of significance 3 The sample size n 4 The effect size ES power increases with 0 04 0 04 0 03 0 03 0 02 0 02 0 01 0 01 0 0 400 420 440 460 480 500 400 520 420 440 Power 0 2244 0 7756 480 500 520 for 0 01 0 8708 Power 0 1292 0 04 0 04 0 03 0 03 0 02 0 02 0 01 0 01 0 0 400 4 460 Sample Mean Sample Mean 420 440 460 480 500 520 400 420 440 460 Sample Mean Sample Mean 5 6 480 500 520 SAMPLE SIZE EFFECT SIZE GENERAL APPROACH larger samples result in smaller standard error which increases power for n 144 X 8 33 0 67 Power 0 33 power increases with effect size for Ha 465 ES 10 0 67 Power 0 33 the same basic approach works for other situations e g two sample case for the difference of means 0 04 0 04 0 03 0 03 0 02 0 02 H 0 1 2 0 0 01 0 01 0 400 0 400 420 440 460 480 500 420 440 460 480 500 520 Sample Mean 520 Sample Mean for n 576 X 4 17 0 2266 Power 0 7734 for Ha 475 ES 20 0 2236 Power 0 7764 0 03 0 06 find area under sampling distribution for Ha greater than tcv note need a good table of t scores 0 02 0 01 0 02 0 0 400 420 440 460 480 500 520 can calculate power for an ES of 6 Ha 1 2 6 0 04 0 08 0 04 Ha 1 2 0 find tcv 400 420 440 460 480 500 520 Sample Mean Sample Mean 7 8 9 EXAMPLE EXAMPLE HYPOTHESIS TEST Calculating power is useful because it lets you know whether your experiment is appropriate for what you want Your field of study expects you will use 0 05 you will want to run a test like this Suppose you are a graduate student investigating 5th graders with ADD You have a pretty good estimate of the variability of reading scores among ADD students s 30 Ha 80 0 05 n 7 your PhD thesis is based on the idea that a special remedial program will improve the reading scores of ADD students You have a pretty good estimate of the mean reading scores of ADD students without the remedial program X 80 you estimate that the remedial program might improve the scores by as much as 10 points You go to the local schools to get students to participate in your research project You get 7 students to volunteer to participate in your remedial program H0 80 What is the probability of being able to reject H0 if there really is an effect power this is in effect the probability that your research project works well enough for you to graduate Will you be able to graduate 10 11 12 COMPUTE THE POWER COMPUTE THE POWER COMPUTE THE POWER First find the critical value for the corresponding hypothesis test Second suppose the remedial test really did work and the mean reading scores of ADD students in the remedial program is actually 10 points higher Ha 90 Power is the probability that we ll reject the null hypothesis if the remedial program really does work as you expect it to we assume that s 30 is a pretty good estimate of the population standard deviation of reading scores for ADD students we find the critical value for the hypothesis test is zcv 1 645 which in terms of reading scores is X zcv X s 30 X 11 33 n 7 So X 80 1 645 11 33 98 63 then our sampling distribution will be centered on 90 What is the probability that we ll get X 98 63 and thus reject H0 Convert to a z score 98 63 90 z 0 761 11 33 That s the area under the curve of the sampling distribution a normal distribution to the right of z 0 761 We look that up in standard normal distribution table to find power 0 2236 which means if the remedial program actually works there s a 22 chance that you will be able to prove that it works So you will reject H0 if X 98 63 Note we use the same estimate of the standard error as we used before because we don t have anything better 13 14 15 BITS AND PIECES CONCLUSIONS NEXT TIME We treated the sampling distribution as a normal distribution factors that effect power designing efficient experiments why you should care standardized effect size but with a sample size of n 7 it would really be a t distribution in the sense that you reject the H0 How to spend your money wisely often this doesn t make a big difference because things are pretty similar for the normal distribution and the t distribution you could repeat the calculations and use the t distribution but not with the tables in the textbook tables like this exist on the internet and are easy to use 16 17 18