Chapter 9 Statistical Power 9 1 Type 2 Errors and Power Table 9 1 is a reproduction of Table 8 8 in Section 8 3 that presented the ideas of Type 1 and Type 2 errors In Chapter 8 we focused on the first column of this table the column which states that the null hypothesis is correct In particular by assuming that the null hypothesis is correct we are able to obtain the exact or an approximate sampling distribution for a test statistic This work led to the new notion of a critical region and its associated significance level for a test For most of the current chapter we will examine what happens when we assume that the alternative hypothesis is true There are two things to remember about assuming that the alternative hypothesis is true 1 As demonstrated in Chapter 5 whereas there is only one way for the null hypothesis to be true namely that the Skeptic is correct there are figuratively and sometimes literally an infinite number of ways for the alternative hypothesis to be true As a result we will be able to obtain probabilistic results only by assuming that the alternative is true in a particular way 2 The first step to focusing on the second column i e seeing what happens if one assumes the alternative hypothesis is correct is to obtain the critical region of a test In other words we must study what happens when the null hypothesis is correct before we can hope to study what happens when the alternative is correct Table 9 1 Types 1 and 2 errors in a test of hypotheses Action by researcher Fails to reject H0 Rejects H0 Truth Only Nature knows H0 is correct H1 is correct Correct action Type 2 Error Type 1 Error Correct action 193 As you might suspect the ideas behind this chapter are pretty complicated As a result in the next section I will present an extended complete analysis of a very simple and familiar study First however let me present a few basic ideas We are in a situation in which we have a sampling distribution for a test statistic exact or approximate and a target value for the significance level of the test We have found a critical region which has significance level which is close to the target You have had practice doing this in the Practice Problems and Homework of Chapter 8 The critical region serves the following purpose After the data are collected the observed value of the test statistic is found either to lie in the critical region or not to lie in the critical region in the former case the null hypothesis is rejected and in the latter case the researcher fails to reject the null hypothesis In other words by comparing the critical region to the data as summarized by the observed value of the test statistic we find out which row of Table 9 1 is occurring When we focused on the first column of Table 9 1 the possible actions were fail to reject the null hypothesis the correct action for column 1 and reject the null hypothesis a Type 1 error Each of these actions has a probability of occurring and because there are only two possible actions the two probabilities must sum to one I mentioned that we denote the probability of a Type 1 error by the symbol Obviously the probability of correctly failing to reject a true null hypothesis is equal to 1 I have never witnessed anybody referring to this latter probability let alone giving it a name either verbally or in writing So why do I mention it You will see very soon As mentioned above when we attempt to calculate probabilities on the assumption that the alternative hypothesis is correct we must specify exactly how it is correct Once that specification is made it is sensible to seek the probability of a Type 2 error The probability of a Type 2 error typically is denoted by get it Type 1 Type 2 the first two letters of the Greek alphabet In column 2 however unlike in column 1 we give a name to 1 it is called the power of the test for the particular alternative being considered The idea is that we would like to have a test that has a low probability of making a Type 2 error in other words we would like to have a test that has a large power 9 2 An Extended Example Cathy s Study of Running Cathy s study of her running was introduced near the end of Chapter 2 I am guessing that you remember the general motivation of the study but have not memorized Cathy s data If I am wrong about the former please read about it again Section 2 4 on page 40 if I am correct about the latter you will appreciate my reproduction of Cathy s data in Table 9 2 For the purpose of this section we will suppose that Cathy chose the alternative i e that her times on treatment 1 the high school would be larger i e she would run slower than her times on treatment 2 the park This is not a ridiculous choice Cathy might have believed that the natural beauty of the park energized her resulting in faster times In any event the sampling distribution of U for Cathy s study is given in Table 9 3 it is easy to see that the critical region U 9 67 gives 0 05 exactly 194 Table 9 2 Cathy s times in seconds to run one mile HS means she ran at the high school and P means she ran through the park Trial 1 2 3 Location HS HS P Time 530 521 528 4 5 6 P HS P 520 539 527 Table 9 3 The sampling distribution of U for Cathy s CRD u P U u u P U u 9 67 0 05 3 00 0 10 9 00 0 05 2 33 0 05 7 67 0 05 1 67 0 05 5 00 0 05 1 67 0 05 4 33 0 05 2 33 0 05 3 67 0 05 3 00 0 10 u P U u 3 67 0 05 4 33 0 05 5 00 0 05 7 67 0 05 9 00 0 05 9 67 0 05 Cathy s actual value of u is 5 00 This value is not in the critical region thus the action is that Cathy fails to reject the null hypothesis Earlier when we were focusing on P values we found that the P value for Cathy s data and the alternative is equal to 0 20 Then as now the conclusion action was to fail to reject At this point a student might be tempted to ask Tell me Bob did Cathy take the correct action My answer of course is Who do you think I am Nature We are now approaching the tricky part of this story I am now going to focus on the idea that the alternative hypothesis …
View Full Document
Unlocking...