Chapter 21 Chapter21 Presentation 1213 More About Tests Copyright 2009 Pearson Education Inc 1 Zero In on the Null To perform a hypothesis test the null hypothesis must be a statement about the value of a population parameter We then use this hypothesized value to help us compute the probability that the observed sample statistic or something even farther from the hypothesized value will occur Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 2 Alternative Hypothesis How do we choose the alternative hypothesis From what we see in the data From the research question Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 3 How to Think About P Values A P value is a conditional probability the probability of the sample results or results more unusual than that given that the null hypothesis is true Be careful to interpret the P value correctly Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 4 What to Do with a High P Value When we see a small P value we could continue to believe the null hypothesis and conclude that we just witnessed a rare event Instead we trust the data and use it as evidence to reject the null hypothesis However big P values just mean what we observed isn t surprising assuming the null hypothesis is true so we have no reason to reject the null hypothesis Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 5 Alpha Levels So far we have defined a small p value as one that is 0 05 or less and a big p value is one that is above 0 05 Is this always the cut off between small and big What should one consider before making the cut off between small and big something different Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 6 Alpha Levels cont Although a common cut off we don t have to define rare event as 0 05 probability or less we can pick any threshold we like If our P value falls below our threshold we ll reject the null hypothesis We call such results statistically significant The threshold is called an alpha level denoted by Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 7 Alpha Levels cont Common alpha levels are 0 10 0 05 and 0 01 You have the option almost the obligation to consider your alpha level carefully and choose an appropriate one for the situation The alpha level is also called the significance level When we reject the null hypothesis we say that the test is significant at that level Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 8 Alpha Levels cont The P value gives the reader far more information than just stating that you reject or fail to reject the null In fact by providing a P value to the reader you allow that person to make his or her own decisions about the test Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 9 Statistical vs Practical Significance What do we mean when we say that a test is statistically significant If the test is statistical significant does that imply the results have some practical importance Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 10 Connection Between Confidence Intervals and Hypothesis Tests Confidence intervals and hypothesis tests are built from the same calculations They have the same assumptions and conditions You can approximate a hypothesis test by examining a confidence interval Just ask whether the null hypothesized value is consistent with a confidence interval for the parameter at the corresponding confidence level Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 11 Connection Between Confidence Intervals and Hypothesis Tests cont Because confidence intervals are two sided they correspond to two sided tests In general a confidence interval with a confidence level of C corresponds to a two sided hypothesis test with an level of 100 C Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 12 Making Errors When we perform a hypothesis test we can make mistakes in two ways The null hypothesis is true but we mistakenly reject it Type I error The null hypothesis is false but we fail to reject it Type II error Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 13 Making Errors cont Which type of error is more serious is context dependent Here s an illustration of the four possible outcomes in a hypothesis test Type I Error Type II Error Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 14 Making Errors cont How often will a Type I error occur The probability of a Type I error is our level The researcher has complete control over the probability of a Type I error they pick the level Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 15 Making Errors cont When H0 is false and we fail to reject it we have made a Type II error We assign the letter to the probability of this mistake Is there just one value for like there is for Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 16 What Impacts the Size of 1 How false is H0 The difference between the hypothesized value of the parameter and the actual value of the parameter is called the Effect Size The larger the effect size the smaller will be Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 17 What Impacts the Size of cont 2 What is the Level The more we re willing to accept a Type I error the less likely we will be to make a Type II error So the larger the researcher is willing to set the value of the smaller will be Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 18 What Impacts the Size of cont 3 What is the Sample Size You are bound to make a better decision if you base it on more information So the larger the sample size n the smaller will be Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 19 How to Make Both and Smaller Once the null and alternative hypotheses are in place the researcher has no control over the Effect Size The researcher can easily make smaller but that automatically makes bigger So how can both and be made smaller Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 20 Power The power of a test is the probability that it correctly rejects a false null hypothesis The power of a test is 1 Chapter21 Presentation 1213 Copyright 2009 Pearson Education Inc 21 Power cont When we calculate power we imagine that the null hypothesis is false The numerical value of the power depends on how far the truth lies from the null hypothesis value The distance between the null