STA 291 Lecture 26 Two types of errors in testing hypothesis Connection between testing hypothesis and confidence intervals STA 291 Lecture 26 1 A P value that is smaller than 0 01 must also be smaller than 0 05 A P value that is smaller than 0 05 may or may not be smaller than 0 01 STA 291 Lecture 26 2 Read the wiki page on P value http en wikipedia org wiki P value STA 291 Lecture 26 3 Bonus due this week Get familiar with the final exam formula sheet 70 of Final exam will cover contents after 2 nd midterm STA 291 Lecture 26 4 In testing a hypothesis If our conclusion is to reject the null hypothesis then we either made a correct decision or we made a type I error STA 291 Lecture 26 5 If our conclusion is do not reject the null hypothesis then we either made a correct decision or made a type II error STA 291 Lecture 26 6 Type I and Type II Errors Type I Error The null hypothesis is rejected even though it is true Type II Error The null hypothesis is not rejected even though it is false Setting the alpha level significance level low protect us from type I Error the probability of making a type I error is less than alpha STA 291 Lecture 26 7 The chance of making a Type II error can be made small by increasing the sample size assume you use the correct testing procedure STA 291 Lecture 26 8 Decisions and Types of Errors in Tests of Hypotheses Terminology The alpha level significance level is a threshold number such that one rejects the null hypothesis if the p value falls below it The most common alpha levels are 0 05 and 0 01 The choice of the alpha level reflects how cautious the researcher wants to be when it comes to reject null hypothesis STA 291 Lecture 26 9 Type I and Type II Errors Decision True the null hypothesis False Reject Do not reject Type I error Correct Correct Type II error STA 291 Lecture 26 10 If after all the calculations and review the evidences you decide to reject the null hypothesis H0 you may be right or you may made a type I error If after all the calculations you decide not to reject the null hypothesis H0 you may be right or you may made a type II error STA 291 Lecture 26 11 When sample size s increases both error probabilities could be made to decrease Strategy keep type I error probability small by pick a small alpha Increase sample size to force the probability of making type II error Beta small or increase Power STA 291 Lecture 26 12 A connection between confidence intervals and testing hypothesis of two sided HA Testing H 0 3 HA 3 If the 95 confidence interval includes the value 3 3 is a possible value the p value of the test must be larger than 5 not reject also true STA 291 Lecture 26 13 If the 95 confidence interval do not includes the value 3 the p value of the test must be smaller than 5 reject also holds True for other parameters True for other confidence levels Only works for two sided HA hypothesis STA 291 Lecture 26 14 Confidence interval for a parameter consist of those values that are plausible not rejectable in a testing setting of two sided HA hypothesis STA 291 Lecture 26 15 So the confidence interval consists of those values of parameters that are compatible with the observed data data 95 confidence 5 error p value of 5 90 confidence 10 error p value of 10 etc STA 291 Lecture 26 16 suppose the 95 confidence interval computed from data for is 2 2 4 1 any test of H 0 3 HA 3 H 0 2 8 H A 2 8 H 0 4 HA 4 Would result a p value larger than 5 not reject i e any value inside 2 2 4 1 are plausible STA 291 Lecture 26 17 Now suppose 95 confidence interval computed from data for mu is 2 2 4 1 Any test of based on the same data H 0 1 9 H 0 4 8 H A 1 9 H A 4 8 Would result a p value smaller than 5 reject null using alpha 0 05 STA 291 Lecture 26 18 Why not always confidence interval In some cases confidence interval is hard to obtain Yet testing a specific hypothesis is easier Confidence interval amounts to obtain all values 0 that you cannot reject as H0 STA 291 Lecture 26 19 Pair or not pair If there is a possibility of pairing then pair usually is better Some clue that things are not paired there were different number of cases in two samples The two samples are obtained at different times with different experiments STA 291 Lecture 26 20 Paired Experiment focus on the differences One subject contribute two results we often can focus on the difference of the two from the same subject Sometimes not possible how long a mice live before cancer kill Same mice cannot be used twice Strength of the shipping packaging test of strength would destroy the package STA 291 Lecture 26 21 65 randomly chosen subjects are given two bottles of shampoos A and B After a week each subject state which one they prefer Subject 1 A B Subject 2 Subject 3 prefer prefer prefer STA 291 Lecture 26 22 125 randomly chosen subjects are given two bottles of pills A and B After a month on each pill report LDL cholesterol level Subject 1 Subject 2 Subject 3 A 167 155 233 B 188 159 214 STA 291 Lecture 26 23 After we take the difference for each subject the problem becomes a one sample problem If the preference has 50 50 chance If the difference has mean zero STA 291 Lecture 26 24 Focus on the difference Prefer A not prefer A prefer A For example 30 out of 65 prefer A In example two 21 4 19 For example X 11 4 STA 291 Lecture 26 s 7 8 25 For the first problem 0 4615 0 5 z 0 6202 0 5 1 0 5 65 P value is 2P Z 0 6202 2P Z 0 6202 0 535 STA 291 Lecture 26 26 If you use the computer to do the problem the p value will be slightly different Due to the fact that our calculation is only an approximation use CLT Computer is more accurate For sample size very large the difference goes away For example 300 subjects out of 650 prefer A STA 291 Lecture 26 27 For the cholesterol problem H 0 0 H1 0 11 4 0 z 16 34 7 8 125 P value 2P Z 16 34 0 00000000000 STA 291 Lecture 26 28 What would be a 95 confidence interval for the mu the population mean of the difference of cholesterol when using pill A B STA 291 Lecture 26 …