STA 291 - Lecture 24 1STA 291Lecture 24• Two kinds of Error in Testing hypothesis• Examples.About bonus projectDue in Lab April 20- 22• Some survey show a majority people believe in “hot hand”. A follow-up question then is: if there were “hot hand”, how much better/worse a shooter can become by the previous shoots? (i.e. what is a reasonable difference to expect)• i.e. whether he made or missed the two previous shots, how much difference do you think this has the effect on the present shoot? (in terms of hitting percentages) 5%? 10%? or even 20%?STA 291 - Lecture 242• Since a small difference will need more data to detect. • A larger difference can be discovered with less data.• Some clue: margin of error calculations.STA 291 - Lecture 24 3• Bonus is worth equivalent to one LAB• Lab will start to give “practical quiz”STA 291 - Lecture 24 4STA 291 - Lecture 24 5Decisions and Types of Errors in Tests of Hypotheses• Terminology:– The alpha-level (significance level) is a threshold number such that one rejects the null hypothesis if the p-value is less than or equal to it. The most common alpha-levels are 0.05 and 0.01– The choice of the alpha-level reflects how cautious the researcher wants to be (when it come to reject null hypothesis)STA 291 - Lecture 24 6Type I and Type II Errors• Type I Error: The null hypothesis is rejected, even though it is true.• Type II Error: The null hypothesis is not rejected, even though it is false.• Setting the alpha-level low protect us from type I Error. (the probability of making a type I error is less than alpha)STA 291 - Lecture 24 7Type I and Type II ErrorsDecisionReject nullDo not reject nullthe null hypothesisTrueType I errorCorrectFalseCorrectType II errorSTA 291 - Lecture 24 8Type I and Type II Errors• Terminology:– Alpha = Probability of make a Type I error– Beta = Probability of make a Type II error– Power = 1 – Probability of a Type II error = 1 - Beta• For a given data, the smaller the probability of Type I error, the larger the probability of Type II error and the smaller the power• i.e. If you set alpha very small, it is more likely that you fail to detect a real difference (larger Beta).STA 291 - Lecture 24 9• When sample size(s) increases, both error probabilities could be made to decrease.• Our Strategy: -- keep type I error probability small by pick a small alpha.-- Increase sample size to make Beta small.• Depend on how expensive to obtain data, a Beta = 0.15 is not uncommon.STA 291 - Lecture 24 10Type I and Type II Errors• In practice, alpha is specified, and the probability of Type II error could be calculated, but the calculations are usually difficult ( sample size calculation )• How to choose alpha?• If the consequences of a Type I error are very serious, then chose a smaller alpha, like 0.01.• For example, you want to find evidence that someone is guilty of a crime.• In exploratory research, often a larger probability of Type I error is acceptable (like 0.05 or even 0.1)Example: New drug development• The null hypothesis usually state that the new drug is “no difference” to the placebo.• A type I error in this context is: falsely conclude a drug is useful when it is actually “NO effect”• A type II error in this context is: falsely dismiss a useful drug. STA 291 - Lecture 24 11STA 291 - Lecture 24 12Alternative and p-value computationOne-Sided TestsTwo-Sided TestalternativeHypothesisp-value:AHpp0<0:0=Hpp:AHpp0>:AHpp0≠µ00(1)/obsppzppn0−=−()obsPZz<()obsPZz>2(||)⋅>obsPZzExample• Two consumer products (shampoo, laundry detergent etc) comparison. Call them A vs. B• n consumers are given both products in the identical packaging. After one week of use of both products, state a preference.• If there were no difference, then we should see 50%-50%STA 291 - Lecture 24 13• Suppose in n=236 consumers, 110 prefer product A. Let p = popu. proportion prefer A. Use alpha = 0.05• Null: • Alternative: • Compute STA 291 - Lecture 24 140:0.5Hp=:0.5AHp≠obsz• Sample proportion, = 110/236 =0.4661• Finally look the Z table for P-value:• P-value=2P(Z>1.04)=2(1- 0.8508) =0.2984STA 291 - Lecture 24 15µp0.46610.51.041560.5(10.5)/236obsz−==−−• Conclusion, we do not reject null hypothesis since P-value is not less than alpha.• Since 0.2984 is not less than 0.05STA 291 - Lecture 24 16STA 291 - Lecture 24 17Two sample cases are similar, with two differences:• Hypothesis involve 2 parameters from 2 populations• Test statistic, , is different, involve 2 samplesobszSTA 291 - Lecture 24 18Alternative and p-value computationOne-Sided TestsTwo-Sided TestalternativeHypothesisp-value12:0AHpp−<012:0Hpp−=12:0AHpp−>12:0AHpp−≠()obsPZz<()obsPZz>2(||)obsPZz⋅>STA 291 - Lecture 24 19Two p’s=−=−=−−+0120121212: which is equivalent to :0,ˆˆˆˆˆˆ(1)(1)obsHppHppppzppppnnSTA 291 - Lecture 24 20• Where the in the denominator is the combined (pooled) sample proportion.= Total number of successes over total number of observationsSo there are 3 different sample proportions: from sample one, from sample two and from both samples.ˆp• P for P-value in a test hypothesis setting• p for population proportion•• for sample proportion • for the hypothesized population proportion valueSTA 291 - Lecture 24 21ˆp0pSTA 291 - Lecture 24 22Attendance Survey Question 24• On a 4”x6” index card–Please write down your name and section number–Today’s Question: –Probability of making a type II error is denoted by:a. Alpha b. Beta c. PowerExample: compare 2 proportions• A nation wide study: an aspirin every other day can sharply reduce a man’s risk of heart attack. (New York Times, reporting Jan. 27, 1987)• Aspirin group: 104 Heart Att. in 11037• Placebo group: 189 Heart Att. in 11034• Randomized, double-blinded studySTA 291 - Lecture 24 23Example – cont.• Let aspirin = group 1; placebo = group 2p1 = popu. proportion of Heart att. for group 1p2 = popu. proportion of Heart att. for group 2STA 291 - Lecture 24 24=−=012012: which is equivalent to :0HppHpp≠−≠1212: or :0AAHppHppExample – cont.• We may use software to compute a p-value• p-value = 7.71e-07 = 0.000000771Or we can calculate by hand:STA 291 - Lecture 24 25−=−−+1212ˆˆˆˆˆˆ(1)(1)obsppzppppnnExample – cont.•