Summary of methodThe method is designed to test whether the mean of some pop-ulation, µ say, is equal to some predetermined value µ0, where µ0could be the result for a larger population, or from an older sur-vey, or could be some target value (e.g. specifying that the meanconcentration of an environmental pollutant should be lower thana safe level determined by the EPA).We assume we have a sample mean ¯x, with a standard deviations, calculated from a random sample of size n.11. Determine H0: µ = µ0and Ha(either µ > µ0or µ < µ0orµ 6= µ0)2. Calculate the test statistict =¯x − µ0S.E.=¯x − µ0s/√n.3. Calculate df = n − 1.4. Determine the P value — either exactly using Excel or Minitab,or using Table B in the back of the book. Note that if youuse Table B,, you will not be able to determine P exactly butwill be able to identify the nearest value among 0.1, 0.05,0.025, 0.01, 0.005 and 0.001 (one-sided) or among 0.2, 0.1,0.05, 0.02, 0.01 and 0.002 (two-sided),5. Report your conclusions. Usually if P> 0.05 we simply reportthat the result was not statistically significant, but if P≤ 0.05we report that fact and our nearest guess for the exact valueof P.2Example ProblemsQuestion 8.69, page 420. A bank wants to evaluate which of twocredit cards is more attractive to its customers. For a randomsample of 100 customers, 40 say they would prefer one that hasan annual cost but low interest. Software reports:Test of p=0.50 vs p not = 0.50X N Sample p 95& CI Z-value P-value40 100 0.40000 (0.304,0.496) -2.00 0.04550Explain how to interpret these results. What would you reportto the company?3Solution. For calculating a confidence interval, the sample pro-portion is ˆp and the standard error isrˆp(1−ˆp)n=q0.4×0.6100=.049. Hence the 95% CI is 0.4 ± 1.96 × 0.049 = 0.04 ± 0.096 =(0.304, 0.496).For the hypothesis test of H0: p = p0= 0.5, we recalculatethe standard error based on p0, asq0.5×0.5100= 0.05. Then the zstatistic is0.4−0.50.05= −2.0, and the corresponding one-sided P-value is .0228 (from Table A). Therefore, the two-sided P-valueis 2 × 0.0228 = 0.0456. [Slight rounding error here: the exactone-sided P-value is .02275013 and twice that would be .0455,but this makes no difference in practice.]Report to the company: Less than half the customers favor thenew card, and that is a statistically significant difference (thoughonly barely).4Question 8.73, page 420.For a quantitative variable you want to test H0: µ = 0 againstHa: µ 6= 0. The 10 observations are 3,7,3,3,0,8,1,12,5,8.1. Show: (a) ¯x = 5.0, (b) s = 3.71, (c) standard error is 1.17,(d) test statistic is 4.26, (e) df=9.2. The P-value is 0.002. Interpret, and make a decision usinga significance level of 0.05.3. If you had instead used Ha: µ > 0, what would the P-valuebe? Interpret.4. If you had instead used Ha: µ < 0, what would the P-valuebe? Interpret.5Solution1. (a) and (b) are direct calculation (you could also use Excel).(c)3.71√10= 1.17. (d) t =5−01.17= 4.27 (slight rounding error).(e) df = n − 1 = 10 − 1 = 9.2. Since 0.002 < 0.05, we reject the null hypothesis. Basedon the data, there seems to be significant evidence that themean is not 0.3. For a one-sided test, the P-value is half that for a two-sidedtest, i.e. 0.001. This also leads to the conclusion that themean is not 0.4. The P-value for Ha: µ < 0 would be the probability thatt < 4.27 based on df = 9. This is 1 minus the probabilitythat t > 4.27, i.e. about 0.999. In this case, since the onlyalternative of interest is µ < 0 and the data support µ > 0,we have no choice but to accept H0.6Limitations of Significance Tests1. Statistical significance does not necessarily mean practicalsignificance.2. Significance tests are generally less useful than confidenceintervals.7Example. Suppose we take a group of 200 patients whose meancholesterol level is 250 or higher. We given them a cholesterol-reducing drug and measure the reduction in cholesterol aftersome period of time. Over all the patients, the mean reductionin cholesterol is 10 points and the standard deviation is 50.Is this a significant result?8The standard error is50√200= 3.53 so we calculate z =103.53=2.83. The associated P-value is approximately 0.002 (one-sided).So it is (statistically) significant.But is it practically significant? Most doctors recommend acholesterol level lower than 200 — a reduction of 10 points isnot necessarily of practical significance for someone over 250.In other words, the difference is big enough that it could nothave occured by chance (that’s the definition of statistical sig-nificance), but that doesn’t mean the drug is effective in practice.9The alternative approach is to calculate a confidence interval forthe mean reduction in cholesterol.Since ¯x = 10 and the standard error is 3.53, a 95% confidenceinterval is 10 ± 1.96 × 3.53 = (3.1, 16.9). In the opinion of moststatisticians, that gives a much clearer picture of the uncertaintyin the data than simply saying we rejected the null hypothesisµ = 0.10Common Misinterpretations of Significance Tests1. If a significance test results in the conclusion “do not rejectH0”, that does not mean H0is true.2. The P-value is not the probability that H0is true.3. Many medical researchers only report their results if they arestatistically significant. This leads to publication bias.4. Some results are statistically significant by chance. There-fore, you cannot conclude that every “significant” result pub-lished in the literature is true.11Type I and Type II ErrorsType I Error: H0true but rejectedType II Error: H0false but not rejectedThe significance level is the probability of a type I error.12ExampleA new drug therapy is proposed to reduce the risk of heart attack.Among the category of patients for whom the drug is intended,the chance of a heart attack within 5 years is considered to be25%.We test the drug on 120 patients so that the type I error is0.01 in a one-sided test (H0: p = 0.25 against Ha: p < 0.25).Suppose the drug is effective, so that for a patient who takes thedrug, the chance of a heart attack within 5 years is only 10%.What is the probability of a type II error?13First define the test for a significance level of 0.01.The standard error isq0.25×0.75120= 0.0395.Reject H0if ˆp < 0.25 − 2.33 × 0.0395 = 0.158.(The z value 2.33 comes from the 0.01 significance level, viaTable A.)If p = 0.1, the standard error is nowq0.1×0.9120= 0.0274.The probability that ˆp < 0.158 is