Testing Hypotheses12Motivating discussionA newspaper story discussed the possible increase of skin cancerin marathon runners, based on a paper Malignant melanomain marathon runners, by Ambros-Rudolph et al. (a group ofresearchers at the Medical University of Graz, Austria).The initial research was prompted by a group of 8 ultramarathon-ers with malignant melanoma (Table 1).34To study this in more detail, the researchers recruited 210 marathonrunners (166 male, 44 female), all white. They also formed acontrol group of 210 white non-runners, matched by age and sexto the marathon running group. Each participant was given aquestionnaire to identify risk factors (skin type etc.), then a fulldermatological examination and skin cancer screening.In the marathon group, 24 had potentially cancerous moles orlesions that were referred for further treatment; in the controlgroup, only 14 did.Is this a statistically significant difference?Detailed results are in Table 2 of the paper.56“Significant” results in the table are marked by P =.04, .03, .02,.01 or .001. It is worth pointing out that although a numberof results in that table are flagged as statistically significant,the result about 24 v. 14 referrals for treatment is not flaggedas statistically significant. However, among the “high training”group (more than 70km. per week), 6 out of 31 runners had tobe referred and that is statistically significant, according to thetable.7My conclusion was that both the newspaper report and the ti-tle of the article itself exaggerated what the study had actuallyproved. The study made a number of comparisons between thetreatment and control groups, but most of them were not sta-tistically significant, and some of them showed that it was thecontrol group that was at greater risk.However, this is also a small study. Despite the increase in skincancer in recent years, it’s still a relatively rare disease — wellunder 1% of the total population. In 420 subjects in the study,the researchers may well not have seen enough cases to make ameaningful comparison.In this chapter, we discuss some general principles related tostatistical significance and P-values that often come out in thissort of study.8Steps to performing a significance testThe melanoma example is actually a little different from theexamples discussed in this chapter because it is about the com-parison of two proportions — whether the proportion of a certainskin problem among marathon runners is higher than among thecontrol group (in a situation where both proportions are un-known in the population at large). This is actually the subjectof Chapter 9. For the purposes of Chapter 8, let’s pretendthat the control group was actually much larger, and that the14/210 in the control group who had to be referred for treatmentwas actually representative of the whole population — 6.7% orp = 0.067. The question that then arises is whether either theproportion that had to be refereed for treatment among themarathon runner sample (24/210 or 11.43%), or the proportionamong the “heavy trainers” (6/31 or 19.35%), are statisticallydifferent from the general population.9In the text they use an example related to astrology — an as-trologer is given three possible personality profiles correspondingto a particular individual, and he/she has to guess which one iscorrect based on the individual’s birth date. If there is no astro-logical effect, the proportion of correct guesses will be p =13. Ifthere is an effect, presumably the astrologer will guess correctlygreater than one-third of the time. As with the skin cancerexample, the question is whether the observed proportion of aparticular outcome in an experiment is significantly different fromthe proportion we would expect to see by chance if there was noeffect.10Five steps to performing a significance test:(1) Specify the assumptions. For example, many (if not most)studies require randomization.(2) Define the hypotheses of interest. Typically, this kind ofproblem is formulated as a choice between the null hypothesisand the alternative hypothesis. The null hypothesis meansthere is no effect, e.g. the proportion of skin problems amongmarathon runners is no different from that of the generalpopulation, or the astrologer guesses correctly only one-thirdof the time. The alternative hypothesis is when the nullhypothesis is not correct. The null hypothesis and alternativehypothesis are often written H0and Ha. So in the skin cancerexample we may say that if p is the proportion of marathonrunners referred for treatment,H0: p = 0.067, Ha: p > 0.067.11(3) Test statistic: Calculate some summary of the data thatmay be used to discriminate between the null and alternativehypotheses.(4) P-value: Calculate the probability that a result, equal to ormore extreme than the one actually observed, would occurif H0was correct. This is called the P-value (not to beconfused with small p which represented the proportion wewere trying to test).(5) Report the conclusions. If the P-value is sufficiently small,we conclude that the null hypothesis is very unlikely to becorrect and therefore conclude that the alternative hypothesisis correct.12Consider the skin cancer example applied to the “heavy training”group (6 out of 31 runners referred for treatment). In that case,these steps work out as follows:(1) Assume that X, the number of marathon runners referredfor treatment, has a binomial distribution with n = 31 andunknown p.(2) The natural null and alternative hypotheses are H0: p =0.067 and Ha: p > 0.067.(3) The test statistic is the sample proportion ˆp =631= 0.1935.If H0is correct, the standard error of ˆp isrp(1−p)n=q0.067×0.93331=.0449. Therefore, the z statistic isz =0.1935 − 0.0670.0449= 2.82.13(4) Referring to the normal distribution table, the probabilitythat a standard normal random variable is greater than 2.82is .0024. Therefore, the P-value is .0024 in this case.(5) The value .0024 is rather small — well under a 1% probabilitythat this result could have occurred by chance. Therefore,we’d be justified in concluding that p is not 0.067 — inother words, the marathon runners really did have a higherincidence of this particular skin problem.14One comment here is that we used the normal approximationto the binomial distribution when one of the conditions for thatto be value, that np ≥ 15, is violated. In fact, if n = 31 andp = .067, then np = 2.08 so this is definitely wrong. In thepresent case, we