Tests of HypothesesBios 662Michael G. Hudgens, [email protected]://www.bios.unc.edu/∼mhudgens2006-09-19 16:40BIOS 662 1 Tests of HypothesesTest of Hypotheses1. Set up a hypothesis2. Collect data3. Infer from the data whether the hyp is true• Examples:Is BP the same in diabetics and non-diabetics?Will folic acid supplementation reduce the risk of stroke?BIOS 662 2 Tests of HypothesesTests of Hypotheses: H0• Null hypothesis H0: hypothesis to be tested• Example null hyp for folic acid study:the incidence of stroke will be the same in those takingfolic acid supplements and those not taking folic acidsupplements• See Note 4.16 text: typically H0(i) prevailing view or straw man(ii) most parsimonious hypothesisBIOS 662 3 Tests of HypothesesTest of Hypotheses• In a test of a hyp, we are testing whether some popula-tion parameter has a particular value• For example,H0: θ = θ0where θ0is a known constant• The alternative hypothesis is complement of null hypHA: θ 6= θ0BIOS 662 4 Tests of HypothesesTests of Hypotheses• Once the data are collected, we will compute a teststatistic related to θ, say S(ˆθ)• S(ˆθ) is a random variable, since it is computed from asample• S(ˆθ) will have a particular probability distribution un-der the assumption H0, say F0[S(ˆθ)]BIOS 662 5 Tests of HypothesesTests of Hypotheses• Using F0, we compute the probability that we wouldobserve S(ˆθ) or a value more extreme than S(ˆθ) if thenull H0was true• If the probability is large, the data are consistent withH0• If the probability is small, two possibilities:1. An event with small probability has occurred2. H0is not trueBIOS 662 6 Tests of HypothesesTests of Hypotheses• Usually if the prob is small, we conclude H0is not true;i.e., we “reject” H0• If the prob is large, we have not proved H0. We saythat “we failed to reject H0”• We can never prove H0is true!• Also: don’t “accept the alternative”BIOS 662 7 Tests of HypothesesTests of Hypotheses• How do we decide if the probability is too small?• Prior to seeing the data, we select a value α such that:if the computed probability is less than or equal to α,we reject H0• α is known as significance levelBIOS 662 8 Tests of HypothesesTests of Hypotheses• We have a statistic S(ˆθ) with distribution F0under thenull hyp• We specify α and using F0determine a critical regionCαsuch thatPr[S(ˆθ) ∈ Cα|H0] = α• Values at the boundaries of Cαare call critical valuesBIOS 662 9 Tests of HypothesesTests of Hypotheses• From the data we compute S(ˆθ)• If S(ˆθ) ∈ Cα, we reject H0• If S(ˆθ) /∈ Cα, the data are consistent with H0and wedo not reject H0BIOS 662 10 Tests of HypothesesTests of Hypotheses1. Design study2. Establish null hyp3. Determine test statistic to be employed4. Choose significance level α and establish Cα5. Carry out study and collect data6. Compute statistic from data7. If statistic is in Cα, reject H0BIOS 662 11 Tests of HypothesesTests of Hypotheses: Example• Does calcium supplementation affect blood pressure inAfrican Americans with high blood pressure?• Study: take 10 AA men with hypertension; measureBP; ask them to take calcium tablets for 3 weeks andre-measure their BPBIOS 662 12 Tests of HypothesesTests of Hypotheses: Example• Let θ denote the mean BP change after 3 weeks• HypothesesH0: θ = 0 vs HA: θ 6= 0• Note sometimes the notation θ0will be used to denotethe value of θ under H0; in this case, θ0= 0• Let Yi= BP at 3 weeks - BP at baseline for the ithindividual in the study, i = 1, . . . , 10•ˆθ =¯YBIOS 662 13 Tests of HypothesesTest of Hypotheses: Example• Intuition: want to reject H0if¯Y is far from θ0= 0, i.e.,if|¯Y | > cfor some constant c• In particular, want c such thatPr[|¯Y | > c|H0] = α• Or, equivalently,Pr[−c ≤¯Y ≤ c|H0] = 1 − αBIOS 662 14 Tests of HypothesesTest of Hypotheses: Example• Equivalently, want c such thatPr[−cs/√n≤¯Ys/√n≤cs/√n|H0] = 1 − α• Assuming Yiare iid N(µ, σ2), under H0¯Ys/√n∼ tn−1• Thus choose c such thatcs/√n= tn−1,1−α/2BIOS 662 15 Tests of HypothesesTest of Hypotheses: Example• So we reject H0if|¯Y | > c = tn−1,1−α/2s/√ni.e. if|¯Y |s/√n> tn−1,1−α/2• EquivalentlyCα= {t : |t| > tn−1,1−α/2}wheret =¯Ys/√nBIOS 662 16 Tests of HypothesesTests of Hypotheses: Example• Returning to calcium example,S(ˆθ) = S(¯Y ) =¯Y − θ0s/√n∼ t9• If α = 0.05, the critical region isCα= {t : |t| > t9,.975= 2.26}wheret =¯Y − 0s/√10BIOS 662 17 Tests of HypothesesTests of Hypotheses: ExampleCalcium supplementation in African-American mentreatment before after diff1. calcium 107 100 -72. calcium 110 114 43. calcium 123 105 -184. calcium 129 112 -175. calcium 112 115 36. calcium 111 116 57. calcium 107 106 -18. calcium 112 102 -109. calcium 136 125 -1110. calcium 102 104 2BIOS 662 18 Tests of HypothesesTests of Hypotheses: ExampleTest of blood pressure change in calcium treated groupOne-sample t test Number of obs = 10------------------------------------------------------------------------------Variable | Mean Std. Err. t P>|t| [95% Conf. Interval]---------+--------------------------------------------------------------------diff | -5 2.764859 -1.80841 0.1040 -11.25455 1.254545------------------------------------------------------------------------------Degrees of freedom: 9Ho: mean(diff) = 0Ha: mean ~= 0t = -1.8084BIOS 662 19 Tests of HypothesesTests of Hypotheses: Example• Since the observed t = −1.8084 is not in the criticalregion, we do not reject H0• The data are consistent with no effect of calcium on BPBIOS 662 20 Tests of HypothesesTests of Hypotheses: ErrorsNatureH0true HAtrueDo not reject H0√Type IIDecisionReject H0Type I√• Type I error: Reject H0when H0truei.e. false positive• Type II error: do not reject H0when HAtruei.e. false negativeBIOS 662 21 Tests of HypothesesTests of Hypotheses: Errors• Type I errorα = Pr[S(ˆθ) ∈ Cα|H0]• Type II errorβ = Pr[S(ˆθ) /∈ Cα|HA]• Power1 − β = Pr[S(ˆθ) ∈ Cα|HA]i.e., prob reject H0when HAis trueBIOS 662 22 Tests of HypothesesTests of Hypotheses: Power• Recall HA: θ 6= θ0• Power: Pr[S(ˆθ) ∈ Cα|HA]• Power depends on the value of θPr[S(ˆθ) ∈ Cα|θ] ≡ P (θ)• NoteP (θ0) = αBIOS 662 23 Tests of HypothesesTests of Hypotheses: P-value and HA• p-value: the probability of obtaining a test result as ormore