Two-sample hypothesis testing, IMore on the tradeoff between Type I and Type II errorsMore on the tradeoff between Type I and Type II errorsMore on the tradeoff between Type I and Type II errorsMore on the tradeoff between Type I and Type II errorsMore on the tradeoff between Type I and Type II errorsFor a two-tailed testType I and Type II errorsStatistical powerExample: why we care about powerExample: why we care about powerExample: why we care about powerPower and response curvesHow to compute power (one-sample z-test example)How to compute power (one-sample z-test example)How to compute power (one-sample z-test example)Computing power: an exampleHow to figure out the power of a z-testFirst, find the criterion for rejecting the null hypothesis with a=0.05Step 2Step 2How to increase powerIncrease aReduce SE by either reducing SD, or increasing NIncrease the difference in meansOne-sample vs. two-sample hypothesis testingExample one-sample situationsExample two-sample questionsRecall the logic of one-sample tests of means (and proportions)Recall the logic of one-sample tests of means (and proportions)The logic of hypothesis testing for two independent samplesTwo-Sample t-testMean and variance of the sampling distribution of the difference between two meansMean and variance of the sampling distribution of the difference between two meansShape of the sampling distribution for the difference between two meansExample: z-test for large samplesExample 1: Is there a significant difference between math scores in 1978 vs. 1992?Example 1: Is there a significant difference between math scores in 1978 vs. 1992?Example 2: Test for significant difference in proportionsExample 2: Is there a significant difference in computer use between men and women?Example 2: Is there a significant difference in computer use between men and women?When can you use the two-sample z-test?When do you not have independent samples (and thus should run a different test)?When do you not have independent samples (and thus should run a different test)?Small sample tests for the difference between two independent meansCase 1: Sample size is small, standard deviations of the two populations are equalCase 1: Sample size is small, standard deviations of the two populations are equalOK, we’re ready for an exampleEffect of reward on motor learningEffect of reward on motor learningEffect of reward on motor learningComputing confidence intervals for the difference in meanTwo-sample hypothesis testing, I9.073/09/2004• But first, from last time…More on the tradeoff between Type I and Type II errors• The null and the alternative:µοµaSampling distributionof the mean, m, given mean µa. (Alternative)This is the mean for thesystematic effect. Often we don’t know this.Sampling distribution of the mean, m, given mean µo. (Null)More on the tradeoff between Type I and Type II errors• We set a criterion for deciding an effectis significant, e.g. α=0.05, one-tailed.µοµacriterionα=0.05More on the tradeoff between Type I and Type II errors• Note that α is the probability of saying there’s a systematic effect, when the results are actually just due to chance. = prob. of a Type I error.µοµacriterionα=0.05More on the tradeoff between Type I and Type II errors•Whereas β is the probability of saying the results are due to chance, when actually there’s a systematic effect as shown. = prob. of a Type II error.µοµacriterionαβMore on the tradeoff between Type I and Type II errors• Another relevant quantity: 1-β. This is the probability of correctly rejecting the null hypothesis (a hit).µοµacriterion1−ββFor a two-tailed testAccept H0Reject H0Reject H01−β(correctrejection)α(Type I error)β(Type II error)Type I and Type II errors• Hypothesis testing as usually done is minimizing α, the probability of a Type I error (false alarm).• This is, in part, because we don’t know enough to maximize 1-β (hits).• However, 1-β is an important quantity. It’s known as the power of a test.1−β = P(rejecting H0| Hatrue)Statistical power• The probability that a significance test at fixed level α will reject the null hypothesis when the alternative hypothesis is true.= 1 - β• In other words, power describes the ability of a statistical test to show that an effect exists (i.e. that Hois false) when there really is an effect (i.e. when Hais true).• A test with weak power might not be able to reject Hoeven when Hais true.Example: why we care about power• Suppose that factories that discharge chemicals into the water are required to prove that the discharge is not affecting downstream wildlife.• Null hypothesis: no effect on wildlife• The factories can continue to pollute as they are, so long as the null hypothesis is not rejected at the 0.05 level.Cartoon guide to statisticsExample: why we care about power• A polluter, suspecting he was at risk of violating EPA standards, could devise a very weak and ineffective test of the effect on wildlife.• Cartoon guide extreme example: “interview the ducks and see if any of them feel they are negatively impacted.”Cartoon guide to statisticsExample: why we care about power• Just like taking the battery out of the smoke alarm, this test has little chance of setting off an alarm.• Because of this issue, environmental regulators have moved in the direction of not only requiring tests showing that the pollution is not having significant effects, but also requiring evidence that those tests have a high probability of detecting serious effects of pollution. I.E. they require that the tests have high power.Cartoon guide to statisticsPower and response curvesPermissiblelevels ofpollutantAbovestandardSeriouslytoxicDecreasingpowerProbabilityof “alarm”How to compute power (one-sample z-test example)(1) For a given α, find where the criterion lies.Accept H0αReject H0How to compute power (one-sample z-test example)(2) How many standarddeviations from µais thatcriterion? (What’s its z-score?)??µaHow to compute power (one-sample z-test example)α(3) What is 1-β?Computing power: an example• Can a 6-month exercise program increase the mineral content of young women’s bones? A change of 1% or more would be considered important. • What is the power of this test to detect a change of 1% if it exists, given that we study a sample of 25 subjects?How to figure out the power of a z-test• Ho: µ=0% (i.e. the exercise program has no effect