OverviewExampleDefinitionsHypothesesTest StatisticsP-valuesInterpretations and PitfallsProof by ContradictionMore about P-valuesTwo-tailed TestsSummaryBasic SummaryRecipeRevisiting the ContextChapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextStat 220, Part VIIIHypothesis Tests and Statistical DecisionsLecture 22Chapter 26: Tests of Significance(+ a few pages from Ch. 29)Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextContextWe are entering the last, and easily most controversial, part ofthe course material. Here’s a reminder why:Give me the Bottom Line.Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextSome ControversiesAs we saw with the Iraq example, estimates too can becontroversial sometimes. But in general, the tasks of estimationand prediction are rather objective and straightforward: youneed to get close to the true value.By contrast, Part VIII is about using statistics to makedecisions.Here we encounter several layers of controversy:1 About the decision itself. Usually someone will stand togain or lose no matter what is decided.2 About the way statistics is used. Unlike probabilitycalculations (which are either right or wrong), there aremany ways to do this. It is an open problem.3 Moreover, this is where it is easiest to “lie with statistics.”Or just do an extremely poor job without noticing.Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextHypothesis TestsWe will learn the most basic standard approach to statisticaldecisions, known as “classical” or “frequentist”. It has a lot ofadvantages, but it is not perfect.This appr oach focuses on the simplest type of decision:choosing between two options. It is based on tests ofsignificance, more often known as hypothesis tests.A test of significance translates the decision-making problem,into a question whether a set of observations can be reasonablyexplained by chance, or not. These tests are widely used, so it isimportant to understand how they work.New terminology: null hypothesis, alternative hypothesis,test-statistic, P-valueChapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextExample 1A senator introduces a bill that simplifies the tax code. Heclaims that the bill is ’revenue neutral’. This means that thegovernment will not lose money from it. The way the bill isviewed by everyone else depends on whether he is right inclaiming this.To check the claim, the Treasury Department uses a computerfile of 100,000 representative tax returns. For a simple randomsample of 100 returns, it calculateschange = tax under new rules − tax under old rulesThe sample average comes out as a change of -$219, with anSD of $725.Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextExample 1Practically, we need to choose between two differentexplanations for the average change of -$219.(i) The senator is right, and the bill is indeed revenue neutral.In other words, the true average change among the100,000 returns is zero, and the observed average changeof -$219 (from the sample of 100) is all due to chanceerror.(ii) The senator is wrong, and the bill is not revenue neutral.The difference is real, and tax revenues will go down underthe new bill.Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextExample 1The idea of a test of significance is the following. Assume that(i) is true. Then the observed difference of -$219 is due tochance variability. We can set up a box model:100,000 tickets, each showingthe change for that taxpayer.Average of box: $0SD of box: $725Under assumption (i), the average change is like the average of100 draws from the box.Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextExample 1Expected value for the average: $0SE for the average:$725√100= $72.50The observed average is -$219, or(−$219) − ($0)$72.50≈ −3in standard units (standardized by the SE for the average).Thanks to the CLT, we assume the probability histogram forthe average of the draws is very close to the normal curve.Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextExample 1The area under the normal curve between -3 and 3 is about99.7%. The chance in our example is equal to the area underthe curve to the left of -3. That area is approximately 0.15%.So if explanation (i) is true, there is only one chance in 600 orso, to get an average of -$219 or below. This is strong evidenceagainst explanation (i). If we stick to it, we need a smallmiracle to explain the data. What will you decide?Chapter 26OverviewExampleDefinitionsHypothesesTestStatisticsP-valuesInterpretationsand PitfallsProof byContradictionMore aboutP-valuesTwo-tailedTestsSummaryBasicSummaryRecipeRevisiting theContextThe Null and the AlternativeNull and Alternative Hypotheses: definitionIn a test of significance, the null hypothesis expresses the ideathat an observed difference is due to chance. To make a test,the null hypothesis has to be set up as a box model for the data.The alternative hypothesis is another statement about the box;it says that the difference is real.In Example 2,•null hypothesis - explanation (i); the average of the boxequals $0•alternative hypothesis - explanation (ii); the average of thebox is less than $0Chapter