1Statistical Inference2Two simple examples Lady tasting tea Human energy fields These examples provide the intuition behind statistical inference3Fisher’s exact test A simple approach to inference Only applicable when outcome probabilities known  Lady tasting tea example Claims she can tell whether the milk was poured first In a test, 4/8 teacups had milk poured first The lady correctly detects all four Should we believe that she has milk-first detection ability? To answer this question, we ask, “What is the probability she did this by chance?” If likely to happen by chance, then we shouldn’t be convinced If very unlikely, then we should maybe believe her This is the basic question behind statistical inference Null hypothesis People seem poorly equipped to make these inferences, in part because they forget about failures, but notice success: e.g. Dog ESP, miracles  Other examples: fingerprints, DNA, HIV tests, regression coefficients, mean differences, etc. Answer? 70 ways of choosing four cups out of eight How many ways can she do so correctly?4Lady tasting tea: Prob. of identifying by chance0.0140.2290.5140.2290.0144 3 2 1 0# Milk-first teacups correctly identifiedBy chance, she would only guess all four correctly with probability (1/70 = ) 0.014. So, we can be quite confident in her milk-first detection ability.5Second simple exampleHealing touch: human energy field detection“A Close Look at Therapeutic Touch” Linda Rosa; Emily Rosa; Larry Sarner; Stephen Barrett. 1998. JAMA(279: 1005 – 1010)6Human energy field: Prob. of success by chance0.000.010.040.120.210.250.210.120.040.010.000 1 2 3 4 5 6 7 8 9 10# of successful detectionsn (the number of trials) = 10k (number of successes)p (prob. of success for the null model) = .5Binomial distribution7Human energy field detection: Confidence in ability0.060.080.110.140.180.230.280.340.400.470.530.600.660.720.770.820.860.890.920.940.960.970.980.990.990.9965676971737577798183858789# of successful detections out of 150 trials8Linda Rosa; Emily Rosa; Larry Sarner; Stephen Barrett. 1998. “A Close Look at Therapeutic Touch” JAMA, 279: 1005 - 1010.9Null hypothesis In both cases, we calculated the probability of making the correct choice by chance and compared it to the observed results. Thus, our null hypothesis was that the lady and the therapists lacked any of their claimed ability. What’s the null hypothesis that Stata uses by default for calculating p values? Always consider whether null hypotheses other than 0 might be more substantively meaningful. E.g., testing whether the benefits from government programs outweigh the costs.10Assessing uncertainty With more complicated statistical processes, larger samples, continuous variables, Fisher’s exact test becomes difficult or impossible Instead, we use other approaches, such as calculating standard errors and using them to calculate confidence intervals The intuition from these simple examples, however, extends to the more complicated one11Standard error: Baseball example In 2006, Manny Ramírez hit .321 How certain are we that, in 2006, he was a .321 hitter? Confidence interval? To answer this question, we need to know how precisely we have estimated his batting average The standard error gives us this information, which in general is (where s is the sample standard deviation)  Equation?nserr. std.12Baseball example The standard error (s.e.) for proportions (percentages/100) is? For n = 400, p = .321, s.e. = .023 Which means, on average, the .321 estimate will be off by .02302.01000)37.1(37.)1(npp13Baseball example: postseason 20 at-bats N = 20, p = .400, s.e. = .109 Which means, on average, the .400 estimate will be off by .109 10 at-bats N = 10, p = .400, s.e. = .159 Which means, on average, the .400 estimate will be off by .15914Using Standard Errors, we can construct “confidence intervals” Confidence interval (ci): an interval between two numbers, where there is a certain specified level of confidence that a population parameter lies ci = sample parameter + multiple * sample standard errorConfidence intervalyMean.000134.39894223423468%95%99%N = 20; avg. = .400; s = .489; s.e. =.109.400.400+.109=.511.400-.109=.290.400+2*.109=.615.400-2*.109=.185s.e. is estimate of σ16 Much of the time, we fail to appreciate the uncertainty in averages and other statistical estimates Postseason statistics Boardgames Life17Two types of inference Testing underlying traits E.g., can lady detect milk-poured first? E.g., does democracy improve human lives? Testing inferences about a population from a sample What percentage of the population approves of President Bush? What’s average household income in the United States?Example of second type of inference:Testing inferences about a population from a sampleFamily income in 200619Certainty about mean of a population based on a sample: Family income in 20060.002.004.006.008.01Density0 50 100 150 200Histogram of Family IncomeX= 65.8, n = 31,401, s = 41.7Source: 2006 CCES20Calculating the Standard Error on the mean family income of $65.8 thousand dollarsEquation?nserr. std.For the income example,std. err. = 41.6/177.2 = $0.23 thousands of dollarsX= 65.8, n = 31401, s = 41.7The PictureyMean.000134.39894223423468%95%99%N = 31,401; avg. = 65.8; s = 41.6; s.e. = s/√n = .265.865.8+.2=66.065.8-.2=65.665.8+2*.2=66.265.8-2*.2=65.4Where does the bell-shaped curve come from?That is, how do we know that two + standard errors covers 95% of the distribution?23Could this possibly be right? Why? Central limit theorem24Central Limit TheoremAs the sample size n increases, the distribution of the mean of a random sample taken from practically any population approaches a normaldistribution, with mean μ and standard deviation Xn25Illustration of Central Limit Theorem:Exponential DistributionFractioninc0 500000 1.0e+060.271441Mean = 250,000Median=125,000σ = 283,474Min = 0Max = 1,000,00026Consider 10,000 samples of n = 100Fraction(mean) inc0 250000 500000 1.0e+060.275972N = 10,000Mean = 249,993s = 28,559What will the distribution of these means look like?27Consider 1,000 samples of various sizes10 100 1000Mean =250,105s = 90,891Mean = 250,498s = 28,297Mean = 249,938s = 9,376Fraction(mean)