Review for Exam 26. Statistical Inference: Significance TestsFive Parts of a Significance TestSlide 4Slide 5Significance Test for MeanSignificance Test for a Proportion Slide 8Error TypesLimitations of significance testsSlide 11se for difference between two estimates (independent samples)CI comparing two proportionsQuantitative Responses: Comparing MeansComments about CIs for difference between two parametersComparing Means with Dependent SamplesChap. 8. Association between Categorical VariablesChi-Squared Test of Independence (Karl Pearson, 1900)Slide 19Residuals: Detecting Patterns of AssociationMeasures of AssociationLimitations of the chi-squared testCh. 9. Linear Regression and CorrelationSlide 24Measuring association: The correlation and its squareSlide 26Review for Exam 2 Some important themes from Chapters 6-9Chap. 6. Significance TestsChap. 7: Comparing Two GroupsChap. 8: Contingency Tables (Categorical variables)Chap. 9: Regression and Correlation (Quantitative var’s)6. Statistical Inference: Significance Tests A significance test uses data to summarize evidence about a hypothesis by comparing sample estimates of parameters to values predicted by the hypothesis. We answer a question such as, “If the hypothesis were true, would it be unlikely to get estimates such as we obtained?”.Five Parts of a Significance Test•Assumptions about type of data (quantitative, categorical), sampling method (random), population distribution (binary, normal), sample size (large?)•Hypotheses: Null hypothesis (H0): A statement that parameter(s) take specific value(s) (Often: “no effect”)Alternative hypothesis (Ha): states that parameter value(s) in some alternative range of values•Test Statistic: Compares data to what null hypo. H0 predicts, often by finding the number of standard errors between sample estimate and H0 value of parameter•P-value (P): A probability measure of evidence about H0, giving the probability (under presumption that H0 true) that the test statistic equals observed value or value even more extreme in direction predicted by Ha. –The smaller the P-value, the stronger the evidence against H0.•Conclusion: –If no decision needed, report and interpret P-value–If decision needed, select a cutoff point (such as 0.05 or 0.01) and reject H0 if P-value ≤ that value–The most widely accepted minimum level is 0.05, and the test is said to be significant at the .05 level if the P-value ≤ 0.05.–If the P-value is not sufficiently small, we fail to reject H0 (not necessarily true, but plausible). We should not say “Accept H0”–The cutoff point, also called the significance level of the test, is also the prob. of Type I error – i.e., if null true, the probability we will incorrectly reject it.–Can’t make significance level too small, because then run risk that P(Type II error) = P(do not reject null) when it is false is too largeSignificance Test for Mean•Assumptions: Randomization, quantitative variable, normal population distribution•Null Hypothesis: H0: µ = µ0 where µ0 is particular value for population mean (typically no effect or change from standard)•Alternative Hypothesis: Ha: µ  µ0 (2-sided alternative includes both > and <, test then robust), or one-sided•Test Statistic: The number of standard errors the sample mean falls from the H0 value0 where /yt se s nsem-= =Significance Test for a Proportion •Assumptions:–Categorical variable–Randomization–Large sample (but two-sided test is robust for nearly all n)•Hypotheses:–Null hypothesis: H0: 0–Alternative hypothesis: Ha: 0 (2-sided)–Ha: 0 Ha: 0 (1-sided)–(choose before getting the data)•Test statistic:•Note •As in test for mean, test statistic has form(estimate of parameter – null value)/(standard error)= no. of standard errors estimate falls from null value•P-value: Ha: 0 P = 2-tail prob. from standard normal dist.Ha: 0 P = right-tail prob. from standard normal dist.Ha: 0 P = left-tail prob. from standard normal dist.•Conclusion: As in test for mean (e.g., reject H0 if P-value ≤ )^^ ^0 00 0(1 ) /znpp p p psp p- -= =-ˆ0 0 0ˆ ˆ(1 ) / , not (1 ) / as in a CIse n se nps p p p p= = - = -Error Types•Type I Error: Reject H0 when it is true•Type II Error: Do not reject H0 when it is false Test Result –True StateReject H0Don’t RejectH0H0 True Type I Error CorrectH0 False Correct Type II ErrorLimitations of significance tests•Statistical significance does not mean practical significance •Significance tests don’t tell us about the size of the effect (like a CI does)•Some tests may be “statistically significant” just by chance (and some journals only report “significant” results)Chap. 7. Comparing Two GroupsDistinguish between response and explanatory variables, independent and dependent samplesComparing means is bivariate method with quantitative response variable, categorical (binary) explanatory variableComparing proportions is bivariate method with categorical response variable, categorical (binary) explanatory variablese for difference between two estimates (independent samples)•The sampling distribution of the difference between two estimates (two sample proportions or two sample means) is approximately normal (large n1 and n2, by CLT) and has estimated 2 21 2( ) ( )se se se= +CI comparing two proportions•Recall se for a sample proportion used in a CI is•So, the se for the difference between sample proportions for two independent samples is •A CI for the difference between population proportions is(as usual, z depends on confidence level, 1.96 for 95% conf.)ˆ ˆ(1 ) /se np p= -2 21 1 2 21 21 2ˆ ˆ ˆ ˆ(1 ) (1 )( ) ( )se se sen np p p p- -= + = +1 1 2 22 11 2ˆ ˆ ˆ ˆ(1 ) (1 )ˆ ˆ( ) zn np p p pp p- -- � +Quantitative Responses: Comparing Means•Parameter: 2-1•Estimator: •Estimated standard error:–Sampling dist.: Approx. normal (large n’s, by CLT), get approx. t dist. when substitute estimated std. error in t stat.–CI for independent random samples from two normal population distributions has form–Alternative approach assumes equal variability for the two groups, is special case of ANOVA for comparing means in Chapter 122 1y y-2 21 21 2s ssen n= +( ) ( )2 21 22 1 2 11 2( ), which iss sy y t se y y tn n- � - � +Comments about CIs for