Significance TestsElements of Significance Test (I)Elements of Significance Test (II)Elements of Significance Test (III)Significance Test for Mean (Large-Sample)Example - Mercury LevelsSlide 7Miscellaneous CommentsExample - Crime Rates (1960-80)Example - Crime Rates (1960-80)Significance Test for a Proportion (Large-Sample)Significance Test for a Proportion (Large-Sample)Decisions in TestsError TypesSlide 15Miscellaneous IssuesSmall-sample Inference for mSmall-Sample 95% CI for mt test for a meanSignificance Tests•Hypothesis - Statement Regarding a Characteristic of a Variable or set of variables. Corresponds to population(s)–Majority of registered voters favor health care reform–Average salary progressions differ for male executives whose spouses work than for those whose spouses “stay at home”•Significance Test - Means of using sample statistics (and their sampling distributions) to compare their observed values with hypothesized value of corresponding parameter(s)Elements of Significance Test (I)•Assumptions–Data Type: Quantitative vs. Qualitative–Population Distribution: Some methods assume normal–Sampling Plan: Simple Random Sampling–Sample Size: Some methods have sample size requirements for validity•Hypotheses –Null Hypothesis (H0): A statement that parameter(s) take on specific value(s) (Often: “No effect”)–Alternative Hypothesis (Ha): A statement contradicting the parameter value(s) in the null hypothesisElements of Significance Test (II)•Test Statistic: Quantity based on the sample data to test the null hypothesis. Typically is based on a sample statistic, parameter value under H0 , and the standard error.•P-value (P): The probability that we would obtain a test statistic at least as contradictory to the null hypothesis as our computed test statistic, if the null hypothesis is true.–Small P-values mean the sample data are not consistent with the parameter value(s) under H0Elements of Significance Test (III)•Conclusion (Optional)–If the P-value is sufficiently small, we reject H0 in favor of Ha . The most widely accepted minimum level is 0.05, and the test is said to be significant at the .05 level.–If the P-value is not sufficiently small, we fail to reject (but not necessarily accept) the null hypothesis.–Process is analogous to American judicial system•H0: Defendant is innocent•Ha: Defendant is guiltySignificance Test for Mean (Large-Sample)•Assumptions: Random sample with n  30, quantitative variable•Null Hypothesis: H0:  = 0 (typically no effect or change from standard)•Alternative Hypothesis: Ha: 0 (2-sided alternative includes both > and <)•Test Statistic:•P-value: P=2P(Z  |zobs|) nsYYzYobs/0^0Example - Mercury Levels•Population: Patients visiting private internal medicine clinic in S.F. (High-end fish consumers)•Variable: Mercury levels (microg/L)•Sample: 66 Females•Recommended maximum level: 5.0 microg/L•Null hypothesis: H0:  = 5.0 (Mean level=RML)•Alternative hypothesis: Ha:   5.0 (Mean RML)•Sample Data: 85.112.8156615661515^YnsYExample - Mercury Levels•Test Statistic: 41.585.11085.1515^0YobsYz• P-Value: P=2P(Z  5.41) < 2P(Z  5.00) = 2(.000000287)= .000000574  0• Conclusion: Very strong evidence that the population mean mercury level is above RMLSource: Hightower and Moore (2003), “Mercury Levels in High-End Consumers of Fish, Environ Health Perspect, 111(4):A233Miscellaneous Comments•Effect of sample size on P-values: For a given observed sample mean and standard deviation, the larger the sample size, the larger the test statistic and smaller the P-value (as long as the sample mean does not equal 0)•Equivalence between 2-tailed tests and confidence intervals: If a (1-)100% CI for  contains 0, the P-value will be larger than •1-sided tests: Sometimes researchers have a specific direction in mind for alternative hypothesis prior to collecting data.Example - Crime Rates (1960-80) •Sample: n=74 Chicago Neighborhoods•Goal: Show the average delinquency rate in the population of all such neighborhoods has increased from 1960-1980•Variable: Y = DR1980-DR1960•H0:  = 0 (No change from 1960-1980)•Ha:  > 0 (Higher in 1980, see Y above)•Sample Data:57.37473.307473.3026.41^YnsYExample - Crime Rates (1960-80)•Test Statistic:6.1157.3026.41^0YobsYz• P-value: (Only interested in larger positive values since 1-sided)0)6.11()(  ZPzZPPobs• Conclusion: Strong evidence that the true mean delinquency rate among all neighborhoods that this sample was taken from has increased from 1960 to 1980.Source: Bursik and Grasmick (1993), “Economic Deprivation and Neighborhood Crime Rates, 1960-1980”, Law & Society Review, Vol. 27, pp 263-284Significance Test for a Proportion (Large-Sample) •Assumptions:–Qualitative Variable–Random sample–Large sample: n 10/min(0 , 1- 0)•Hypotheses:–Null hypothesis: H0: 0–Alternative hypothesis: Ha: 0 (2-sided)–Ha+ : 0 Ha- : 0 (1-sided, prior to data)Significance Test for a Proportion (Large-Sample)•Test statistic:•P-value: –Ha: 0 P = 2P(Z  |zobs|)–Ha+ : 0 P = P(Z  zobs)–Ha- : 0 P = P(Z  zobs)•Conclusion: Similar to test for a meannzobs/)1(000^0^^Decisions in Tests--level (aka significance level): Pre-specified “hurdle” for which one rejects H0 if the P-value falls b elow it. (Typically .05 or .01)P-Value H0 Conclusion Ha Conclusion.05Reject Accept> .05 Do not Reject Do not Accept• Rejection Region: Values of the test statistic for which we reject the null hypothesis• For 2-sided tests with  = .05, we reject H0 if |zobs|1.96Error Types•Type I Error: Reject H0 when it is true•Type II Error: Do not reject H0 when it is false Test Result –True StateReject H0Don’t RejectH0H0 True Type I Error CorrectH0 False Correct Type II ErrorError Types•Probability of a Type I Error: -Level (significance level)•Probability of a Type II Error:  - depends on the true level of the parameter (in the range of values under Ha ).•For a given sample size, and variability in data, the Type I and Type II error rates