Introduction to InferenceIntroduction to InferenceTests of SignificancegSection 6.2 (Continued)Today’s objectivesDevelop a nuanced view of significance testing for: Useful interpretation of results Understanding truth, evidence, and decision-making Planning future samples or experimentsRecall: steps of significance testing Formulate the hypotheses, H0versus Ha Calculate the test statistic and the relevant P-value Compare the P-value with a stated significance level, α State your conclusion in contextOne-sample z-test Assumptions: SRS of size n from a population with unknown mean, μ, and known standard deviation σ,μ, Hypotheses*: H0: μ = μ0versus a one- or two-sided Ha Test statistic: P-value: P(Z ≤ z) for Ha: μ < μ0P(Z≤z)forH:μ>μP(Z≤-z) for Ha: μ> μ02P(Z ≤ -|z|)for Ha: μ≠μ0* The null hypothesis is in a standard form, H0: μ = μ0, which might represent H0: μ≥μ0or H0: μ≤μ0. This simplifies concepts, and the procedure is the same.Interpretation of α The significance level, α, is the “long run” proportion of times that H0would be rejected if H0was true*, after 0j0,many, many repetitions of the experiment The test against Ha: μ≠μ0will reject H0at significance level α precisely when μ0lies outside of a (1 – α)100% confidence interval for μμ A P-value is the smallest significance level α at which H0is rejected* Truth of H0sets μ = μ0even if it is formulated as H0: μ≥μ0or H0: μ≤μ0.Introduction to InferenceIntroduction to InferenceUsing Significance TestsSection 6.3What to set α?The required strength of evidence depends mainly on: Plausibility of H0: stronger evidence is required to reject a notion more firmly established in the status quo The consequences of rejecting H0: stronger evidence is required to favor a more radical change inevidence is required to favor a more radical change in thinking.Standard significance levels of α = 0.10, 0.05, and 0.01 are often used purely by conventionWhat does is mean to accept or reject H0? Accepting H0is a failure to observe sufficiently strong evidence against it.g “Accept H0” ≠ “H0is true” Rejecting H0does not necessarily imply a “practically” significant departure from H0. Explore with graphical and descriptive statisticspgp p A conclusion about H0may be invalid if data production was poorly designedSearching for significanceStatistical significance will be found, by chance, if searched for among multiple experimentsgppExample: Cell phones and brain cancer No significant association with brain cancer generally Significant association in 1 of 20 tests of individual ggliomasbutα=005⇒1 in 20 tests are significant“by… but, α 0.05 ⇒1 in 20 tests are significant by chance”Lesson:αis only for a pre-specifiedHversusHLesson:αis only for a pre-specified H0versusHaIntroduction to InferenceIntroduction to InferencePower and Inference as a DecisionSection 6.4Power in significance testing Power is the “long run” proportion of times that H0would be rejected if Hawas true*, after many, many ja,y,yrepetitions of the experiment. Power measures the sensitivity of a test to detect Ha Check when H0is not rejected Power helps to plan new experiments Typically want power to be 0.80* Truth of Hasets μ = μafor some “alternative” value, μa, of the parameter.Example: Exercise and bone densityCan exercise increase bone density? Measure the percent change in bone density before and after an pg yexercise programH0:μ=0versusHa:μ>0H0: μ 0 versusHa: μ 0Variability is σ = 2 percent. Plan to sample n = 25 subjects Setα=005subjects. Set α 0.05.Increase of μa= 1 percent is medically important. At what μapyppower is such an increase detected?Example: Exercise and bone density (continued)Step 1: For what values of the test statistic is H0rejected?jReject H0if P-value = P(Z ≤ -z) ≤α= 0.05.⇔ Reject H0if⇔ Reject ifExample: Exercise and bone density (continued)Step 2: What is P(“reject H0”) if Hais true with μa= 1?If μa= 1, is Cutoff to“RejectH”σ/√n = 2/√25⇒ Power isCutoff to Reject H0μa=1μa 1Suggestions to increase power Increase the significance level, α Protect H0less ⇔detect Hamore easily0ay Examine μafurther from H0 Detect stronger effects more easilyIncrease the sample sizenIncrease the sample size, n More data increases power Decrease population variability, σ, if possible Better data increases powerInference as a decision problemStatistical inference becomes a decision problem when conclusions translate directly to actionsyExample: Acceptance sampling Manufacturer receives “lots” of parts from a supplierSlth thi h ti f d f ti tSome lots have a too-high proportion of defective parts Decide whether to accept or reject a lot by inspecting* a sample of its parts* Inspection is often costly, and sometimes destructive. We want to get the parameters of decision-making just right.Errors in decision makingH0true H0falseReject H0Type I errorCorrect decisionj0ypAccept H0Correct decision Type II error Type I error: reject H0when H0is true P(“Type I error”) = α, the significance level Type II error: accept H0when H0is false P(“Type II error”) = 1 – power Standard approach: strictly control P(“Type I error”) while minimizing P(“Type II