Chapter 13Analysis of Variance (ANOVA)ANOVA is an extension of what you have learned inChapter 9. ANOVA techniques compare the means ofseveral groups (two or more populations) using anindependent sample from each group.Identifying groups as a categorical variable, we havea quantitative variable (the variable of interest) that isexplained by a categorical variable (the groups).In ANOVA, the categorical variables that identify thegroups are called the factors. When there is only onefactor the technique is called one–way ANOVA. 13.1 One–way ANOVASuppose there is one factor with g levels (groups orpopulations, called treatments in ANOVA). In eachtreatment group we measure a quantitative responsevariable and compare the means of the response ineach treatment group. One of the hypotheses of interest is the equality ofthe population means, i.e., Ho: µ1 = µ2 = … = µg. IfHo is not true, what can we say about the relationbetween the means?Chapter 13, Fall 2007 Page 1 of 45Ha: At least two of the population means are unequal.Equivalently we may specify these hypotheses asHo: µ1 = µ2 = … = µg = µ vs.Ha: At least one of µi ≠ µ.[See Figure 13.2 for a graphical example]Analysis of variance:To test if there are significant differences between thepopulation means the test statistic is22BetweenWithinVariability between groupsFVariability within groupsss= =- If thereIS a significant difference between groupmeans, then variability between groups >>variability within groups, i.e., F will be verylarge.- If there is NO significant difference betweengroup means, then variability between groupswill be approximately equal to the variabilitywithin groups, i.e. F ≈ 1Chapter 13, Fall 2007 Page 2 of 45ANOVA TESTS:1. Assumptionsa) Random samplesb) Normal populations (i.e., distribution of theresponse variable is normal in each group).Critical for small samples, not too importantfor large samples.c) Equal variances for all treatment groups(populations). If sample sizes are equal thenthis assumption is not crucial. Otherwise checkif two times the smallest sample standarddeviation is more than the largest samplestandard deviation.2. Hypotheses:Null hypothesis: Ho: µ1 = µ2 = … = µg = µAlternative hypothesis: Ha: At least one µi ≠ µ3. Test statistic: F = MSB / MSW4. The p-value: Right tail probability Compare with values from F-tables at theintersection of df1 = g – 1 and df2 = N – g, whereN = sum of all samples = Total number ofobservations and g = number of groups.5. Decision Rule: Reject Ho if p-value - α always.6. Conclusion: State decision in layman’s languageChapter 13, Fall 2007 Page 3 of 45ANOVA TABLESource df SS MS FBetween Groups g – 1 SSB1SSBMSBg=-MSBFMSE=Within Groups(Error)N – g SSESSEMSEN g=-Total N – 1 SSTNotation:yij = jth observation in ith group, j = 1, 2, …, ni; i = 1, 2, …, giy= 1/inij ijy n=� = sample mean for ith groupy= Overall sample mean= Average of all observations=1 1 1/ /ing gij i ii j iy N n y N= = =� �= �� �� ��� �ni = number of observations from ith groupN = gii 1n=�= n1 + n2 + … + ng = Total number of observationsChapter 13, Fall 2007 Page 4 of 45g = Number of groups (treatments)Chapter 13, Fall 2007 Page 5 of 45Sum of Squares21( )gi iiSSB n y y== -�= Between sum of squares21( 1)gi iiSSE n S== -�= Error (within) sum of squares21 1( )ingiji jSST y y= == -��= total sum of squares2 21( ) /( 1)ini ij i ijS y y n== - -� = St. Deviation for group iMean Squares = SS / df (General rule)MSB = SSB / (g – 1), MSE = SSE / (N – g)Example: Compare the average weight loss for 3diets after a month. LowFat LowCal LowCarb22 24 2818 21 2721 26 3025 27 32121.5y =224.5y =329.25y =12.887S =22.646S =32.217S =14n =24n =34n =Chapter 13, Fall 2007 Page 6 of 45Identify the symbols: N = 12 = Total number of observationsg = 3 = number of groups.25.08y = = overall meanUsing the above formula and the given data can youshow that SSB = 122.17 and SSE = 60.75. Complete the ANOVA table.Source df SS MS FBetween Groupsg – 1 122.171SSBMSBg=-MSBFMSE=Within Groups(Error)N – g 60.75SSEMSEN g=-Total N – 1 SSTSource df SS MS FBetween Groups2 122.17..=122 17261 085..=60 7696 75Within Groups (Error)9 60.75..=60 7696 75Total 11 182.92Conduct the ANOVA test using the above table.Chapter 13, Fall 2007 Page 7 of 451. Assumptions a. Random sample of people who are overweightand random assignment to each diet (Hope so)b. Normal distribution of weight loss in all threethe populations (Check to see if there is anyoutlier)c. Equal population variances for all 3 groups(Compare sample standard deviations: Is thesmallest S - 2 times > Largest S?(2.217) - 2 = 4.434 > 2.887 OKNote that since all ni are equal (= 4) we donot need to check this.2. Hypotheses:Ho: µ1 = µ2 = µ3 (No difference) vs.Ha: Not all µi equal3. Test Statistic Fcal = 9.05 (From ANOVA table)4. The p-value: From the table of the F-distributionwe obtain 4.26 (at df1 = 2 and df2 = 9). This meansP(F - Fcal) = P(F - 4.26) = 0.05. Since we foundFcal = 9.05 which is much larger than 4.26, we canwrite p-value = P(F - 9.05) < 0.055. Decision: Reject Ho at 5% level of significance6. Conclusion: The observed data indicatedifferences in the average weight loss by threediets of the populations of all overweight peoplewho use these diets at 5% level of significance.Some notes about the F-distribution:i. Skewed to the rightChapter 13, Fall 2007 Page 8 of 45ii. Starts at zero, i.e., F > 0 ALWAYSiii. Peak (mode) at 1, iv. So if F ≈ 1 do not reject Hov. P-value = Area to the right of Fcalvi. Table gives values of F such that P(F - table value) = 0.05 = upper tail areaIn the above example we have rejected Ho. Thismeans that the data indicate difference(s) inpopulation means. WHICH POPULATION MEAN(S) IS/AREDIFFERENT FROM THE OTHERS?To answer this we must compare the populations inpairs. This is called “pair-wise comparison.” Wecarry out pair-wise comparisons ONLY when Ho isrejected.Chapter 13, Fall 2007 Page 9 of 4513.2 Pair-wise comparisons: Follow up to ANOVAWhen using ANOVA if we decide to reject Ho, weconclude that at least one group has a mean differentfrom the others. However, we do not know whichone or ones are difference from the others. So we