The Effect SizeExamples of Different Types of Effect Sizes: The Major LeaguesExamples of Different Types of Effect Sizes: Two from the Minor LeaguesWhat Makes Something an Effect Size for Meta-Analytic PurposesThe Standardized Mean DifferenceThe Correlation CoefficientThe Odds-RatioMethods of Calculating the Standardized Mean DifferenceThe different formulas represent degrees of approximation to the ES value that would be obtained based on the means and standard deviationsSlide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Data to Code Along with the ESInterpreting Effect Size ResultsSlide 24Translation of Effect SizesMethodological Adequacy of Research BaseConfounding of Study FeaturesConcluding CommentsEffect Size Overheads 1The Effect Size•The effect size (ES) makes meta-analysis possible.•The ES encodes the selected research findings on a numeric scale.•There are many different types of ES measures, each suited to different research situations.•Each ES type may also have multiple methods of computation.Effect Size Overheads 2Examples of Different Types of Effect Sizes:The Major Leagues•Standardized Mean Difference–group contrast research•treatment groups•naturally occurring groups–inherently continuous construct•Odds-Ratio–group contrast research•treatment groups•naturally occurring groups–inherently dichotomous construct•Correlation Coefficient–association between variables researchEffect Size Overheads 3Examples of Different Types of Effect Sizes:Two from the Minor Leagues•Proportion–central tendency research•HIV/AIDS prevalence rates•Proportion of homeless persons found to be alcohol abusers•Standardized Gain Score–gain or change between two measurement points on the same variable•reading speed before and after a reading improvement classEffect Size Overheads 4What Makes Something an Effect Sizefor Meta-Analytic Purposes•The type of ES must be comparable across the collection of studies of interest.•This is generally accomplished through standardization.•Must be able to calculate a standard error for that type of ES–the standard error is needed to calculate the ES weights, called inverse variance weights (more on this latter)–all meta-analytic analyses are weightedEffect Size Overheads 5The Standardized Mean Difference•Represents a standardized group contrast on an inherently continuous measure.•Uses the pooled standard deviation (some situations use control group standard deviation).•Commonly called “d” or occasionally “g”.pooledGGsXXES21   21121222121nnnsnsspooledEffect Size Overheads 6The Correlation Coefficient•Represents the strength of association between two inherently continuous measures.•Generally reported directly as “r” (the Pearson product moment coefficient).rES Effect Size Overheads 7The Odds-Ratio•The Odds-Ratio is based on a 2 by 2 contingency table, such as the one below. FrequenciesSuccess FailureTreatment Group a bControl Group c dbcadES •The Odds-Ratio is the odds of success in the treatment group relative to the odds of success in the control group.Effect Size Overheads 8Methods of Calculating the Standardized Mean Difference•The standardized mean difference probably has more methods of calculation than any other effect size type.Effect Size Overheads 9The different formulas represent degrees of approximation to the ES value that would be obtained based on the means and standard deviations–direct calculation based on means and standard deviations–algebraically equivalent formulas (t-test)–exact probability value for a t-test–approximations based on continuous data (correlation coefficient)–estimates of the mean difference (adjusted means, regression B weight, gain score means)–estimates of the pooled standard deviation (gain score standard deviation, one-way ANOVA with 3 or more groups, ANCOVA)–approximations based on dichotomous dataGreatGoodPoorEffect Size Overheads 10Methods of Calculating the Standardized Mean DifferencepooledsXXnnnsnsXXES2121222121212)1()1(Direction Calculation MethodEffect Size Overheads 11Methods of Calculating the Standardized Mean Difference2121nnnntESAlgebraically Equivalent Formulas:2121)(nnnnFESindependent t-testtwo-group one-way ANOVAexact p-values from a t-test or F-ratio can be convertedinto t-value and the above formula appliedEffect Size Overheads 12Methods of Calculating the Standardized Mean DifferenceA study may report a grouped frequency distributionfrom which you can calculate means and standard deviations and apply to direct calculation method.Effect Size Overheads 13Methods of Calculating the Standardized Mean Difference212rrESClose Approximation Based on Continuous Data --Point-Biserial Correlation. For example, the correlationbetween treatment/no treatment and outcome measuredon a continuous scale.Effect Size Overheads 14Methods of Calculating the Standardized Mean DifferenceEstimates of the Numerator of ES --The Mean Difference-- difference between gain scores-- difference between covariance adjusted means-- unstandardized regression coefficient for group membershipEffect Size Overheads 15Methods of Calculating the Standardized Mean DifferenceEstimates of the Denominator of ES --Pooled Standard Deviation1 nsespooledstandard error of the meanEffect Size Overheads 16Methods of Calculating the Standardized Mean DifferenceEstimates of the Denominator of ES --Pooled Standard DeviationFMSsbetweenpooled1)(22knnXnXMSjjjjjbetweenone-way ANOVA >2 groupsEffect Size Overheads 17Methods of Calculating the Standardized Mean DifferenceEstimates of the Denominator of ES --Pooled Standard Deviation)1(2 rssgainpooledstandard deviation of gainscores, where r is the correlationbetween pretest and posttestscoresEffect Size Overheads 18Methods of Calculating the Standardized Mean DifferenceEstimates of the Denominator of ES --Pooled Standard Deviation2112errorerrorerrorpooleddfdfrMSsANCOVA, where r is thecorrelation between thecovariate and the DVEffect Size Overheads 19Methods of Calculating the Standardized Mean DifferenceEstimates of the Denominator of ES --Pooled Standard DeviationWABBWABBpooleddfdfdfSSSSSSsA two-way factorial ANOVAwhere B is the irrelevant factorand AB is the interactionbetween the irrelevant factorand group