New version page

UNC-Chapel Hill BIOS 662 - Unified Power Analysis for t-Tests through Multivariate Hypotheses

Upgrade to remove ads
Upgrade to remove ads
Unformatted text preview:

Manuscript for Chapter 8 (pp. 297-344) of book edited by Edwards LK (1993), Applied Analysis of Variancein the Behavioral Sciences, New York: Marcel Dekker. Translated to pdf format: 4 July 1998.8Unified Power Analysis for t-Tests through Multivariate HypothesesRalph G. O'Brien1Keith E. Muller2Cleveland Clinic Foundation University of North CarolinaCleveland, OH Chapel Hill, NC8.1 INTRODUCTION Determining adequate and efficient sample sizes is often critical in designing worthystudies. Yet too many studies have sample sizes that are too small to ensure enoughstatistical power to confirm meaningful effects. Freiman, Chalmers, Smith, and Kuebler(1979) concluded this about clinical trials in medicine. Sedlmeier and Gigerenzer (1989)reached a similar judgment about studies in psychology. The message in both articles iscogent to all fields that rely on statistical inference. Perhaps such articles are havingpositive effects, for we see signs that researchers are now paying more attention to power.For example, reviewers of research proposals now often require that sound power analysesbe done before they will recommend funding or access to facilities and subject populations.Going through the process of determining and justifying the sample size also has animportant ancillary effect: it catalyzes the synergism between science and statistics at thestudy's conception. The statistician who performs a thorough power analysis is more likelyto scrutinize the proposed design, assess issues regarding data management, and develop asound plan for the data analysis. Such involvement can improve the proposal in a numberof ways, thus increasing its chance for approval, funding, scientific success, andpublication.In this chapter, we present a strategy for performing power analyses that is applicableto the broad range of methods subsumed by the classical normal-theory univariate ormultivariate general linear models. First, we introduce the requisite concepts of statisticalpower using concepts from the familiar t-test to compare two independent group means.Second, we proceed to the comparison of two correlated means (matched-pairs problem)and on to the one-way analysis of variance (ANOVA) with contrasts for a completelyrandomized design. Third, we develop power analysis for the univariate general linearmodel, thus providing a broad range of applications. We illustrate this with an analysis ofcovariance (ANCOVA) problem that has unequal distributions of the covariate's values1Supported in part by grants from the US National Institutes of Health (GCRC: RR00082) and theUniversity of Florida Division of Sponsored Research, which funded Zhanying Bai and Yonghwan Umin their wriitng of one portion of the freeware module. Dan Bowling helped in manyways. E-mail: [email protected] 2Supported in part by grants from the US National Institutes of Health (NICHD: P30-HD03110-22,NCI: P01 CA47982-04, GCRC: RR00046). E-mail: [email protected] Unified Power Analysisamong the groups as well as heterogeneous slopes. Fourth, we broaden the range stillfurther by outlining an approximation for determining power under the multivariate generallinear model. This is illustrated with a repeated-measures problem solved by using themultivariate analysis of variance (MANOVA) approach. Fifth and finally, we outlinepower analysis strategies developed for other types of methods, especially for tests tocompare two independent proportions.With writings on sample-size choice and power analysis for many methods now soplentiful, why do we offer yet another one? Like Kraemer and Thiemann (1987), wepresent an approach that unifies many seemingly diverse methods. We think our methodis intuitive, because we develop the strong parallels between ordinary data analysis andpower analysis. To make the methods easier to use, we distribute modules of statementsto direct the popular SAS® System (1990) to perform and table (or graph) sets of powercomputations. Our ultimate goal is to show how a single approach covers a broad class oftests.Rather than restricting attention to the power of the traditional tests (e.g., overall maineffects and interactions), our methods allow one to easily examine statistical hypothesesthat are more tailored to specific research questions. Departing from most writings onstatistical power, we take unbalanced designs to be the norm rather than the exception.Many effective research designs use unequal sample sizes, as when certain types ofsubjects are easier than others to recruit or when certain treatments are more expensive persubject to apply. Thus, researchers and statisticians must decide how the total sample sizewill be allocated among the different groups of cases, with a balanced allocation being aspecial case.We avoid oversimplifying the concept of effect size, as researchers often do when theyemploy rules of thumb, such as Cohen's (1988, 1992) “small,” medium,” and “large”categorizations. A tiny effect size for one research question and study could be a hugeeffect size in another. Researchers often claim that their studies promise “medium” effectsizes, but they have no objective grounds to justify such a claim. Our scheme forcesresearchers to give specific conjectures or estimates for the relevant statistical parameters,such as the population means and standard deviations for an ANOVA problem. Theconjectures are then used directly to calculate effect sizes, which determine statistical powerfor a proposed sample size. Our detailed examples illustrate how straightforward it is to dothese things. Sample-size analysis is not harder to do than data analysis; as we shall see,the two problems are very similar.The best hypothesis-driven research proposals include quite definite plans for dataanalyses, plans that merge the scientific hypotheses with the research design and the data.A good sample-size analysis must be congruent with a good plan for the data analysis.There are hundreds of other common statistical procedures besides normal-theory linearmodels, and there are thousands of uncommon methods and an unlimited number of“customized” ones that are developed for unique applications. While we can cover powerfor many of the common methods for statistical inference, one chapter cannot beexhaustive of all known or possible sample-size methods.One final general point is in order. Many treatments on statistical power choose to8.2 t-Tests and One-way ANOVA

View Full Document
Download Unified Power Analysis for t-Tests through Multivariate Hypotheses
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...

Join to view Unified Power Analysis for t-Tests through Multivariate Hypotheses and access 3M+ class-specific study document.

We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Unified Power Analysis for t-Tests through Multivariate Hypotheses 2 2 and access 3M+ class-specific study document.


By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?