Experimental Validity Brunswik, Campbell, Cronbach, and Enduring Issues Linda Albright Department of Psychology Westfield State College Thomas E. Malloy Department of Psychology Rhode Island College ABSTRACT Donald Campbell and Lee Cronbach had a long history of mutual respect for and fundamental disagreement with each other's ideas about experimental validity. Issues that Campbell labeled as external validity, Cronbach labeled internal validity. Issues that Campbell labeled internal validity, Cronbach suggested are trivial. Nevertheless, these methodological pioneers share much common ground, in part because of their alliance with Egon Brunswik. As science moved from a deterministic to a probabilistic paradigm, all 3 endeavored to protect behavioral science from validity-threatening practices that could result from naive use of the Fisherian approach to scientific investigation. This review shows that issues concerning the prioritization of types of validity still need to be resolved and that most social scientists do not understand internal validity. Several empirical practices for enhancing validity are suggested. Behavioral scientists believe that establishing a causal relationship requires designing studies that can withstand the argument that the observed effect was caused by something other than the causal factor under consideration. This strategy is commonly called maximizing internal validity (an erroneous label from Donald Campbell's perspective, as discussed subsequently). Internal validity is achieved through random assignment of units to experimental conditions and controlled variation of treatment. In the ideal case, these procedures create a methodological inoculation against the list of potential threats to the legitimacy of conclusions that can be drawn from the observations of the study. In other words, investigators who use the experimental method can make public claims regarding the causal effect of the stimulus. These widely shared beliefs reflect the seminal ideas of Donald T. Campbell. In 1957 , Campbell introduced the distinction between internal validity and external validity, being motivated by a "lopsided and complacent emphasis" on Fisherian analysis of variance that reflected the belief that random assignment to treatment was the only methodological control necessary to ensure experimental validity. Accordingly, threats to external validity were defined as threats to validity not controlled by random assignment, whereas threats to internal validity were controlled by random assignment to conditions. Later, Campbell revised and re-revised this distinction and added two other forms of experimental validity. What did remain constant, however, was his continued and untempered emphasis on the primacy of internal validity over all other validity concerns. In opposition to this approach is a scheme developed by Lee J. Cronbach. Emanating from generalizability theory, Cronbach's approach emphasizes the validity with which claims made from the local level of particular experimental realizations can be generalized or applied to the larger domain Review of General Psychology © 2000 by the Educational Publishing Foundation December 2000 Vol. 4, No. 4, 337-353 For personal use only--not for distribution. Page 1 of 209/27/2001http://spider.apa.org/ftdocs/gpr/2000/december/gpr44337.htmlrepresented by the peculiarities of a single study. This issue is labeled internal validity in Cronbach's system, but it is a combination of external and construct validity in Campbell's system. Ironically, although they disagree fundamentally about the most important validity issue, they both label the primary concern as internal validity. Differences in nomenclature notwithstanding, the issue labeled as internal validity by Campbell is seen as trivial by Cronbach, as discussed subsequently. Although Campbell and Cronbach could not seem to be more at odds, in fact, their general missions in the field were compatible, in that their positions were allied with and influenced by Egon Brunswik. Determinism, Probabilism, and the Concept of Validity Early-20th-century psychology witnessed philosophical turmoil that questioned some of its basic assumptions ( Kruger, Gigerenzer, & Morgan, 1987 ). Developments in natural science, especially Heisenberg's (1930) discovery that position and velocity (the first derivative of position) cannot be solved simultaneously in a differential equation, cast doubt on the likelihood of a deterministic universe that could be known with certainty. Even advocates of a knowable, deterministic world (e.g., Einstein, 1961 ) acknowledged that observation is a function of relative position to the object of observation. Within this context, probability theory became the basis for classical psychometrics and statistical inference in psychology. The classical model simultaneously posits a deterministic true score and a probabilistic error component, and reliability is the ratio of true variance to observed variance. Analysis of variance, the basic tool of experimental psychology, cast uncertainty on the denominator of the test statistic (error variance) and a causally deterministic effect in the numerator. The issue of individual differences (a special form of uncertainty) rested on its own mathematics, the correlation coefficient, in which inherent variability constrained covariability. These primary mathematical devices all index a signal to noise ratio, or the ratio of deterministic to probabilistic processes. Although psychology developed into two disciplines ( Cronbach, 1975 ), each assumed probabilism as the basic model for science. Within this ferment of early-20th-century science, Egon Brunswik developed probabilistic functionalism in an attempt to integrate the concepts of uncertainty and Darwinian adaptation ( Gigerenzer, 1987 ; Hammond, 1990 ). His theoretical and methodological approach influenced Donald T. Campbell, who was equally affected by the postmodern movement within science. Both Brunswik and Campbell established models of knowing that explained how uncertain, fallible stimulus cues could be used to make adaptive judgments and responses in the world. Methodologically, both developed approaches for psychology that fully recognized uncertainty of measurement and estimation while maintaining quantified empiricism as a viable epistemological—methodological approach. Mid-20th-century refinements of mathematical theory ushered in the
View Full Document
Unlocking...