Statistical Principles of DesignBret LargetDepartment of StatisticsUniversity of Wisconsin - MadisonOctober 27, 2004Statistics 371, Fall 2003Principles of Design• Previous chapters have dealt with methods of analyzing datato make inferences about a single population or to makecomparisons between two populations.• Future chapters will focus on methodology for the analysisof data that arises in additional settings.• The present chapter looks instead at the problem ofdesign,the methods for collecting data.• How should data be collected so that analysis of the dataleads to valid inferences?• What are the pitfalls of poor design choices, and how canthese affect inference?• What are the main statistical principles of design?Statistics 371, Fall 2004 1Observation versus Experiment• We make a distinction between an experiment in which theresearchers intervene in the experimental conditions and anobservational study, in which the researchers merely observean existing situation.• The distinction is importantin the interpretation of theresults of an analysis.• Consider an analysis to compare two groups. In an experi-ment,the researchers assign the groups.• In an observational study,the groups are simply observed.Statistics 371, Fall 2004 2Case Study: Cigarettes and Smoking• In a study, pregnant women were questioned about theirsmoking habits, diet, and other variables. The babies werefollowed up for some time.• There was strong statistical evidence thatthe mean birthweight of smokers’ babies was lower than the mean birthweight of nonsmokers’ babies. Low birth weight is associatedwith a number of health problems in babies, which makes theproblem of finding causes of low birth weight important.• We say that smoking and low birth weight areassociatedwith one another.Statistics 371, Fall 2004 3Case Study: Cigarettes and Smoking• However, this single study alone is insufficient to support theconclusion thatsmoking caused the lower birth weight.• The smokers and nonsmokers differed on a number ofpossibleexplanatory variables, and it is unclear which of thesevariables may have caused the difference.• We say that the possible effects of smoking on birth weightareconfounded with many other possible explanations.• Confounding has the potential to mislead•Association is not causation!Statistics 371, Fall 2004 4Comparison• To attribute a causal relationship between an explanatoryvariableand a response variable (such as smoking and lowbirth weight), we would like to be able to make a comparisonbetween two groups that differ only in the explanatoryvariable under study with all other possible explanatoryvariables the same between the two groups.• While in experimental settings it is possible for the researcherto create groups in which a single explanatory variable isthe largest difference between two groups, there are manysettings for which experiments are either impossible, twoexpensive, or unethical.• It is not impossible to attribute a causal relationship basedon observation studies alone, but it is far more difficultbecause the researchers essentially need to identify and ruleout or control for the effects of the other possible explanatoryvariables.Statistics 371, Fall 2004 5More on smoking and low birth weight• In one study, a large numb er of variables were measured.A complex statistical method that simultaneously estimatesthe effects of several explanatory variables found that evenafter making adjustments for these other variables, smokingstill had an effect on birth weight.• A second study found differences in the placenta betweensmokers and nonsmokers, and that some of the differenceswere associated with chemicals found in cigarettes.• This same study also found that having smokers not smokefor three hours caused a change in blood flow to the placenta.• A third study identified 159 women who smoked during a firstpregnancy but not during a second pregnancy. These womenwere matched with 159 other women who had smoked duringboth pregnancies and for whom other explanatory variableswere similar. This study found that the second babies ofthe women who quit smoking were heavier than the secondbabies of their matched controls who continued to smoke.Statistics 371, Fall 2004 6The First Study• The first study attempts to make comparisons of similargroups bya statistical analysis that attempts to adjustfor the effects of other explanatory variablesand so leavea comparison where the only important difference is theexplanatory variable of interest.• Interpretation of causality from such a study, however,assumes that the statistical model for the joint effects ofall the variables is an accurate description of reality, often adubious assumption.Statistics 371, Fall 2004 7The Second Study• The second study attempted to establish a link betweensmoking and low birth weight by establishing a link betweensmoking and the placenta, where the link between theplacenta and birth weight is perfectly plausible withoutstatistical justification on biological grounds alone.• This study even included an experiment in which the bloodflow to the placenta could be compared within the samewoman after she had smoked and when she had abstainedfrom smoking.Statistics 371, Fall 2004 8The Third Study• The third study attempts to make a comparison between likegroups by constructing groups that are as similar as possibleon the basis of other explanatory variables that are thoughtto also have an effect.Statistics 371, Fall 2004 9Comments on Confounding• Notice that there are several possible ways to addressconfoundingwith the objective to eventually establish acausal relationship through a series of observational studies.• Several different large observational studies are often neces-sary to present a convincing case for establishing causality.Statistics 371, Fall 2004 10Example: A Common Cold• Researchers invited college students to volunteer in an ex-periment to test the effectiveness of a vaccine for preventingthe common cold.• The volunteers were randomly assigned to two groups. Onegroup received the vaccine, the other group took a placebo.• The study wasblinded. The subjects did not know whatgroup they were in.• Both groups reported dramatic decreases in the number ofcolds.mean number of coldsGroup n previous year current yearVaccine 201 5.6 1.7Placebo 203 5.2 1.6Statistics 371, Fall 2004 11Importance of Control Groups• This experiment shows the
View Full Document
Unlocking...