Chapter 4 – Gathering DataRecall that after we ask a statistical question (step 1: see Chapter 1), we collect our data. How dowe do that? It depends partly on the question we’ve asked and partly on what resources we have available to us.Definition: an experiment is conducted when the researcher assigns subjects to certain conditions and then observes outcomes on the response variable. The experimental conditions, which correspond to assigned values of the explanatory variable, are called treatments.Definition: an observational study (sometimes correlational study) occurs when values for both respons and explanatory variables are observed (i.e., not assigned), without anything being done to the subjects. They can only infer correlation.Example: Smoking and Lung CancerOver the course of decades, doctors and scientists have observed a higher incidence rate (proportion) of lung cancer among people who smoke more frequently than among non-smokers.Is this an experimental or observational study?There are advantages to conducting experiments:- reduction of the impact of lurking variables due to randomization of assignment among treatments- establishment of cause-and-effect relationships via control and manipulation of treatments and randomizationBut we don’t always experiment. Why not?- resources (it costs time and money and people, among other things)- feasibility (if the effect is expected to manifest over a long period, the length of the experiment may rule it out)- ethics (e.g., smoking and lunch cancer)- the statistical question may not be after causality“The plural of anecdote is not data” – Roger Brinner (probably).There is a difference between archival data (formally collected) and data that is just available to you by memory or because of some story you heard (in fact, there are psychological biases aboutavailability of “data”).It is best to rely on data from reputable sources, or to collect it (scientifically) yourself.Definition: a sampling frame is the list of individuals from which the sample will be drawn. Wetry to define our sampling frame well, as our sample will only be as good as our sampling frame.Good Sampling DesignsThe following are random sampling schemes.- Simple Random Sample: under simple random sampling (SRS), each sample of size n has equal probability of being selected. That is, each set (combination) of n individuals has equal chance of being drawn. This is done using random numbers.- Stratified Sampling: when the population is first sliced into homogeneous groups (strata), and then an SRS is drawn from within each stratum. The results are combined at the end.- Cluster Sampling: when the population is split into clusters and the clusters themselves are then randomly sampled; a census is performed within each cluster.- Systematic Sampling: individuals or subjects are selected according to a predefined system or rule (e.g., every 5th person in line or on an alphabetized list), but where the starting point was randomly selected. The random starting point is imperative to assure a random sample.Using random sampling methods, a larger sample size will result in better inference (more statistical power).Biased Sampling Methods (but they still get used)Besides the random sampling methods, which will yield, on average, a representative view of thepopulation, there are other sampling methods that will yield biased samples and should be avoided when possible.Volunteer Sample: individuals are invited to participate in a study or survey, and all who do respond are counted. Thus, people volunteer to participate.With a volunteer sample, the extremes are typically overrepresented and the middle is underrepresented.Convenience Sample: we include the people who are convenient to us (e.g., we have contact information for them, they are nearby)Potential Biases in Sampling:- nonresponse bias, which occurs when nonrespondents all share common characteristics (e.g., a telephone survey misses everyone who doesn’t have a phone; that might be because they are all lower income)- undercoverage, when some parts of the population are not represented (e.g., volunteer samples often only capture the extrem es in a population and miss people who fall in the middle of the spectrum)- response bias, when actual responses given are biased, because of how the questions were asked, because of demand characteristics, or socially desirable responses.NB: a larger sample using biased methods will just result in a larger, biased sample. Increasing the sample size in these cases will not reduce or remove the bias inherent in the process. A Good ExperimentA good experiment will have some familiar elements:- randomization to average out the effects of lurking variables- avoids bias in assignment between treatment groups (random assignment)- achieves balance on variables known to affect the response (randomization)When dealing with human participants (subjects), particularly in medical trials, it is common to use a blind experiment, where the participant does not know what treatment group they are in. This is one way to minimize the placebo effect which is improvement or change in a subject’s response even when they are in the control group (no change in the explanatory variable). Other studies go even farther, using double blind methods, where neither the researcher nor the participant knows which treatment group the participant is in. This minimizes response bias on the participant’s part and experimenter