1Class 13ExperimentsDefining Experiments• By experiment we refer to that type ofresearch in which variables aremanipulated and their effects uponother variables observed.Internal Validity• Internal validity means that theconclusion drawn from a study is validfor the respondents participating in thestudy.2ExampleDoes taking a class in Race and Ethnicitymake white students' racial attitudes moretolerant towards minorities?Consider a one-group pretest-posttest design:(Y = tolerance)Y2Taking Soc134Y1Time 2Time 1Example• If Y1< Y2, can we be certain that it isthe class that increased students'racial tolerance toward minorities?• No, there are other possibleexplanations for the change. Thesecompeting explanations are calledthreats to internal validity.Threats to Internal Validity:External Events• History: the specific events occurringbetween the first and secondmeasurement in addition to theexperimental variable.3Threats to Internal Validity:Endogenous Change• Testing: the effects of taking a test upon the scoresof a later testing.• Instrumentation: changes in the measurement ofthe dependent variable in pretest and posttest.• Statistical regression: subjects have been selectedon the basis of extreme scores.• Maturation: processes within the respondentsoperating as a function of the passage of time perse (not specific to particular events), including, e.g.,growing older and growing more tired.A Better Design is:• Have we solved all the problems?• No, the two groups may not be “equivalent.”Two-group pretest-posttest designY2Y1Group 2Y2Taking 134Y1Group 1Time 2Time 1Threats to Internal Validity:Selection Bias• Selection biases: differential selectionof respondents into the comparisongroups.• Experimental mortality: differentialattrition rates of the comparisongroups.4Threats to Internal Validity:Contamination• Contamination: either theexperimental group or the controlgroup is aware of each other andinfluenced in the posttest as a result.Threats to Internal Validity:Treatment Misspecification• Placebo effect: Subjects’ belief in theeffectiveness of the treatment affectstheir posttest results.• Hawthorne effect: Participation in thestudy makes subjects feel special,which affects their posttest results.• Expectation of the researcher:Positive expectations of research staffmay affect subjects.– Solution: double-blind proceduresExperimental Designs• Proper experimental designs can helpsolve the above problems andestablish internal validity.• Two-group pretest-posttest design +randomization: the best designRandomizationY2Y1ControlgroupY2Taking 134Y1ExperimentalgroupTime 2Time 15Experimental and ControlGroups• Must be as similar as possible.• The purpose is to exclude alternativeexplanations for the observed treatmenteffect.• Use random assignment or matchingmethod.• Control group represents what theexperimental group would have been like,had it not been exposed to the experimentalstimulus—i.e., the counterfactual.Random Sampling andRandomizationRandom samplingRandomizationPopulationStudy subjectsSampleExperimental groupControl group• Random sampling ensures generalizability of survey results.• Randomization in experiments ensures comparability of theexperimental group and the control group when the pool of studysubjects is large.MatchingRandomizationExperimental groupControl group• First match respondents on relevant variables. Then randomizematched pairs to the experimental group and the control group.• Matching ensures comparability of the experimental group and thecontrol group with respect to matched variables.MatchingMatched pairsStudy subjects6Randomization and Matching• Randomization works well if you havea large pool of subjects.• Matching works well if the pool ofsubjects is small and you know whichvariables are relevant.TreatmentTreatmentPosttest-Only DesignRandom Assignment Treatment PosttestXTaking Soc134 ToleranceY1Y2RR: Random AssignmentO: ObservationX: TreatmentExperimental groupControl groupTest scorePosttestTreatment effectIf group assignment is random and the pool of subjects relativelylarge, pretests results should be similar for exp. group and controlgroup. Therefore, we can have a posttest-only design.Test scorePretest PosttestGrowth of experimental groupGrowth of control groupCausality in ExperimentsCriteria for causal relationships• Association between the dependent andindependent variables– Compare experiment and control group on thedependent variable.• Time order: cause occurs before effect– Treatment should be given before posttest.• Non-spuriousness– Randomization guarantees comparability ofexperiment and control groups on all potentialconfounding variables.7External Validity• External Validity – Are experimentalresults generalizeable to the real world?• Threats to External Validity —– Generalizability of the sample– Generalizability of the treatment and setting– Interaction of testing and treatment: Pretest maysensitize subjects to the treatment.Strength and Weakness ofExperimental MethodStrength:• Isolation of the experimental variablethrough randomization.Weakness:• Artificiality of laboratory setting: Socialprocesses that occur in a lab mightnot occur in a more natural socialsetting.Quasi-Experiment(Natural Experiment)• Treatment is natural, notmanipulated.• In quasi-experiment, the comparisongroup is predetermined to becomparable to the experimentalgroup in critical ways.• Before-and-after design: All casesare exposed to
View Full Document
Unlocking...