Intro to experimentation and internal validityThe “Ideal” ExperimentCreate a situation in which two groups are perfectly equal at baseline…Then introduce a single treatment, a single change, a single manipulation to one of the groups and take a measurementIf we can do this, we have high internal validity:Refers to how confidently one can conclude that the observed effects were produced solely by the independent variable and not extraneous onesWhy might we have poor internal validity?There might be random error affecting our DV ScoresAspects of the testing environment that affect both groups equally and that create “noise” in our dataEx) a flickerling light, an offensive shirt worn by the experiementer, the temperature of the room, clutter, experimenter mood, ad infinitum…May cause us to “miss” detecting an effect, but is less of a problem if both groups are equally subjectedAccounted for by running more controlled experiments and statistically with null hypothesis testingThere might be a systematic error affecting our DV scoresGroups differ on a dimension that makes them unequal at baseline, that is unrelated to the IV and that may influence DV scoresEx) having two conditions always run in two separate rooms or by two separate experimenters, or at two times of dayVery problematic! Must critically think how to avoid such confoundsThreats to Internal ValidityI’m studying the effect of electric shock on memory. For the people in the front of the room, they take a memory as normal. For the people in the back of the room, they get an electric shock under their seat immediately before the test. The people in the back perform better on the memory test.Conclusion: Electric shocks increases in memoryInternal Validity Threat: SelectionOur two groups may be predetermined by a characteristic other than our IV (ex: personalities of those who sit up front vs. sit in back; or rushing to class & heart rate).Need random assignmentThreats to Internal Validity 2I’m testing the effect of puzzle solving under different lighting. I have all participants complete a set of puzzles under bright, then medium, then dark light. They solve the most puzzles under dark light.Conclusion: Dark light causes better puzzle-solving abilityInternal validity threat: MaturationSometimes participants change over time, and it has nothing to do with your manipulationIt can be due to short-term effects like boredom, fatigue, or practice (maturation)Ex) someone does four 30-minute visual acuity tests under 4 different conditions (they’ll get tired [or better] by the end)In this case, we can either counterbalance our testing conditions…(3, 2, 1; then 1, 2, 3; then 2; 3, 1, etc…)Or we can run a between-groups study (see next slide)Protecting Against Maturation EffectsBetween Group Designs vs. Within Group DesignsBetween group designs:All participants experience only one of the experimental conditionsEx) 2 groups, two separate conditions. Placebo group vs. Drug groupWithin group designs (aka Repeated Measures)All participants experience all experimental conditionsEx) everyone measured under placebo, drug and high drug dosagesWe need to counterbalance to minimize maturation effectsHowever, much less prone to “random error” because participants act as their own baseline controlThreat to Internal Validity 3Research shows that adolescents need more sleep than both young children and adults. A school board in Minnesota used this info and started their school day an hour later to allow for more sleep. The students that year scored, on average, 100 points higher on the SAT’s compared to prior yearsConclusion: later school start times cause increases in SAT scoresInternal Validity Threat: HistorySometimes participants change over time, and it has nothing to do with your manipulationThis can be due to historical circumstances that systematically affect only one group (history threats)Ex) A new SAT prep book may have been published that yearEx) Testing all control participants before 9/11 and all treatment groups afterwardA control group is needed in this study for comparison, preferably one that experiences the same shared “history”Threats to Internal Validity 4I’m studying marital satisfaction at different time points in people’s marriage for those who had “arranged marriages” and those who had “love marriages”. I take measures at 3 points: Honeymoon phase, 2 years out, 5 years out. I started with 200 couples in each condition, and my final testing session is 100 couples in each condition. Their marital satisfaction scores at the 3 time points don’t significantly differ, for either group.Conclusion: Marital satisfaction does not change over time for arranged or love marriagesInternal Validity Threat: AttritionLosing subjects over time in a way that may be systematically related to the IV or DVMost problematic in longitudinal research, though can happen in any within-groups design where participants must return laterPrevention = careful planning, getting permission to search public databases for contact info, statistical techniques called “imputation”Threats to Internal Validity 5I’m studying the effect of threat on racial prejudice. I have a group of 30 participants watch either a fearful movie scene or a happy movie scene, both in mass testing sessions. They then fill out a measure of prejudice. The fearful session is run by one experimenter, the happy session by a different experimenter. They are more prejudiced after the fearful move.Conclusion: fear causes racial prejudiceThreat to internal validity: Instrumentation or Experimenter EffectsIf the testing “apparatus” changes over the course of the experiment, it reduces error unrelated to your DVThe experimenter effects threat is why the gold standard of experiments is a “randomized, double-blind, placebo control experiment.”“Double-blind” means the experimenter doesn’t even know what condition the participant is inPrevention = scripted protocols, awareness of issue, track who ran whoThreats to Internal Validity 5: Poorly Operationalized IV’sResearchers always need to make trade-offs between practicality and experimental realism when operationalizing independent variablesHow would you operationalize each of the following IV’s:FearInjusticeParanoiaLoveLow self-esteemJealousyLow social statusDesign a study to Test one of the following hypotheses:Social rejection causes increased aggressionSexual jealousy
View Full Document
Unlocking...