Research Methods Notes 02 25 2013 Intro to experimentation and internal validity The Ideal Experiment o Create a situation in which two groups are perfectly equal at baseline o Then introduce a single treatment a single change a single manipulation to one of the groups and take a measurement o If we can do this we have high internal validity Refers to how confidently one can conclude that the observed effects were produced solely by the independent variable and not extraneous ones Why might we have poor internal validity o There might be random error affecting our DV Scores Aspects of the testing environment that affect both groups equally and that create noise in our data Ex a flickerling light an offensive shirt worn by the experiementer the temperature of the room clutter experimenter mood ad infinitum May cause us to miss detecting an effect but is less of a problem if both groups are equally subjected Accounted for by running more controlled experiments and statistically with null hypothesis testing o There might be a systematic error affecting our DV scores Groups differ on a dimension that makes them unequal at baseline that is unrelated to the IV and that may influence DV scores Ex having two conditions always run in two separate rooms or by two separate experimenters or at two times of day Very problematic Must critically think how to avoid such confounds Threats to Internal Validity o I m studying the effect of electric shock on memory For the people in the front of the room they take a memory as normal For the people in the back of the room they get an electric shock under their seat immediately before the test The people in the back perform better on the memory test o Conclusion Electric shocks increases in memory Internal Validity Threat Selection o Our two groups may be predetermined by a characteristic other than our IV ex personalities of those who sit up front vs sit in back or rushing to class heart rate o Need random assignment Threats to Internal Validity 2 o I m testing the effect of puzzle solving under different lighting I have all participants complete a set of puzzles under bright then medium then dark light They solve the most puzzles under dark light o Conclusion Dark light causes better puzzle solving ability Internal validity threat Maturation o Sometimes participants change over time and it has nothing to do with your manipulation o It can be due to short term effects like boredom fatigue or practice maturation Ex someone does four 30 minute visual acuity tests under 4 different conditions they ll get tired or better by the end o In this case we can either counterbalance our testing conditions 3 2 1 then 1 2 3 then 2 3 1 etc o Or we can run a between groups study see next slide Protecting Against Maturation Effects Between Group Designs vs Within Group Designs o Between group designs All participants experience only one of the experimental conditions Ex 2 groups two separate conditions Placebo group vs Drug group o Within group designs aka Repeated Measures All participants experience all experimental conditions Ex everyone measured under placebo drug and high drug dosages We need to counterbalance to minimize maturation effects However much less prone to random error because participants act as their own baseline control Threat to Internal Validity 3 Research shows that adolescents need more sleep than both young children and adults A school board in Minnesota used this info and started their school day an hour later to allow for more sleep The students that year scored on average 100 points higher on the SAT s compared to prior years Conclusion later school start times cause increases in SAT scores Internal Validity Threat History with your manipulation Sometimes participants change over time and it has nothing to do This can be due to historical circumstances that systematically affect only one group history threats o Ex A new SAT prep book may have been published that year o Ex Testing all control participants before 9 11 and all treatment groups afterward A control group is needed in this study for comparison preferably one that experiences the same shared history Threats to Internal Validity 4 I m studying marital satisfaction at different time points in people s marriage for those who had arranged marriages and those who had love marriages I take measures at 3 points Honeymoon phase 2 years out 5 years out I started with 200 couples in each condition and my final testing session is 100 couples in each condition Their marital satisfaction scores at the 3 time points don t significantly differ for either group Conclusion Marital satisfaction does not change over time for arranged or love marriages Internal Validity Threat Attrition Losing subjects over time in a way that may be systematically related to the IV or DV Most problematic in longitudinal research though can happen in any within groups design where participants must return later Prevention careful planning getting permission to search public databases for contact info statistical techniques called imputation Threats to Internal Validity 5 I m studying the effect of threat on racial prejudice I have a group of 30 participants watch either a fearful movie scene or a happy movie scene both in mass testing sessions They then fill out a measure of prejudice The fearful session is run by one experimenter the happy session by a different experimenter They are more prejudiced after the fearful move Conclusion fear causes racial prejudice Threat to internal validity Instrumentation or Experimenter Effects If the testing apparatus changes over the course of the experiment it reduces error unrelated to your DV The experimenter effects threat is why the gold standard of experiments is a randomized double blind placebo control experiment Double blind means the experimenter doesn t even know what condition the participant is in Prevention scripted protocols awareness of issue track who ran who variables Threats to Internal Validity 5 Poorly Operationalized IV s Researchers always need to make trade offs between practicality and experimental realism when operationalizing independent How would you operationalize each of the following IV s o Fear o Injustice o Paranoia o Love o Low self esteem o Jealousy o Low social status Design a study to Test one of the following hypotheses Social rejection causes increased aggression Sexual jealousy causes increased possessiveness The Logic of Null Hypothesis
View Full Document
Unlocking...