Slide 1Logic of ExperimentsWithin-Subjects Design and the Power of Randomization_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 2Between & Within Subjects• Within-subjects designs. Each subject gets all treatments. Powerful design IF– Feasible– Treatments are reversible– No interference between treatments• Between-subjects designs. Each subject gets only one treatment.• Our projects will be between-subjects._____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 3Within Subject Example:Discrimination Studies• Mary is black. Applies for a job at a shop on State Street. Is not given an application, told they are not hiring. Is this evidence of discrimination?• What if Jane who is white goes there half an hour later and is given an application?• Logic of audit studies• Necessity of control, concerns about reactivity_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 4Example of Between Subjects Problem• Imagine course like statistics, chemistry• Want to see if tutoring program helps students do better• Independent variable = whether tutored (yes, no)• Dependent variable = grade in class_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 5Tutoring Program• Survey students, ask if they had a tutor• This is final grade or score• What do we conclude? Does tutoring help?• Why is this happening?753.0Not Tutored602.5TutoredScoreGrade_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 6Change Over Time2.31.7Grade6050ScoreFinalMidtermChange scores for those tutored. Compare score on first exam and second.Did tutoring make a difference?_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 7Comparison Group• Only those who got below a C (2.0) on the first test• Before/after comparison• Is this a valid test?1.51.5Not Tutored2.61.5TutoredTest 2Test 1_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 8Another Comparison• One semester, no tutor. Random sample of 20. Average grade 2.6.• Next semester, tutor. Random sample of 20. Average grade of 2.9.• How about this test?_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 9Matching• Suppose you match by age, height, sex, eye color, past GPA, ACT math score• Will that solve the problem?• Problem of inherent motivation• No amount of matching can solve the problem of selection bias_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 10Problem of Selection Bias• Who goes to a tutor?• Non-equivalent control groups• Inherent motivation, performance (dependent variable) correlated with factors related to independent variable_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 11Randomization• Take a group of subjects, selected any way you want• Actual random assignment to groups (no cheating)• Equated in the statistical long run if sample is large enough. (Ideally 30 per group or more)_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 12The “Magic” of Randomization• All subject characteristics are statistically equated automatically, whether or not you can list them• Selection bias automatically controlled• Depending on design, other factors may be controlled too. _____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 13Manipulated Variables• MANIPULABLE independent variable = experimenter can control which category of independent variable subject can fall into• CANNOT do experiments with subject characteristics as independent variables• Important to understand this distinction, understand what a manipulable independent variable is.• Experimenter decides who is in which treatment group_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 14Value of Experiments• If you can manipulate an independent variable, a true experiment is always the best method• Best = able to isolate effect of independent variable as only cause of dependent variable• Use ability to manipulate to
View Full Document
Unlocking...