Jacklyn Giampa STATS241 Lecture 7 2 9 16 Variance in Research Design Sources threats to internal validity and Noise Variance the degree to which your numbers differ from one another 5 5 5 5 no variance 4 6 5 5 a little variance with a mean of 5 1 9 6 4 more variance with a mean of five In any piece of research there are three sources of variance 2 Error variance 1 Primary variance totally unexplained due to chance happenings such as moment to the variability in the DV that occurs as a result of the influence of the IV good source of variance intentionally trying to produce moment fluctuations in your subjects attention or fluctuations in your ability to accurately measure your DV due to chance variations inaccuracy of equipment 3 Secondary variance secondary variables as the experimenter you want to maximize your primary variance minimize error variance and control secondary variance you cannot totally eliminate error variance there will always be some error secondary is where the researcher has the best opportunity to improve the chances of obtaining good internal and external validity variance in the DV that occurs as a result of the influence of Problems caused by secondary variance Secondary variance can cause problems in two ways 1 2 If the secondary variable co varies along with the independent variable then the secondary variable will create a threat to internal validity or confound If the secondary variable cause a lot of overall variability in your DV then this may mask or hide any effects of the IV The 2nd variables create Noise in the data that make it harder to detect an effect of your IV Threats to Internal Validity Confounds Five different threats to internal validity Two threats resulting in non equivalent groups occur in independent group or between subject type designs random matched natural groups 1 Example of threat 1 Subjects went to office the effect of viewing a violent stimulus on future aggressive tendencies Office desk had one of two objects on the desk One half of the subjects walked in and there was a tennis racket on the desk Other half of the subjects walked in and on the desk was a gun Then took a test to measure their aggressive tendencies Those who saw the gun would show to be more aggressive than those who saw the tennis racket Went to two different high schools Discovers students exposed to the gun showed more aggressive tendencies than those who saw the tennis racket Why should we question the researcher Because there are so many other variables different high schools different environments cause confounds certainly deadly and cannot draw real conclusions this is called a threat of selection occurs when participants subjects in one level of the IV differ initially from p s in another level of the IV due to systematic selection differences This is usually the result of the use of intact groups or lack of random assignment of subjects to groups It is a between subject or independent group type issue Solution Take one half of students from each schools show one half the tennis racket show one half the gun so that students from both schools see both IV s 2 Example of threat 2 is punishment more effective than reward in learning a skill What method of training soldiers to decipher between enemy and sided planes One week study 10 cents for enemy electric shock when wrong 80 reward group 92 punishment group Why should we question the researcher Because he lost more subjects from the punishment group than the reward group The people getting shocks because they weren t good at the task left more of them versus those who left in the reward group The difference in subjects is a threat Mortality attrition occurs when you lose subjects from one level of IV more than the other level systematically Differential loss of subjects from levels of the IV Mortality death attrition losing subjects It is a between subject or independent group type issue 3 Example of threat 3 effect of remedial math course for entering freshmen on math grades in college level math courses this one is hard have faith Teachers want to see improvements Teachers add remedial math program to help students who score low on a math See that many students are deficient in math skills placement course They take the freshman class 5 lowest scoring students and have them go through a one week long math program Test them at the end of the week to see if they have improved They did show an improvement after the one week remedial program why should we question REGRESSION TOWARDS THE MEAN extreme scores will become less extreme with repeated measurement very low scores will regress upward toward the mean and very high scores will fall down toward the mean luck factors come into play across a fair amount of students causes regression towards the mean They should have taken the lowest 5 and have half of them take the course and half not and have them both take the test in order to compare the effect of the course Repeated measures type design 4 Example of threat 4 the effects of a program on the benefits of alternative energy sources on attitudes toward the use of alternative energy sources College students volunteer for 5 day info sessions about alternative energy Each student filled out questionnaire about their attitude towards alternative energy before and after the program By the end they take the survey and there is a fairly large increase in gas prices History the occurrence of an event other than the treatment that produces changes in the participants behavior The event is not under the researchers control This is a repeated measures problem the longer the time between measurements the more likely it is that such an event will occur and internal validity will be jeopardized 5 Example threat 5 the effects of a pre school story time program on adjustment to a school environment Implementing a story time so that they can help preschoolers learn how to adjust One month story time every day The students showed an adjustment Why should we question this Because they had time to adjust It was not merely the program it was a month later and thus the children also had time to get used to things Could be several factors Fix this by using two groups of children one group with story time one without compare adjustment afterwards Maturation changes occur to your participants between measurements These changes are not related to your IV Practice boredom and fatigue are considered threats as well This is a