POS3713 Understanding Political Science Research Prof John B Ryan Midterm 2 February 21 2012 Study Guide Internal Validity o Can you demonstrate that x causes y o about causation o you can rule out other possibilities and isolate the causal effect o does our independent variable really cause the change we see in our dependent How can we ensure internal validity o internal validity is the goal of true experimental designs isolating the true causal effect o we manipulate x while holding other possible cause constant and observe changes in y Criteria the goal is to eliminate all alternative explanations o the cause precedes the effect in time temporal precedence o the cause and the effect are related covariation and o there are no plausible alternative explanations for the observed covariation nonspuriousness External Validity o about the results from your study o do they apply to other areas o are the results from your study generalizable o ex study from Florida and applying it to another state o comes up frequently in discussions about laboratory experiments o the subject pool is usually college students o if student pool is significantly different from general population it is not externally valid Good Research Designs Account for The Unit of Interest Temporal spatial variation of interest Cross Sectional o many cases at one point in time Time Series o one case at many points in time Threats to Causality things that will screw up research validity good research design will secure internal validity History o threat to validity o ex bombing of USS Cole had an effect on opinion 3 weeks later presidential election o Events outside of the study experiment or between repeated measures of the dependent variable may affect participants responses to experimental procedures o Often large scale events o Affect participants attitudes and behaviors o Becomes impossible to determine whether any change on the dependent measures is due to the independent variable or the historical event Maturation o something over time changing the effect o or something changing over time o ex survey what do you think about the president o feelings linked to partisanship o statements that vary more o changes of opinion because of length of being on the phone and o Subjects change during the course of the experiment or even between getting annoyed measurements o For example young children might mature and their ability to concentrate may change as they grow up o Both permanent changes such as physical growth and temporary ones like fatigue provide natural alternative explanations thus they may change the way a subject would react to the independent variable o So upon completion of the study the researcher may not be able to determine if the cause of the discrepancy is due to time or the independent variable Testing o ex testing without notice and testing with notice of the test o Behavioral change study o Repeatedly measuring the participants may lead to bias o Participants may remember the correct answers or may be conditioned to know that they are being tested o Repeatedly taking the same or similar intelligence tests usually leads to score gains but instead of concluding that the underlying skills have changed for good this threat to Internal Validity provides good rival hypotheses Selection o This occurs when the subject related variables color of hair skin color etc and the time related variables age physical size etc interact o If a discrepancy between the two groups occurs between the testing the discrepancy may be due to the age differences in the age categories Regression to the Mean o the idea that there is a stable level that you should be at and sometimes randomly you ll be above that level and sometimes you ll be below that level o ex basketball player playing really good and then playing at his normal level the next game o This type of error occurs when subjects are selected on the basis of extreme scores one far away from the mean during a test o For example when children with the worst reading scores are selected to participate in a reading course improvements at the end of the course might be due to regression toward the mean and not the course s effectiveness o If the children had been tested again before the course started they would likely have obtained better scores anyway o Likewise extreme outliers on individual scores are more likely to be captured in one instance of testing but will likely evolve into a more normal distribution with repeated testing Experimental Designs o strongest in internal validity o researcher randomly assigns subjects to receive a treatment anything else is not an experiment o randomization ensures internal validity o because a group that received a particular variable should be the same in every way except for the assignment of that independent variable o since variable was assigned randomly particular characteristics are correlated to the o random assignment controls for things you can and cannot measure treatment Field experiments world Natural experiments o randomly assign individuals into groups but perform the manipulation in the real o goes out into the real world and applies the treatment out there o highest experimental design with external validity o an outside event separates people into control and treatment groups o not really an experiment o random event sorts people into the control and treatment group o ex conducting a survey in a course of a month during month something big happens difference between people at the beginning of a survey and those taking the survey after the event Statistical Control o A statistical technique used to eliminate variance in dependent variables caused by extraneous sources o In evaluation research statistical controls are often used to control for possible variation due to selection bias by adjusting data for program and control group on relevant characteristics o get measures from all other statistical alternative and include them in your statistical model o downside if you cannot get a measure from these control variables if you can t find them you don t include them if you can t think of them you don t include them Reliability o consistency in measurement o same results time after time after time o ex scale giving you the same weight Test Retest same measure two different points in time problem things may change over time o ex weighing yourself at two different times Alternative Form two different measure at two
View Full Document
Unlocking...