POS3713 STUDY GUIDE EXAM 2 Lecture Notes Chapter 4 Research Design What is Research Design Research Design The procedures we use to draw inferences test hypothesis What data are relevant What comparisons do we need to make A good Research Design ensures that the data we examine will allow us to draw inferences and answer our research question Purpose of Research Design Purpose of Research Design to ensure that the inferences we draw are valid To achieve this goal good designs 1 Account for threats to valid inference 2 Identify an appropriate unite of observation 3 Identify an appropriate temporal spacial domain Comparison Comparisons are the cornerstones of Research Design Compare values of y across space e g states individuals counties etc Compare values of y across time e g months years decades etc The design identifies the comparisons we will make Casual Process Must be the SAME across Units We must compare 2 or more units to assess if we may infer that x causes y Does rainfall x influence turnout y Is the turnout greater in a precinct that received NO rain compared to a precinct that received rain Answer the How Else question Were those two precincts different in ways besides rainfall that might be important How Do We Compare Maximize Comparability the units should be identical EXCEPT for the individual variable of interest Random Assignment often difficult or impossible Statistical Control observe world and collect data control for other variables statistically Matching focus on cases that have similar values of x Internal Validity Defined Does the change in our I V really cause the change we observe in the values of our D V Threats to Internal Validity History the values in the past having some impact that we re not accounting for Maturation some process occurring over time and things are going to change because of that occurring process Testing there can be mistakes in observation or test effect people are aware that they are being tested or that testing is going on so they change their behavior Regression to the Mean things tend to go back to their average think about the professor s weight example Selection our sample is a non random sample from our population How do we Improve Internal Validity That is how can we isolate the causal effect Experimental Designs seek to ensure that only X changes Manipulate X while holding all other possible relevant variables constant The goal is to rule out all rival explanations for change in Y except the change in X Standard Experimental Design Randomly divide subjects into two groups A Treatment group and a Control Group Do NOT present Control group with stimulus i e they get a different value of X Measure Y in each group afterwards Any difference in Y across the two groups was caused by the treatment Why Because the RANDOM assignment to the groups makes it very very likely that the groups are on average the same in every way except the stimulus Key Elements of Experimental Design Random Selection Each case has the same chance of being in the experiment Random Assignment Each case has the same chance of being assigned to control or treatment Used together Random Selection and Random Assignment ensures that the groups are equivalent in all ways except the value of X The Beauty of Randomization Randomization makes the two groups identical on average in all ways except for the treatment Eliminates the threat of a spurious relationship Controls for observables Things we could have measured but did not Controls for un observables Differences between units we cannot measure Challenges that Limit the Use of Experimental Design The value of many independent variables cannot be randomly assigned e g gender religion war Differences between lab and actual world External Validity Convenience Samples i e the samples we can get Additional Experimental Designs Field Experiment Randomly assign individuals into groups but perform the manipulation in the real world Natural Experiment an event outside the social scientist s control separates people into control and experiment groups Non Experimental Designs Must Cross the 4 Hurdles When we cannot conduct experiments we collect data as they occur and study them but the logic of inference is precisely the same There are however different challenges to valid causal inference What is an Observational Study Take world as it is and study naturally occurring differences between units Two types of observational studies 1 Cross Sectional Many units sampled over a single period of time 2 Time Series Single unit samples over many time periods Control Variables In experiments we control through random assignment and selection and by holding specific X values constant In non experimental studies we cannot do this Instead we measure the X s we want to hold constant Statistics permit us to estimate the impact of a given X upon Y as if the other X s had been held constant Problem we can only control for other X s that we measure and include in our study Research Design Summary Theory Drives Design Good Research Design Helps establish validity of causal inferences Considers other factors that may be moving the Dependent Variable Spuriousness and Controlling for other factors to allow comparison across like units Chapter 5 Research Design Measurement Operationalization Must be able to measure values of theoretical concepts of interest DV IV controls in order to test for suspected cause and effect Without good measurement inference is suspect Steps for Measuring Social and Political Phenomena 1 Begin with good theoretical understanding of phenomena of interest 2 Construct good theoretical definition 3 Use that theoretical definition to develop the operational definition Operational definition explains what the concept will look like and how it can be measured in the empirical world Consider how others have measured concept of interest 4 WE WANT valid measures reliable measures and unbiased measures Conceptual Clarity Define the characteristics and boundaries of concept Know your unit of interest e g individuals states etc Know your variation of interest over time Between units BE PRECISE Definition Validity Validity extent to which your instrument measures the construct of interest Does what you re measuring map onto the theoretical construct you intended to measure Assessment is your measure of a construct related to other measures as predicted by theory Face Validity do other people think your argument is accepted by
View Full Document
Unlocking...