Research DesignGeneral CommentsThe Road MapDifferent Ways of Doing Empirical ResearchMajor Components of Research DesignsResearch QuestionTheoryDataCausalityDefinitions of CausalityThe Biggest Problem in Causal ResearchHow to Establish CausalityDesign typesOne-shot Case StudyOne-group Pre-test/Post-testStatic group comparisonPre-test/Post-test Control GroupSolomon Four-Group DesignPost-test only experimentWhere do standard political science studies fall among the Stanley/Campbell designs?Social Science vs. Natural Science and EngineeringResearch Design17.871Spring 2003General Comments•The road map of political science•Different ways of doing political science research•Major components of research designs•Designing research to ferret out causal relationships•Social science vs. natural science/engineeringThe Road MapPhilosophyNormativePositiveCausalCorrelationalDescriptiveTheoreticalEmpiricalDifferent Ways of Doing Empirical Research•Interpretive•Small-n case study–Haphazard–Structured•Large-n statistical analysis•Interactions among these waysMajor Components of Research Designs•Research question•Theory•DataResearch Question•Important–Not too general–Not too specific–Just right•Contribute to literature–How to tell: Social Sciences Citation Index–http://libraries.mit.edu/get/webofsci–E.g.: effect of redistricting on congressional election results•Search for Cox & Katz, “The Reapportionment Revolution and Bias in U.S. Congressional Elections,” AJPS 1999Theory•Definition: A general statement of a proposition that argues why events occur as they do and/or predicts future outcomes as a function of prior conditions•General/concrete trade-off•Desirable qualities of theories–Falsification (Karl Popper)–Parsimony (Occam’s razor)Data•Terms–Cases–Observations–Variables•Dependent variables•Independent variables–Units of analysis•Mapping between the abstract and concrete (we’ll come back to this)–Measures–IndicatorsCausality•Definition of causality•Problems in causal research•Side trip to Campbell and StanleyDefinitions of Causality•Logical–A causes B if the “presence” of A is a sufficient condition for B.•Experiential–A causes B if B occurs following the “exogenous” introduction of A–When does exogeneity occur?•Positive example: “ethnic” names on resumes•Negative example: campaign spendingThe Biggest Problem in Causal Research•Establishing the exogeneity of “causes”How to Establish Causality•Donald Campbell and Julian Stanley, Experimental and Quasi-Experimental Designs for Research (1963)Design types•One-shot case study•One-group pre-test/post-test•Static group comparison•Pre-test/post-test with control group•Solomon four-group design•Post-test only experiment[Running example: racial discrimination in resumes]One-shot Case Study•Summary:X OorO X•Journalism•Common sense•“of no scientific value”One-group Pre-test/Post-test•Summary:O X O•Better than nothing•Standard way of doing most research•Big problems–No comparison group–No random assignment•Encourages “samples of convenience”Static group comparison•Summary:X O1----------- O2•This is most cross-sectional & correlational analysis•Problems–Selection into the two groups–No pre-“treatment” measurementPre-test/Post-test Control Group•Summary:R O1T X O2T --------------------------------R O1C O2C•Effect of treatment:[O2T – O1T] – [O2C – O1C]•This is the classic randomized experiment•Problem: “Hawthorne effect”Solomon Four-Group Design•Summary:R O X OR O OR X OR O•Allows you to control for the effect of the experiment itselfPost-test only experiment•Summary:R X OR O•No prior observation (assume O1T = O1C)•Classical scientific and agricultural experimentalismWhere do standard political science studies fall among the Stanley/Campbell designs?•One-shot case study–Little scientific value, but may be descriptively useful•One-group pre-test/post-test–Often used in policy analysis–Only justified as a “best design” if there are ethical or other constraints•Static group comparison–Correlational studies by far the most common “scientific” social science research•Pre-test/post-test with control group–“Real” experiments uncommon, but growing in frequency–“Quasi-experiments” growing more rapidly•Solomon four-group design–Don’t recall ever seeing this•Post-test only experiment–Leads to weaker statistical testsSocial Science vs. Natural Science and Engineering•Reductionism–Degree of reductionism•Implications –Measures of association weak–Aggregates often better predictors•Why we have
View Full Document
Unlocking...