Course Description Design provides the conceptual framework, using structural elements, from which a study is planned and executed. It also sets the basic conditions from which facts and conclusions are inferred. As such, design warrants special treatment given that even the most sophisticated and elegant statistical procedures can rarely, if ever, correct for poor design. Design is one of three discreet, yet interrelated parts of what social scientists often refer to as method or methodology, and is perhaps the most important. With an emphasis on causal inference and various types of validity, the course consists of systematically studying the theoretical, philosophical, and ideological foundations of and principles for designing experimental, quasi‐experimental, and, to a lesser extent, nonexperimental investigations for applied research and evaluation. Design of quasi‐experimental studies that either lack a comparison group or pretest observation and quasi‐experimental studies that use both control groups and pretests, including interrupted time‐series and regression discontinuity designs, as well as randomized experimental designs, including the conditions conducive to doing them and more practical matters such as ethical considerations, attrition, and random assignment, are the primary foci of the course. Students also will be introduced to design sensitivity/statistical power for individual‐level and group‐level studies. Each of the major designs (as well as statistical power) include data analysis applications, therefore, students should have at least a fundamental knowledge of statistics to succeed in the course. EVAL 6970 Experimental and Quasi‐Experimental Designs for Applied Research and Evaluation Spring 2011 Spring 2012 EVAL 6970 1 Credit and Course Hours 3 semester hours This class meets from 6:00 PM to 8:30 PM on Thursdays beginning January 12, 2012 and ending April 26, 2012. The course meets in Ellsworth Hall, room 4410, in The Evaluation Center. Instructors Chris L. S. Coryn, Ph.D. Assistant Professor of Evaluation, Measurement, and Research Director of the Interdisciplinary Ph.D. in Evaluation Telephone: 269‐387‐5906 E‐mail: [email protected] Website: http://www.wmich.edu/evalphd/2010/05/chris‐coryn/ Anne Cullen, Ph.D. Program Officer Fetzer Institute E‐mail: [email protected] Teaching Assistant Robert McCowen Interdisciplinary Ph.D. in Evaluation Doctoral Student Assessment for Learning Fellow E‐mail: [email protected] Course Website The website for this course is located at http://www.wmich.edu/evalphd/courses/eval‐6970‐experimental‐and‐quasi‐experimental‐designs‐for‐applied‐research‐and‐evaluation/. From this site students can access course lecture no tes, homework assignments, data sets, and other materials related to the course. Office Hours By appointment Course Objectives This course has multiple student learning objectives. Students will be expected to develop the following knowledge, skills, and abilities, including but not limited to: 1. Demonstrate a clear understanding of the vocabulary of research design and related vocabularies (e.g., caus e, effect, counterfactual, random assignment, random selection, attrition, con trol group, comparison group, manipulation check, moderating variable, mediating Spring 2012 EVAL 6970 2 variable, nonequivalent dependent v a riable) and how to apply those concepts to the construction and criticism of designs 2. Demonstrate a clear understanding of the theoretical, philosophical, and ideological foundations of contemporary research paradigms and their inquiry strategies 3. Demonstrate a clear understanding of the differences between causal description and ca usal explanation 4. Demonstrate a clear understanding of the various types of validity and their relation to design, measurement, and analysis 5. Demonstrate a clear understanding of threats to the validity of inferences and conclusions as related to internal and external validity 6. Identify plausible threats to the validity of inferences and conclusions in studies found in the peer‐reviewed, serial literature 7. Design high‐quality, cause‐probing studies under both ideal and less than ideal conditions by using elements of design 8. Make intelligent, informed decisions when designing research and evaluation studies that logically couple research questions to elements of design under consideration of their costs and benefits Throughout the course students will also work on improving their communication and interpersonal skills, which are vitally important in evaluation, including but not limited to: 1. Conveying constructive criticism in a professional, balanced, and tactful manner 2. Facilitating discussion to engage others in dialogu e about research and evaluation design 3. Writing clearly and concisely for both academic and non‐academic audiences 4. Giving high quality, professional oral presentations for both academic and non‐academic audiences Required Textbook The textbook required for this course is: Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi‐experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. Spring 2012 EVAL 6970 3 Supplementary Readings In addition to the readings from the textbooks, the following addi tional readings are encouraged: Boruch, R. F. (1998). Randomized contro lled experiments for evaluation and planning. In L. Bickman & D. J. Rog (Eds.), Handbook of applied social research methods (pp. 161‐192). Thousand Oaks, CA: Sage. Boruch, R. F., & Rui, N. (2008). From randomized controlled trials to evidence grading schemes: current state of evidence‐based practice in social sciences. Journal of Evidence‐Based Medicine, 1(1), 41‐49. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multi‐trait, multi‐method matrix. Psychological Bulletin, 56(2), 81‐105. Cook, T. D. (2002). Randomized