1 Nova Southeastern University Center for Psychological Studies Course Outline Course # & Title: PYCL 507 – Research and Evaluation for Counselors Credit Hours: 3 Location/Section: Palm Beach/PB2 Dates: September 23, 24, 25, 2011 October 14, 15, 16, 2011 Instructor: Craig D. Marker, Ph.D. [email protected] Class Times: Fridays 6:00 PM to 10:00 PM Saturdays 8:30 AM to 6:00 PM Sundays 8:30 AM to 5:30 PM Required Texts: Marczyk, G. R., DeMatteo, D., Festinger, D. (2005) Essentials of Research Design and Methodology, John Wiley and Sons. ISBN: 0-471-47053-8 Course Description: This course seeks to prepare mental health and school counselors to be informed consumers of research and evaluation. It covers basic statistics, research designs, and program evaluation within the counseling and educational fields. It provides experience in developing accountability measures and in reading research and evaluating reports applicable to multicultural populations. Validity issues in research (internal and external), cross-cultural issues, and research ethics are emphasized throughout. Course Objectives: 1. Understands how to critically evaluate research relevant to the practice of counseling and knows models of program evaluation and basic strategies for evaluating counseling outcomes in distinct counseling settings. 2. Understand basic types of research methods, including quantitative, qualitative, single case designs, action research, program evaluation and outcome based research. 3. Knows current methods of using data to inform evidence based practice, decision making and accountability, and develops measurable outcomes for counseling programs, activities, interventions and experiences. 4. Understands the importance of research and demonstrates an ability to analyze2 and use data to increase effectiveness of counseling programs. 5. Understands ethical and culturally relevant strategies for interpreting and reporting the results of research and/or program evaluation studies. 6. Knows the principles, models, and applications of needs assessment, program evaluation, and the use of findings to effect program modifications. 7. Demonstrate knowledge of statistical concepts, including scales of measurement, measures of central tendency, indices of variability, shapes and types of distributions, and correlations 8. Examine statistical methods used in conducting research and program evaluation. 9. Understand reliability (i.e., theory of measurement error, models of reliability, and the use of reliability information) and validity (i.e., evidence of validity, types of validity, and the relationship between reliability and validity). 10. Use professional databases to locate research Calendar: First Weekend Prior to Friday, read Marcyzk et al. Chapters 1, 2, 3, 4, 6 (additional reading assignments and information about the readings may be posted on WebCTor emailed to you) Friday -Introduction and course overview; Scientific Approach (Quiz) Saturday -Developing Research Questions; Extraneous Variability (Quiz) Sunday -Measurement; Methods of Data Collection; Validity (Quiz) Second Weekend Prior to Friday, read Marcyzk et al. Chapters 5, 7, 8, 9 (additional reading assignments will be assigned in class) Friday -Descriptive Statistics (Quiz; Paper due at beginning of class) Saturday -Inferential Statistics (Quiz) Sunday -Inferential Statistics (Quiz) Description of Assignments Quizzes After each topic a short quiz will be given. These quizzes will be based on the lecture and the book. The book is very good at highlighting the most important information in research methods (gray boxes). Although, it may sound intimidating, the quizzes will be short and should be relatively easy if you paid attention to the lectures and read the readings. The quizzes will be a combination of multiple choice, true-false, and short answer. Research Critique Paper In class, we will discuss many research articles and practice our knowledge of research methods. Thus, we will have quite a bit of familiarity with research critiques, with our end goal being that we will be good consumers of research articles. For this assignment, students will critique one selected research article, which will be due at the beginning of the second weekend. Articles will be given out in class. The paper must be a minimum of 5 typewritten, double spaced pages utilizing the publication format3 of the APA. Any outside resources must be referenced. However, this paper is an article critique so it is likely that you will not need outside references. Plagiarism will not be tolerated. If you use someone else’s words, they must quoted. If you use someone else’s idea, it must be referenced. Papers must be turned in by the Friday of the second weekend. Every day late will result in a 5% reduction. Papers will not be accepted after Sunday of the second weekend. A format for critiquing the articles follows: BIBLIOGRAPHIC INFORMATION (provide reference in APA style) SUMMARY OF THE RESEARCH PROBLEM STATEMENT (What is the researcher’s question): BACKGROUND (does the article provide a good rationale for the question): HYPOTHESIS (what does the author predict or what is their guess of what will happen): MEASUREMENT OF VARIABLES What is the dependent variable: What is the independent variable(s): RESEARCH DESIGN (correlational, experimental, quasi-experimental): SAMPLING (Do they use random assignment and/ or random selection): INSTRUMENTATION (how do they measure; is it operationally defined): DATA COLLECTION/ETHICS (how do they collect data / any ethical issues): DATA ANALYSIS: Author's CONCLUSIONS: CRITIQUE Possible Threats to Internal Validity (anything that reduces your confidences in their findings) e.g., History, Maturation, Instrumentation, Testing or Practice Effect, Statistical regression (regression toward the mean), Selection, Mortality, Sequencing, Participant Effects, or Experimenter Effects Possible Threats to External Validity (anything that affects the generalizability of the study Possible Threats to Construct Validity (do they measure the construct they set out to measure) CONCLUSIONS: Do you have confidence in the findings? Why or why not? If you could redo the study to make it better, what would you do? Does the article make sense? Is the article useful? What significance do you see