EDPS 591M: Psychoeducational Measurement Fall 2006 Instructor: Tao (Tom) Hong Office: BRNG 3127 (PUPIL) Telephone: 494-3223 Time: Monday 4:30-7:20 PM Room: BRNG B254 Office Hours: Monday 11:00-12:00 (other times by appointment) Email: [email protected] Required Text: Anastasi, A., & Urbina, S. J. (1997). Psychological testing (7th Ed.). Upper Saddle River, NJ: Prentice Hall. Required Packets: Two course packets (one with readings and another with handouts) are available at Copy Mat located at 135 S. Chauncy Ave.(Phone: 743-5995). PLEASE BRING THE HANDOUT PACKET WITH YOU TO EACH CLASS! Course Purpose: The course purposes are for the student to (a) acquire an understanding of fundamental theory and applications of measurement and (b) be able to critique the psychometric properties of psychological and educational tests. Course Format: The course will include a combination of lecture, discussion, and possibly some lab work, as necessary. Attendance and Participation Policies: The student is expected to attend and come prepared to all classes. Additionally, students are encouraged to ask questions and share information. Course Objectives: The student will acquire knowledge and/or be able to critique tests in terms of: historical, ethical, and legal perspectives of the nature and meaning of assessment. technical properties of standardized and non-standardized testing and other assessment techniques (e.g., scaling of ability or affective items and test scores, norm-referenced and criterion-referenced assessment). statistical concepts, including scales of measurement, central tendency, variability, distributions, correlations. true score theory and reliability. item analysis. validity and gathering validity evidence. issues related to the measurement of diverse populations, including item and test bias. strategies for evaluating instruments.Determination of Grades: Course grades are based on the following: 30% Exam I 30% Exam II 40% Test Review Course Grading Standards: 90 - 100% = A 80 - 89% = B 70 - 79% = C 60 - 69% = D < 60% = F Test Review: Review (summarize and critique) a published test. Students must obtain approval of the test you select by (a) the instructor and (b) your advisor. Tests are available from a variety of sources, including the TRC and clinic, your advisor, your practicum or internship site, and possibly the instructor. The review should be of publishable quality. You should review the instructions for Buros reviewers in the packet, but you should not base your review on an existing review (such as those in the Mental Measurement Yearbook). An example review is included in the reading list folder. You must base the review on the test materials and technical manual that comes with the test itself. Reviews should cover the following: purpose of the test (e.g., theoretical framework, what does it measure, and who is it for? sample item development item analysis scoring methods reliability evidence validity evidence test/item bias studies references Please note: Your review must be typed. Handwritten reviews, incorrect grammar, and misspelled words are unacceptable. Please proofread! Ask someone to proofread your paper. APA 5th edition style must be used. Late papers will not be accepted.Schedule (subject to change, as necessary): Week Date TOPIC READINGS 1 8/21 Orientation 2 8/28 Nature and Uses of Tests Historical Foundations Ethical, Legal, & Social Considerations Chs. 1, 2, 18 Readings: Dalhstrom (1993) 3 9/4 Labor Day No Class 4 9/11 Types of Tests and Scales (Guest Speaker: Dr. Brian French) pp. 411-414, 419-425, 406-408, 459-469 Readings: Motta, Little & Tobin (1993a); Motta, Little & Tobin (1993b) 5 9/18 Review of Statistical Concepts 1. Scales/Levels of Measurement 2. Central Tendency 3. Variability 4. Correlation pp. 49-54 pp. 85-90 6 9/25 Review of Statistical Concepts (contd.) Norms & Standard Scores Ch. 3 7 10/2 Norms & Standard Scores (contd.) True Score Theory & Reliability Ch. 4 Readings: Traub & Rowley (1991); Harvill (1991) 8 10/9 Fall Break No Class 9 10/16 True Score Theory & Reliability (contd.) Midterm Review 10 10/23 Midterm 11 10/30 Item Analysis Ch. 7 Readings: Hambleton & Jones (1993); Harris (1989) 12 11/6 Item Analysis (contd.) 13 11/13 Validity Chs. 5 & 6 Readings: Messick (1989); Campbell & Fiske (1959) 14 11/20 Validity Studies Ch. 7 & pp. 362-370 Readings: Good & Lane (1990); French & Oakes (2004) Maller, Konold, & Glutting (1998) Maller & French (2004) 15 11/27 Diverse Populations and Item and Test Bias pp. 196-200, 164-171, 340-346, 252-270 Readings: Clauser & Mazor (1998) Maller (1998) 16 12/4 Profile Analysis Final Review Readings: McDermott et al. (1990) Maller & McDermott (1997) DUE: Test Review 17 12/11 Final ExamReading List*: **Campbell, D.T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multi-trait multimethod matrix. Psychological Bulletin, 56, 85-105. Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17, 31-47. Dalhstrom, W. G. (1993). Small samples, large consequences. American Psychologist, 48, 393-399. French, B. F., & Oakes, W. (2004). Reliability and validity evidence for the institutional integration scale. Educational and Psychological Measurement, 64, 88-98. Good., R. H., & Lane, S. (1990). Confirmatory factor analysis of the K-ABC and WISC- R for at-risk students: A comparison of hierarchical models. School Psychology Review, 19, 492-504. Hambleton, R. K., & Jones, R. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12, 38-47. Harris, D. J. (1989). Comparison of the 1-,2-, and 3-, parameter IRT models. Educational Measurement: Issues and Practice, 8, 35-41. Harvill, L. M. (1991). Standard error of measurement. Educational Measurement: Issues and Practice, 10, 33-41. Maller, S. J. (1998). Assessing the validity of intelligence test scores through the investigation of differential item functioning. Inquiry, 1, 5-6. Maller, S. J., Konold, T. R., & Glutting, J. J. (1998). WISC-III Factor invariance across samples of children displaying appropriate and inappropriate test-taking behavior. Educational and Psychological Measurement, 58, 467-475. Maller, S.