Unformatted text preview:

Course MaterialsTopics and ReadingsTopics and Readings, continuedAssessmentsTerm Paper:The term paper will review and critique a topic fCourse GradesAccommodationsAcademic IntegrityEDMS 623 Applied Measurement: Issues and Practices Fall 2004 Room 0212 Benjamin Building Instructor: Amy B. Hendrickson, Ph.D. Office: 1230B Benjamin Phone: (301) 405-3627 E-mail: [email protected] Office Hours: By appointment Course Objective This course will provide an introduction to both classical and modern techniques that have been developed to measure inherently unobservable (latent) traits such as ability, preference, and attitude. Students will learn to apply test theory to construct, evaluate and modify tests and measures used in education and the social sciences. Both classical and modern measurement literature will be discussed. The course is designed for students who will be pursuing careers that involve psychometrics, measurement theory or assessment. Course Materials Required Texts: Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. Fort Worth: Holt, Rinehart and Winston Joint Committee on Standards for AERA, APA, and NCME. (1999). Standards for educational and psychological testing. Washington, DC: AERA. Required supplemental articles and chapters are listed later in the syllabus. Students are responsible for obtaining these articles, but two copies of each article will be available at the ETS library. Lecture notes will be distributed in class. 1Readings are given below each topic. Students should read assignments prior to lectures. Coverage dates are not given due to the fact that topic presentation time may vary. However, topics will be covered in the order given below. Topics and Readings Review of syllabus Introduction: Scaling, Testing and Measurement Scaling a. Fechner’s Law b. Weber’s Constant c. Stevens’ Law d. Thurstone’s method of paired comparisons e. Thurstone’s method of successive intervals f. Thurstone’s method of equal appearing intervals. Readings: Text 16-55; Coombs (1950); Green (1954a and 1954b); Guttman (1950); Thurstone (1927 and 1928) Classical Test Theory Assumptions and Derivations Reliability Indices Internal consistency estimates: Spearman-Brown, Cronbach’s alpha, Kuder-Richardson estimates, Hoyt’s ANOVA Test-retest estimates Parallel forms estimates Alternative notions of error Reliability for Criterion-referenced tests Student presentation on standards regarding test user responsibility ____________________________________________ Readings: Text 87-128, 131-154, 192-212; Standards 1-6, 111-118; Dudek (1979); Messick (1989), Harvill (1991) Reliability via Generalizability Theory: A brief introduction G studies D studies Generalizability Coefficient Index of Dependability Calculating variance components in SPSS Student presentation on standards regarding reliability ____________________________________________ Readings: Text 157-189; , Standards 25-36; Brennan (1992) 2Topics and Readings, continued Validity Face validity Content Validity Criterion-related validity Construct validity MTMM technique Factorial validity Obtaining a factor analysis or principal components analysis in SPSS Cronbach and Meehl’s notion of validity Messick’s notion of Validity Student presentation on standards regarding validity ____________________________________________ Readings: Text 217-239; Text 287-306; Campbell and Fiske (1959); Messick (1989), Cizek (2001) Test and Questionnaire Construction The summated scale Likert attitude scales Writing and piloting items for a summated achievement scale Piloting and choosing items for a summated achievement scale Student presentation on standards regarding test construction ____________________________________________ Readings: Text 66-84; 311-336; Standards 9-24; Standards 37-48 Cronbach & Meehl (1955); Cureton (1950); Likert (1932); Messick (1989); Millman & Greene (1989), Moss (1998); Reckase (1998); Spector (1992; pp. 18-46) Setting Standards and Cutoffs Nedelsky procedure Angoff procedure Modified Angoff procedure Contrasting groups procedure Bookmark procedure Student presentation on standards regarding standard setting ____________________________________________ Readings: Text 410-429; Standards 137-150, Cizek (1996); Reckase (2000) 3Topics and Readings, continued Introduction to Item Response Theory Binary response models One-parameter (Rasch) model Two-parameter (Birnbaum) model Three-parameter (Birnbaum) model Item and test characteristic curves Parameter estimation in item response models Logic of maximum likelihood Invariance of item and person parameter interpretations Applications of IRT The item and test information functions Test development Item Banking Computerized adaptive testing Student presentation on standards regarding educational testing ____________________________________________ Readings: Readings: Text 339-371; Standards 137-150; Hambleton, Swaminathan, and Rogers (1991; Chapters 6 &7); Hambleton and Jones (1993); Harris (1989); Mills (1999) Test Norms and Test Equating Developing test norms Equating designs Equipercentile equating Linear equating IRT-based equating methods Student presentation on standards regarding norms and comparability ____________________________________________ Readings: Text 431-479; Standards 49-66; Brennan and Kolen (1987) Bias and Differential Item Functioning (DIF) Definitions of bias and DIF Contingency table approaches to detection of DIF IRT approaches to detection of DIF Follow-up analyses Student presentation on standards regarding test fairness ____________________________________________ Readings: Text 376-392; Standards 73-84; Clauser and Mazor (1998); Hambleton, Swaminathan, and Rogers (1991; Chapter 8); Skaggs & Lissitz (1992) 4Supplemental Readings Brennan, R. L. (1992). Generalizability theory. Educational Measurement: Issues and Practice, 11(4), 27-34. Brennan, R. L., & Kolen, M. J. (1987). Some practical issues in equating. Applied Psychological Measurement, 11, 279-290. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. Cizek, G. J. (1996). Standard-setting guidelines. Educational Measurement: Issues and Practice, 15(1), 13-21. Cizek, G. J. (2001). More


View Full Document

UMD EDMS 623 - Syllabus

Documents in this Course
Load more
Download Syllabus
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Syllabus and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Syllabus 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?