PSYC 612, SPRING 2007Lecture 3: PsychometricsFebruary 6, 2007Contents1 Exam Discussion 21.1 An Apology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.1 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.2 Lab Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Introduction to Psychometrics 33 An Example 44 Part 1: Cursory review of the assigned readings 84.1 Thompson and Vacha-Haase (2000) . . . . . . . . . . . . . . . . . . . . . . . 84.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84.1.2 Key Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2 Cortina (1993) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2.2 Key Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.3 Borgatta and Bohrnstedt (1980) . . . . . . . . . . . . . . . . . . . . . . . . . 104.3.1 Key Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Part 2: Important aspects not covered in the readings 105.1 Classical Test Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105.2 Latent Response Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135.2.1 Latent Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135.2.2 Models Assuming Latent Structures . . . . . . . . . . . . . . . . . . . 145.3 Unidimensionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Part 3: Advanced topics 1511 Exam DiscussionI realize the exams are producing a tremendous amount of anxiety, anger, frustration, andvitriol (aimed at me and my TA’s). First, I want to apologize for some of the rough spots.Those rough spots are due to some problems in the software that I did not anticipate. Second,I want to ensure every student that you will get the feedback that is necessary to learn fromyour mistakes. Finally, I want to go over the rationale of the testing and the lab discussionsso that everyone understands the intent and process.1.1 An ApologyThe feedback I received indirectly - since few of you were bold enough to come see me - wasuniformly in opposition to the current testing practice. Many of you expressed displeasurewith the lecture format but that is another topic that I do not intend to discuss today or atall this semester. I will revisit the format of the lectures after the semester. For this semester,I intend to fix the current testing system to ensure that everyone gets adequate feedback.What adequate means might be something different for each one of you so adequate willmean simply that you will learn about the mechanism for each question. You will also seehow you performed on each item once everyone is done with the exams and the new materialis posted on the exam site (expect it Tuesday evening). For now, please accept my apologyif the item level feedback was not available to date. It is part of the system but the system isnot working as I expected. To rectify the situation, I intend to offer you the option of eitherthrowing out the first two exams or keeping them - you decide. You may drop your lowestgrade from this point forward. Your exam grade will be based upon the raw percent totalat the end of the semester so you have plenty of time to figure out which exam to drop.1.2 RationaleThe following is the rationale for both the testing procedure and the lab discussions. Beforedetailing each, I will cover the rationale for the items and how we select those items. Everyweek, the TA’s and I write items based upon the readings and the lecture notes. We eachwrite three to four items. Those items are aimed at the .5 level of difficulty (i.e., half of thestudents ought to pass each item). Items are written at that level because the grade systemis designed with that level of difficulty in mind. Moreover, a test comprised of items at the .5difficulty level produces the greatest variance - an outcome that is dear to statisticians andprofessors alike. Variance allows us to make reliable decisions on grades. A lack of variancein the performance scores leads to precarious decisions with almost no psychometric defense.Consider the grading criteria detailed in the syllabus. For doctoral students, grades areassigned based upon a weighting system where 60% of their grade is determined by theirexam performance. Masters students’ grades weigh the exams at 70% of their total grade.Consider the situation where a masters student aims for a “B” in the course. To achieve thatgrade, the student must perform at least at a 57% on the exams to get that grade. How didI figure that out? Well the exams are worth 70% and the lab grades are essentially invariant2(provided you complete the assignments). You need only 70 total points (with 5 additionalpoints that come from my benefit of the doubt that almost everyone received). So if you geta free 35 points toward a total 75 needed for a “B” then …