3/23/20081Psy 427Cal State NorthridgeAndrew Ainsworth PhDCal State Northridge - Psy 427 1 Dichotomous Format Two alternatives True/False MMPI/2; MMPI/A Polytomous or Polychotomous Format More than two alternatives Multiple choice Psy427 Midterm, SAT, GRE,Cal State Northridge - Psy 427 2 Distractors Item Formats▪ Incorrect choices on a polychotomous test▪ Best to have three or four BUT -▪ one study (Sidick, Barret, & Doverspike, 1994) found equivalent validity and reliability for a test with two distractors (three items) as one with four distractors(five items). SO, best might be to have two to four (further study is needed)Cal State Northridge - Psy 427 33/23/20082 Depends… Correction for guessing: R is the number correct W is the number incorrect n is the number of polytomous choices If no correction for guessing, guess away. If there is a correction for guessing, better to leave some blank (unless you can beat the odds)Cal State Northridge - Psy 427 4 1WCorrected Score Rn= −− Likert scales On a rating scale of 1-5, or 1-6, 1-7, etc. where 1 = strongly disagree 2 = moderately disagree 3 = mildly disagree 4 = mildly agree 5 = moderately agree 6 = strongly agree rate the following statements….Cal State Northridge - Psy 427 5 Likert scales Even vs. odd number of choices ▪ Even numbers prevents “fence-sitting”▪ Odd numbers allows people to be neutral Likert items are VERY popular measurement items in psychology. Technically ordinal but are often assumed continuous if 5 or more choices With that assumption we can calculate means, factor analyze, etc. Cal State Northridge - Psy 427 63/23/20083 Category format Like Likert, but with MANY more categories▪ e.g., 10-point scale Best if used with anchors Research supports use of 7-point scales to 21-point scalesCal State Northridge - Psy 427 7 Visual Analogue ScaleNo Headache Worst Headache Also used in research dials, knobs time samplingCal State Northridge - Psy 427 8 Both used in qualitative research as well as quantitative research Checklists Present list of words (adjectives) Have person choose to endorse each item Can determine perceptions of concepts using checklists.Cal State Northridge - Psy 427 93/23/20084 Adjective Checklists (from http://www.encyclopedia.com/doc/1O87-AdjectiveCheckList.html) In psychometrics, any list of adjectives that can be marked as applicable or not applicable ▪ to oneself▪ to one's ideal self▪ to another person, OR ▪ to some other entity or concept. Cal State Northridge - Psy 427 10 Checklists When written with initial uppercase letters (ACL), the term denotes more specifically a measure consisting of a list of 300 adjectives, from absent-minded to zany Selected by the US psychologist Harrison G. Gough (born 1921) and introduced as a commercial test in 1952.  The test yields 24 scores, including measures of personal adjustment, self-confidence, self-control, lability, counselling readiness, some response styles, and 15 personality needs, such as achievement, dominance, and endurance. Cal State Northridge - Psy 427 11 Q-Sorts Introduced by William Stephenson in 1935▪ PhD in physics 1926; PhD in psychology in 1929▪ Student of Charles Spearman Goal: to get a quantitative description of a person’s perceptions of a concept Process: give subject a pile of numbered “cards” & have them sort them into piles Piles represent graded degrees of description (most descriptive to least descriptive).Cal State Northridge - Psy 427 123/23/20085 Q-Sorts Means of self-evaluation of client’s current status  The Q-Sort consists of a number of cards, often as many as 40 or 50, even 100 items each consisting of a single trait, belief, or behavior. The goal is to sort these cards into one of five columns ranging from statements such as, ‘very much like me’ to ‘not at all like me.’ There are typically a specific number of cards allowed for each column, forcing the client to balance the cards evenly. Example: California Q-sort , Attachment Q-sortCal State Northridge - Psy 427 13Cal State Northridge - Psy 427 14Cal State Northridge - Psy 427 15Most Uncharacteristic2 3 4 5 6 7 8Most Characteristic128558121618163/23/20086Cal State Northridge - Psy 427 16Attachment Q-sort Distribution (number of items per pile designated)  Methods used to evaluate test items. What are good items? Techniques Item Difficulty (or easiness) Discriminability▪ Extreme Group▪ Item/Total Correlation Item Characteristic Curves Item Response Theory Criterion-Referenced TestingCal State Northridge - Psy 427 17 The proportion of people who get a particular item correct or that endorse an item (if there is no “correct” response, e.g. MMPI) Often thought of as the item’s easiness because it is based on the number correct/endorsedCal State Northridge - Psy 427 183/23/20087 The difficulty can be given in proportion for or it can be standardized in to a Z-valueCal State Northridge - Psy 427 19[ln(1 )] ln( )1.7p pZ− −= For example a test with the difficulty of .84Cal State Northridge - Psy 427 20(ln(.16) ln(.84)) 1.7( 1.83 .17) 1.7 1.66/1.7 1.00( 2 2 )Zis typical range−=− +== −= −− →If you are taking a criterion referenced test in a social psychology course and you need to score a 92 in order to get an A, the criterion isa) Social Psychology *b) Scoring a 92c) Getting an Ad) Not enough info.Cal State Northridge - Psy 427 213/23/20088Cal State Northridge - Psy 427 22[ln(1 )] ln( )1.7(ln(.65) ln(.35)) 1.7( .431 1.050) 1.7 .619 /1.7 .364p pZZ− −=−=− +===The correlation between X and is .54. X has a SD of 1.2 and Y has a SD of 5.4. What is the regression coefficient (b) when Y is predicted by X?a) .12b) 2.43*c) .375d) .45Cal State Northridge - Psy 427 23Cal State Northridge - Psy 427 24[ln(1 )] ln( )1.7(ln(.49) ln(.51)) 1.7( .713 .673) 1.7 .004 /1.7 .00235p pZZ− −=−=− +== −= −3/23/20089 For the following set of data [5 9 5 5 2 4 ], the mean isa) 4b) 5*c) 4.5d) 6Cal State Northridge - Psy 427 25Cal State Northridge - Psy 427 26[ln(1 )] ln( )1.7(ln(0) ln(1)) 1.7 p pZZerror− −=−== Mathematically: half-way between chance and 100%. Steps (assuming a 5-choice test) 1. Find half-way between 100% and chance▪ 1 - .2 = .8,