View Full Document

# irt07abim_lecture08

View Full Document
View Full Document

2 views

Unformatted text preview:

Test Reliability D Development l tU Using i IRT American Board of Internal Medicine Item Response Theory Course Overview Reliability with IRT Item and Test Information Functions Concepts Equations Uses U and dE Examples l Optimal p Test Design g Reliability with IRT We all know that reliability precision is a desirable property for an assessment The more reliable a test is is the more precisely we can measure the construct For F any scaling li procedure d IRT or CTT CTT as reliability goes up the standard error of measurementt goes down d Reliability with IRT In CTT reliability is a one number one number summary of test precision and there is a corresponding single standard error of measurement that is used for any test score In IRT test precision is conceptualized as something called Information which is conditional on the trait level being measured S Some tests t t could ld measure certain t i ttrait it levels l l very well but measure others poorly Reliability with IRT A further advantage of IRT with respect to evaluating reliability is that we can consider the amount of Information an item and or a test provides In CTT CTT measures of item quality exist exist but these are only indirectly related to what the reliability of the test will be be Item Information Function Item Item Information Information indicates an item s item s usefulness for assessing ability By usefulness we basically mean how g good an item is at distinguishing g g examinees with lower ability levels from those with higher ability levels levels Information Precision 1 0 P u 1 0 8 0 6 04 0 4 0 2 1 0 0 0 3 2 1 1 2 3 1 2 3 Ability 0 8 IInfo 0 0 6 0 4 0 2 0 0 3 2 1 0 Ability Item Information Function Items are basically more informative where the slope of the ICC is steepest which happens when steepest when bj is relatively close to i aj is relatively high high and cj is relatively low If cj 0 0 an iitem provides id iits maximum information when i bj 1 0 a 1 0 10 P u 1 0 8 c 0 0 0 6 b 1 0 or 2 0

Unlocking...