KIN 4310 1nd Edition Lecture 7Outline of Last Lecture I. WorkshopII. Excel FunctionsIII. Objectives: Measurement TheoryIV. MeasurementV. Terms MeasurementVI. Measurement ChallengesVII. Variables of InterestVIII. Classical Test Theory: O = T + EOutline of Current Lecture I. Terms: True ScoreII. Terms: Observed ScoreIII. Terms: ErrorIV. Levels of MeasurementV. Nominal ScalesVI. Ordinal ScalesVII. Interval ScalesVIII. Ratio ScalesIX. Levels of MeasurementThese notes represent a detailed interpretation of the professor’s lecture. GradeBuddy is best used as a supplement to your own notes, not as a substitute.X. Objectives: ReliabilityXI. ReliabilityXII. ReliabilityXIII. ReliabilityXIV. ReliabilityXV. Test-Retest ReliabilityXVI. Interrater ReliabilityXVII. Internal Consistency ReliabilityXVIII. Parallel Forms ConsistencyXIX. What is Reliable?XX. Reliability ExampleXXI. Reliability Example cont.XXII. Improving ReliabilityCurrent Lecture*CORRECTION: variability will be NOT be on Exam 1 I. Terms: True Scorea. True score: the actual amount of the attribute you want to measure (e.g., true self-esteem, true dietary intake, etc.)b. Assumption: the construct is real and exists much like a blood level or atomic weight, if only we could measure it accurately II. Terms: Observed Scorea. Observed score is what results from your measurementb. We want our observed score to be as close as possible to the true score, but there is often some discrepancy between what we observe and what is actually thereIII. Terms: Errora. Error: anything you did not intend to measure that affected scoreb. Systematic error recurs on repeated measurements and affects scores predictabilityi. ex. scale, putting clock aheadc. Nonsystematic error is unpredictable and varies with every measurementi. Hard to deal with but can be doneii. Can deal with this by doing repeated measurementIV. Levels of Measurementa. Nominali. Worst out of the allb. Ordinalc. Intervald. Ratioi. Can use all statistical terms but ratio is the best oneV. Nominal Scalesa. Mutually exclusive, unordered categoriesb. For example:i. Genderii. Political affiliationiii. Country of originiv. Movie genreVI. Ordinal Scalesa. Characteristics that can be put in order, but there is no consistent difference between adjacent scoresb. For example:i. Rank (3rd place)ii. Olympic Medal Color (gold, silver, bronze)iii. Movie Ratings VII. Interval Scalesa. Variables refer to quantities of units on a continuumb. Distance between variables is meaningfulc. For example:i. Temperature in Fahrenheit or Celsiusii. Dates (e.g. year)iii. Intelligence Quotient (IQ)d. Generally well use unitse. You can get into the negative interval scale like before year 0 for examplef. There’s no absolute zero g. Can only subtract hereVIII. Ratio Scalesa. Variables refer to quantities of units on a continuumb. Has an absolute zeroc. For example:i. Annual salaryii. Distance traveled in the 12-minute run testiii. Body fat percentageiv. Statured. Can be ratio’s like: “he makes twice as much money as me”e. Can only divide hereIX. Level of Measurement a.X. Objectives: Reliability and Validitya. Define reliabilityb. Define validity and identify different types/purposesc. Apply these concepts to the development of tests and surveysd. Will wait to discuss validity in section twoXI. Reliabilitya. The degree to which scores are: i. Free from errors of measurementii. Consistent, or stable across a variety of conditionsXII. Reliabilitya. Types of reliabilityi. Test-retest reliabilityii. Interrater reliabilityiii. Internal consistency reliability iv. Parallel forms reliabilityb. Refer to book , chapter 6, for this to get more in depthXIII. Reliabilitya. Measures of reliability are correlationsb. Correlations represent consistency over time, across different raters, across different raters, across different assessment forms, or consistency of items withinthe measurec. This is how we tie r, linear correlation coefficient, to reliabilityXIV. Reliabilitya. There are 4 types that we’ll go overb. You can look at the line y=xXV. Test-Retest Reliabilitya. Correlation of scores on a test given at two separate timesb. Longer times require greater stabilityc. Affected by: Change, carry-over effects i. Carry over effects are like when you take a test for the first time, and then again a second time and do better the second time just because you’re more familiar with the test structure d. Alternate Forms or Split Half Reliability can address some of the problemsi. Don’t worry about thisXVI. Interrater Reliabilitya. Correlation of scores measure by two different observers or ratersb. Example:i. Reading an ambiguous scaleii. Subjective assignment of quantitative scores (judging gymnastics, diving, figure skating) because there is subjective erroriii. Operating a stopwatchXVII. Internal Consistency Reliabilitya. When a test consists of multiple items, do all items assess the same dimension?b. A function of the relationship between items on a scale and the number of itemsc. Cronbach’s Alphai. Won’t test over this. It’s a number like rd. Ex. Presidents Test in Public Schooli. Sit upsii. Pushups iii. Flexibility iv. Mile runv. Etc.vi. Its supposed to measure fitnesse. Ex. field sobriety testXVIII. Parallel Forms Reliabilitya. Correlation of scores between two different versions of a testb. Example:i. IQ Testsii. Exam 3A versus Exam 3Bc. Here, it shouldn’t matter what version of the test you get, there should be the same level of difficultyXIX. What is Reliable?a. Reliability coefficients should be positivei. 0.0 to 1.0b. General rules of thumb:i. Test-retest: 0.60 < r <equalto 1.00ii. Interrater: 85% agreementiii. Parallel forms: 0.70 < r<equalto 1.00iv. Don’t have to remember these numbersc. High reliability DOES NOT alone constitute quality or validity!i. Will talk about this laterXX. Reliability Examplea. Tinettie Balance Assessment Tooli. PT’s use thisb. Two PT’s assess the same group of patientsi. r = 0.98ii. Very good interrrater reliability!c. One PT assesses each patients twice in 48 hoursi. r = 0.98ii. Very good test-retest reliability!XXI. Reliability Example cont.a. Self-Efficacy Assessmentb. 20 students write two different self-efficacy tests. The correlation coefficient turns out to be r = 0.32c. Poor parallel test reliabilityd. (This result is not statistically significant according to a two-tailed test with significance of 0.05)XXII. Improving Reliabilitya. Make sure