Chapter 5 Measuring Variables and Sampling variable a condition or characteristic that can take on different values or categories failing to measure your variable accurately can lead to flawed data measurement the assignment of symbols or numbers to something according to a set of rules to as variables 4 scales of measurement nominal ordinal interval ratio also referred nominal scale the use of symbols such as words or numbers to classify or categorize measurement objects into groups or types simplest most basic scale o o nonquantitative o o o used to name categorize or classify ordinal scale a rank order measurement scale identifies types rather than amounts i e gender personality types country you were born in etc o allows you to determine which person is higher or lower on a variable of interest but not exactly how much compared to the other i e order of finish in a marathon social class rank ordering for a job o o used to rank order objects or individuals interval scale scale of measurement with equal intervals of distance between numbers i e Celsius temperature year IQ scores etc o o does not possess an absolute zero point o used to rank order plus has equal intervals or distances between adjacent numbers ratio scale a scale of measurement with rank ordering equal intervals and an absolute zero point o most quantitative level of measurement o i e weight height response time Kelvin temperature annual income two major properties of good measurement are validity and reliability reliability the consistency or stability of scores 4 primary types of reliability test retest equivalent forms internal consistency interrater reliability reliability coefficients are commonly obtained as quantitative indexes of reliability reliability coefficient type of correlation coefficient used as an index of reliability o should be strong and positive i e 0 70 to indicate strong consistency of relationship equivalent forms consistency of a groups individual scores on two or more versions of the same test o o i e SATs ACTs success of this method depends on equivalence of the test forms internal consistency reliability consistency with which items on a test measure a single construct o affected by length of test the longer the more reliable o goal is to obtain high reliability with relatively few items for each construct o most commonly reported index of internal consistency is coefficient alpha or Cronbach s Alpha should be 70 or higher high value means items are consistently measuring the same thing o for multidimensional tests coefficient alpha should be reported for each dimension separately interrater reliability degree of consistency or agreement between two or more scorers judges observers or raters should be strong and positive o o also measured by interobserver agreement the percentage of time that different observers ratings are in agreement validity accuracy of inferences interpretations or actions made on the basis of test scores test defined as any measurement procedure or device validation is an inquiry into the soundness of the interpretations proposed for scores from a test all validity types are part of construct validity operationalization the way a construct is represented and measured in a particular research study o defining how you determine what belongs in the study o i e depressed meaning individuals scoring above a 20 on the Beck Depression Inventory o do the operations produce a correct or appropriate representation of the intended construct validation gathering of evidence regarding the soundness of inferences made from test scores o continual process 3 major ways to collect evidence of validity based on content based on internal structure based on relations to other variables validity evidence based on content content related evidence judgment by experts of the degree to which items tasks or questions on a test adequately represent the construct o 1 do the items appear to represent the thing a researcher is attempting to measure face validity o 2 does the set of items underrepresent the construct s content was o 3 do any of these items represent something other than what the anything important excluded researcher is trying to measure validity evidence based on internal structure must determine how many dimensions are being used o multidimensional construct construct consisting of 2 or more o dimensions contrasted with a unidimensional construct factor analysis statistical technique analysis procedure to determine the number of dimensions present in a set of items o number of subsets of items indicates number of dimensions o indexes are used to indicate the degree of homogeneity of each dimension or factor o homogeneity degree to which a set of items measures a single construct 2 primary indices are the item to total correlation and the coefficient alpha validity evidence based on relations to other variables obtained by relating your test scores with one or more relevant and known criteria o a criterion is a standard that you want to correlate with or predict o accurately on the basis of your test scores validity coefficient type of correlation coefficient used in validation research o o o o o test scores should be related to the criterion in the predicted direction and magnitude criterion related validity degree to which scores predict or relate to a known criterion such as a future performance or an already established test predictive validity degree to which scores obtained at one time correctly predict scores on a criterion at a later time concurrent validity degree to which test scores obtained at one time correctly relate to the scores on a known criterion obtained at approximately the same time o convergent validity evidence based on the degree to which the focal test scores correlate with independent measures of different constructs o o discriminant validity based on the degree to which the focal test scores do NOT correlate with measures of different constructs known groups validity evidence degree to which groups that are known to differ on a construct actually differ according to the test used to measure the construct Using Reliability and Validity Information norming group the reference group upon which reported reliability and validity evidence is based if the people you intend to use a test with are very different from the people in the norming group then the validity and reliability evidence provided with the test becomes questionable 2 most important