ABSC NOTES 2 RELIABILITY quantifying the degree to which you and your partner s observations agreed Why is it important to assess reliability It s our best measure of the accuracy of the observations To rule out experimenter bias To assess observer drift If you don t no one will believe you How often do you assess reliability Before the study starts assess reliability every time Once reliability is at an acceptable level you can start the study Once the study is underway you need to assess reliability periodically 30 of the time seems to be the industry standard Quantifying reliability math time Reliability agreements agreements disagreements What reliability level is acceptable MILLER SUGGESTS 80 is ACCEPTABLE IF IT S A NEW DEFINITION He suggests 90 is acceptable if the definition is established Trial Reliability the method of calculating reliability that we just used used whenever you select interval or time sample recording methods Frequency reliability used whenever you select outcome or event recording FREQUENCY RELIABILITY agree agree disagree LESSON QUIZ 4 OPERANT behavior is behavior affected by reinforcement When instances of behavior are uniform you would use EVENT recording FREQUENCY reliability is used when comparing the total count between two observers Karl Jamie 68 75 Candice counted the number of times Dr Reed said basically in class on Tuesday She got a count of 12 Jerry counted the same behavior on Wednesday and counted 12 also Can we conclude that Candice and Jerry s observations are reliable NOOOOO Mark developed a behavioral definition for being angry as showing redness in the face and bulging of the eyes Mark wanted to determine if his definition of anger was equivalent to how people outside of his study defined being angry Mark was interested in the SOCIAL VALIDITY of his definition Lauren and Molly are observing how many times a child in their daycare room bites her fingers It can be hard to tell sometimes if the child is putting her fingers in her mouth to bite them or if her hands are just near her mouth Occasionally the child would have her hand near her mouth but not biting her fingers and Lauren would count this as an occurrence of the behavior they are looking for Molly would look at Lauren recording this behavior and would record it herself If Molly was not recording these instances Lauren would not either This is a problem Molly and Lauren are not INDEPENDENT observers Do you think that the following observations are reliable Hint this question is asking you to calculate their interrater reliability and determine if it is high enough YES NO Jack counted his professor saying um 42 times in Monday s 9 9 50 class period Savannah did the same observation at the same time and came up with 46 um s YES Behavior analysis is distinct from other areas of psychological research because we make MANY observations until the behavior of interest reaches a stable level Reliability is an estimate of the accuracy of your observations whereas SOCIAL VALIDITY refers to whether you are actually measuring what you intended to measure RELIABILITY is the extent of agreement between two independent sets of observation There must be an agreement between the definition used by an outside judge and the BEHAVIORAL DEFINITION developed by the behavior analyst to achieve social validity To assess social validity you need to have a n INDEPENDENT OBSERVER report whether they believe the definition captures the important characteristics of the behavior being studied Brian counted the number of times Mr Johnson said like in class on Monday Brian counted 15 Kristen counted the same behavior on Wednesday and counted 15 also Can we conclude that Brian and Kristen s observations are reliable NO different days When asking people how often they exercise they are likely to say they exercise much more than they actually do What approach to observing behavior generally deals with problems like this SELF REPORT