Fall 2012 Exam 3 (FINAL EXAM)This is only a guide to aid in your studying for Exam 3. You could potentially be tested on anything we covered in discussion, lecture, or chapters 6-7, even if it is not on this guide. Chapter 6: 1. What is the general picture of adaptation and its relationship to communication?a. Adaptation: involves response, responder appears to adapt on the basis of experience i. Humans experiment with various alternatives and choose some more oftenthan others because of the pleasurable and un-pleasurable consequences they have experienced following the choiceii. Expect these choice processes to increase the effectiveness of behavior in achieving individual goals behavior is adaptively rational iii. Someone or the world communicates to a person who then learns and adaptsb. Basic Idea of Modeli. Assume that human/animals choose some alternative: that following his choice he receives some kind of reward or penalty; that in some way he notes the result and attributes it to his choice. Over time, he reduces his propensity to choose alternatives that have been followed by bad consequences and increases his propensity to choose alternatives that havegood consequencesii. A kid gets in trouble to get attention: the trouble is message that says “attend to me”c. Behavioral Psychotherapy- pg. 282i. Bobby example: withdrawn child 1. Wants Bobby to become more social so: Change in reinforcement conditions (teacher to give attention only when Bobby was with the group)  Resultant change in behavior (Bobby gradually spentmore and more time with the group and less time withdrawn by himself 2. Additional Experiments to make sure the experiment was not a chance accident: a. Teacher return to her original pattern of behavior give Bobby special attention when he was by himself and no special attention when he was with the other childreni. Theory predicts that Bobby should respond to the new rewards by returning to former withdrawn behavior  when the teacher goes back to the new reward system Bobby becomes social again 2. What is the basic model of the chapter? & Understand the Alfred examplea. Common Factorsi. All involve the response of a human/animalii. Response changes over time and adapts on the basis of experienceiii. Pleasurable vs. un-pleasurable consequencesiv. We expect these choice processes to increase the effectiveness of behaviorin achieving individual goal. Thus, behavior is adaptively rational. v. False learning: when people learning in an apparently intelligent way come to believe things that are not true vi. As long as the probability is neither 1.0 nor 0, either of the two responses could actually occur on a particular trial vii. Average probability on any particular trial is also the expected proportion of individuals that will respond in that way if each individual has been exposed to the same experiential situationviii. Results of a simulation depend on three parameters of the model:1. The learning increment associated with reward, a2. The learning increment associated with lack of reward, b 3. The initial probability of choosing one alternative rather than the other, Pl(0)b. Basic Model: Reinforcement Learningi. Alfred mouse experiment: turns right he gets food vs. turn left gets no food1. We observe that behavior (turning right or left) that is reinforced (rewarded) becomes more frequent whereas behavior that is not reinforced becomes less frequent 2. Description of learning processes:a. Pr(0)= initial probability of turning right before the first trialb. Pl(0)=initial probability of turning left before the first triali. If only the right-hand box contains food and the animal is capable of learning about this environment, Pr will increase and Pl will decrease1. Pr(t+1)=Pr(t)+increment3. How could we model the increment? What are the possible models and which one works?i. Increment: the amount that has been learned as a result of the trialii. Two models1. A constant increment model: assume that Alfred happened to turn right initially, was rewarded, and that the learning increment for turning right is 0.2 **Fails because it implies probabilities outside of the 0-1 range and it does not fit data wella. Pr(t+1)=Pr(t)+0.2b. If Alfred was originally neutral in his turning preference: Pr(0)=0.5 then..i. Pr(1)=Pr(0) + 0.2 = 0.5 + 0.2= 0.7c. So at the beginning of the second trial Alfred’s probability to turn right is 0.7. Suppose he happens to turn right on the second trial.i. Pr(2)=Pr(1) + 0.2 = 0.7 + 0.2= 0.9d. So after two trials Alfred has a 90% chance of turning correctly. e. *Any model with a constant increment, no matter how small, must eventually produce illegal probabilities 2. A constant proportion model: the quantity (1-Pr) represents the amount that Alfred has yet to learn about this maze. Example: current probability of turning right were 0.7amount left to learn would be 0.3, however, if it were 1 he would have nothing left to learn about this maze **Produce acceptable probabilities and predictionsa. Assume: In each trial Alfred learns a constant proportion ofthe amount he has yet to learn. b. Call a the learning proportion (or rate), increment will be a(1-Pr) i. Example: initial chance of turning right is 0.5, learning rate, a, is 0.3 1. Pr(1)= Pr(0) + increment2. Pr(1) = Pr(0) + a[1-Pr(0)] = 0.5 + 0.3(1-0.5)a. 0.5 + 0.15 = 0.653. Second trial: Pr(2)=Pr(1) + a[1-Pr(1)] = 0.65+ .0.3 (1-0.65)a. =0.65 + 0.105 = 0.7554. What are the assumptions of the various adaptation equations? *Expansion of the constant proportion modela. Assumption 1: Alternative Behaviors for the Individual: any individual has some set of possible alternative behaviors in which he might engage i. The mouse may turn left or rightii. Start with a list of mutually exclusive alternative behaviors iii. Bobby exampleBobby’s Alternatives1. S- play with others (Designate “S” meaning social)2. A- play alone (designate “A”, meaning alone)b. Assumption 2: State of the Individual: individual is in some state with respect to the alternative behaviors at any point in time, individuals state may be described by a set of probabilities i. T-maze: the mouse has a probability of going left and a probability of going right at any point in time- these probabilities add up to 1ii. Initially going right/left probability is 0.5 iii. As adaptation precedes, probabilities might changeiv. Bobby exampleBobby’s Current State1. Probability of S = 0.252. Probability of