Fall 2012 Exam 3 FINAL EXAM This is only a guide to aid in your studying for Exam 3 You could potentially be tested on anything we covered in discussion lecture or chapters 6 7 even if it is not on this guide Chapter 6 to communication experience 1 What is the general picture of adaptation and its relationship a Adaptation involves response responder appears to adapt on the basis of i Humans experiment with various alternatives and choose some more often than others because of the pleasurable and un pleasurable consequences they have experienced following the choice ii Expect these choice processes to increase the effectiveness of behavior in achieving individual goals behavior is adaptively rational iii Someone or the world communicates to a person who then learns and adapts b Basic Idea of Model i Assume that human animals choose some alternative that following his choice he receives some kind of reward or penalty that in some way he notes the result and attributes it to his choice Over time he reduces his propensity to choose alternatives that have been followed by bad consequences and increases his propensity to choose alternatives that have good consequences ii A kid gets in trouble to get attention the trouble is message that says attend to me c Behavioral Psychotherapy pg 282 i Bobby example withdrawn child 1 Wants Bobby to become more social so Change in reinforcement conditions teacher to give attention only when Bobby was with the group Resultant change in behavior Bobby gradually spent more and more time with the group and less time withdrawn by himself 2 Additional Experiments to make sure the experiment was not a chance accident a Teacher return to her original pattern of behavior give Bobby special attention when he was by himself and no special attention when he was with the other children i Theory predicts that Bobby should respond to the new rewards by returning to former withdrawn behavior when the teacher goes back to the new reward system Bobby becomes social again 2 What is the basic model of the chapter Understand the Alfred example a Common Factors i All involve the response of a human animal ii Response changes over time and adapts on the basis of experience iii Pleasurable vs un pleasurable consequences iv We expect these choice processes to increase the effectiveness of behavior in achieving individual goal Thus behavior is adaptively rational v False learning when people learning in an apparently intelligent way come to believe things that are not true vi As long as the probability is neither 1 0 nor 0 either of the two responses could actually occur on a particular trial vii Average probability on any particular trial is also the expected proportion of individuals that will respond in that way if each individual has been exposed to the same experiential situation viii Results of a simulation depend on three parameters of the model 1 The learning increment associated with reward a 2 The learning increment associated with lack of reward b 3 The initial probability of choosing one alternative rather than the b Basic Model Reinforcement Learning other Pl 0 i Alfred mouse experiment turns right he gets food vs turn left gets no food 1 We observe that behavior turning right or left that is reinforced rewarded becomes more frequent whereas behavior that is not reinforced becomes less frequent 2 Description of learning processes a Pr 0 initial probability of turning right before the first trial b Pl 0 initial probability of turning left before the first trial i If only the right hand box contains food and the animal is capable of learning about this environment Pr will increase and Pl will decrease 1 Pr t 1 Pr t increment 3 How could we model the increment What are the possible models and which one works Increment the amount that has been learned as a result of the trial i ii Two models 1 A constant increment model assume that Alfred happened to turn right initially was rewarded and that the learning increment for turning right is 0 2 Fails because it implies probabilities outside of the 0 1 range and it does not fit data well a Pr t 1 Pr t 0 2 b If Alfred was originally neutral in his turning preference Pr 0 0 5 then i Pr 1 Pr 0 0 2 0 5 0 2 0 7 c So at the beginning of the second trial Alfred s probability to turn right is 0 7 Suppose he happens to turn right on the second trial i Pr 2 Pr 1 0 2 0 7 0 2 0 9 d So after two trials Alfred has a 90 chance of turning correctly e Any model with a constant increment no matter how small must eventually produce illegal probabilities 2 A constant proportion model the quantity 1 Pr represents the amount that Alfred has yet to learn about this maze Example current probability of turning right were 0 7 amount left to learn would be 0 3 however if it were 1 he would have nothing left to learn about this maze Produce acceptable probabilities and predictions a Assume In each trial Alfred learns a constant proportion of the amount he has yet to learn b Call a the learning proportion or rate increment will be a 1 Pr i Example initial chance of turning right is 0 5 learning rate a is 0 3 1 Pr 1 Pr 0 increment 2 Pr 1 Pr 0 a 1 Pr 0 0 5 0 3 1 0 5 a 0 5 0 15 0 65 3 Second trial Pr 2 Pr 1 a 1 Pr 1 0 65 0 3 1 0 65 a 0 65 0 105 0 755 4 What are the assumptions of the various adaptation equations Expansion of the constant proportion model a Assumption 1 Alternative Behaviors for the Individual any individual has some set of possible alternative behaviors in which he might engage i The mouse may turn left or right ii Start with a list of mutually exclusive alternative behaviors iii Bobby example Bobby s Alternatives 1 S play with others Designate S meaning social 2 A play alone designate A meaning alone b Assumption 2 State of the Individual individual is in some state with respect to the alternative behaviors at any point in time individuals state may be described by a set of probabilities i T maze the mouse has a probability of going left and a probability of going right at any point in time these probabilities add up to 1 Initially going right left probability is 0 5 ii iii As adaptation precedes probabilities might change iv Bobby example Bobby s Current State 1 Probability of S 0 25 2 Probability of A 0 75 c Assumption 3 Alternative Responses of the World The world has a repertoire of possible responses to the behavior of the individual Reponses include giving food water love attention etc Within a specific problem focus on a