Conditioning and Learning Test 3 Part 2 7 23 15 Actions and Habits Focus on Operant Learning motivated behavior with the Basal Ganglia Concept of Instrumental Behavior or Operant Learning o This term is used when referring to the study of how behavior is modified by the outcome it produces There are 2 Phases Action and Habit phases Action is the first learning Habit is when that learning lasts sound familiar o Behaviors are viewed as instruments tools to change or modify the environment around the animal What we always have to keep in mind when looking at this though is that there may be competence by not comprehension due to the learning performance distinction we know that behavior does not necessarily ensure that learning alone occurred This is a similar idea that just because the animal can do the task does not at all mean it can understand what it is doing and what the task means Ex Rat Basketball video learning objective 3 Task could be used to make the rats look smart but obviously they do not understand the game of basketball this can be seen when they try to eat the ball which is associated with the reward gets mixed up since they are only working for a reward Resulting behavior looks intelligent but they don t understand what they are doing comprehension is not necessary for the behavior o Animals can learn extremely complex sequences of behavior as long as the outcome satisfies the underlying need state otherwise they won t try to learn the task Where did this concept start o Though we attribute operant learning to B F Skinner we have already learned that Thorndike had the first experimental evidence He searched for proof that cats could not comprehend their actions they were not as smart as humans though they still developed initial associative learning So why do we think of Skinner as the first It was all about how the outcome was measured Thorndike measured time of escape which did not really target the behavior Skinner measured it based on frequency so he made it quantifiable o Skinner s box was still not complete as it measured the outcome rather than the behavior itself neither Thorndike nor Skinner did the full job there was not enough information to tell what the behavior meant So skinner went a little further and said that the two things necessary for the brain to engage in operant behavior were Susceptibility to reinforcement by certain kinds of consequences evaluation of outcomes basically Supply of behavior less specifically committed to eliciting releasing stimuli trial and error These two aspects are known as patterned and un patterned behavior respectively also thought of as behavior under external control and behavior under internal control These formed that basis for trial and error learning The animal has to be able to try out various behaviors to reach the desired goal that goal is put in motion by an internal need state that needs to be fixed The basal ganglia plays a role in both but more on that later Operant Theories for why this Behavior Occurs and What Happens o Thorndike s Law of Effect initial study of instrumental behavior he felt that the Brain was one big collection of stimulus response S R connections and when one of those activated connections led to a rewarding outcome then the connection is strengthened Kind of sounds like LTP that we have all of the connections we ever need they just need to be strengthened or weakened over time The accompanying graph shows that as there were more trials time of escape did decrease there was a lot of variability in time until farther into the experiment All the responses of the animal were elicited by the stimulus situation or the surroundings it was entirely dependent on the box and the task rather than any animal comprehension He wanted a solution that did not require consciousness at all did not want to anthropomorphize at all His law can also be thought of as a primitive version of Hebb s Rule stimulate A to get B going or stimulate the animal to get a strengthened response but it did not require a goal The outcomes produced by the behavior adapt the animal to the situation by strengthening weakening S R connections o Thorndike felt that the animal brain was a house for already created yet untapped reflexes all possible outcomes are already connected to situations these are habits o This was challenged by E C Tolman a father of cognitive psych who felt that mentalizing with the animal was ok and necessary for operant learning He theorized that Operant Learning required that internal goal state and animals learn a three term association to satisfy it they need to comprehend Instrumental behaviors are organized around goals and mediated by experiences associations His format was such the animal learned an association between a stimulus S1 that precedes a response R and that response has a stimulus outcome S2 O in other words a connection is made from an action response to the actions and outcome of the response The goal state is necessary for this without it the response won t occur because the animal won t care enough to do anything when the stimulus is presented The positive outcome will cause the animal to continue to use that strategy after awhile Let s Compare the Two Theories based on Thorndike s Cat Puzzle Box box with the ring that opens it when pulled o Thorndike ring S R connection activated pull ring the animal does not have to want to get out of the box they are compelled to do various actions regardless o Tolman ring S1 pull ring R door opens S2 O animal has to want to get out of the box and there is an understanding that the ring does the trick o They are actually both right in ways It depends on which phase talked about early on of operant learning you re in each theory applies at a different time In the Action Learning Phase you are forming an expectancy and there is an internal need state to satisfy Action Tolman In the Habit Phase once you have really learned the behavior sequence with a rewarding outcome the goal state is not as important and the stimulus controls the behavior Habit Thorndike So We Know the Distinction exists but how do we tell if the behavior in question is still an action or has become a habit Where can we draw a line and how o Reward Devaluation Strategy a satiation method that devalues the reward and makes it less attractive to the animal It focuses on changing the value of the outcome from solving the task The associated diagram shows a monkey undergoing