I. Basic behavior shaping proceduresA. Types of reinforcementi. Secondary reinforcementB. Shaping complex behaviorsII. ExtinctionIII. PunishmentIV. Escape-AvoidanceV. ApplicationsLearning by Trial and Error and Feedback:- Law of Effect: behaviors followed by a “satisfying” effect become more likely to re-occur, those followed by a “discomforting” effect become less likely to re-occurThorndike’s puzzle pox (p.108) – hungry cat inside the box for the first time, food right outside- Simulation of Pavlonian and operant conditioning; behavior of subject is controlled by a set of variables and equations that have developed out of conditioning research and theory- highly realisticBehavior engineering problem (based on “voluntary” nervous system)get the subject to press the lever (bar) on the side of the cage by shaping.Initial conditions:(1) Subject has already learned that food sometimes appears in the food cup (‘magazine training’). Also, the dispenser makes a noise when food is delivered or when the bar is pressed. Creates an association between the noise and food (association by contiguity; classical conditioning).(2) Subject has been food-deprived for 24 hrs.(3) The operant level (current rate) of the desired behavior (the bar press) is very low or zero.- Begin a program of reinforcing successive approximations to the final behavior.Example: subject near food cup turns head towards cup  deliver the food reinforcer immediately. Delayed reinforcement is ineffective and counter-productive; require closer and closer approximations, reinforcing each one (“you’re getting warmer”).Eventually: subject puts paw on bar with sufficient force to depress lever andoperate food-dispensing circuit- known as “magazine training”Possible extension of behavior engineering problem: add some stimulus controlStimulus control: bar press response is now only reinforced if light is on (discriminative stimulus) – light itself has no reinforcing properties at the start- Get rid of the behavior through extinction: enable the stimulus and no reward occurs, continue this until the subject is used to itEx: rat gets no food when the lever is pressed anymore- Partial reinforcement: behavior that’s intense and durable; 1 reinforcement for every 5 responsesPrinciple: behavior is reinforced intermittently and unpredictably, more resistant to extinction than behavior that is continually reinforced- VR = variable ratio; FR = fixed ratioEx: FR-1 = 1 reinforcer for 1 response; VR-3, 1 reward for every 3 responses- Adaptive significance: persistence in an unpredictable, frequently disappointing worldLaw of Effect rewritten- Behaviors with satisfying consequences are strengthened; behaviors with annoying consequences are weakenedProblem: “satisfying” and “annoying” are subjective concepts, inferred from observable behavior- Reinforcer: event that increases the probability of the behaviors ongoing just prior to the eventBehavior is selected (strengthened, weakened) by its consequencesBehavior is instrumental in leading to reinforcers; operates on environment- Reward: satisfying or annoying effect that a stimulus event has on a subject;Shown to be based on neurochemical processes in the brainSecondary (Learned) Reinforcement:- Initially neutral stimulus or event can become a reinforcer if it occurs along with an established reinforcerEx: light in a Skinner box serving as a discriminative stimulusAnimal will learn a new behavior whose only direct consequence is to turn the light onExtends the range of stimuli/events that can serve as reinforcers; originally neutral (non-reinforcing) events can become reinforcers by this processEnables learning of long sequences of behavior (chaining), when real payoff (primary reinforcer) occurs only the end of the sequence- Each behavior in the sequence is reinforced by gaining the opportunity to perform the next response , finally leading to the primary reinforcementLike sub goals, hill-climbing, dolphins doing water actsRandom Reinforcement and “Superstitious” Behavior:Suppose that reinforcement occurs at random – it’s not contingent on any behavior. Nothing the subject does has any true instrumental value. What effect will this have on the animal’s behavior?- Analysis:1. Random occurrence of reinforcement follows some behavior X  behavior X is selected/strengthened.2. The next random occurrence of reinforcement will probably coincide with some other behavior Y  behavior Y is selected/strengthened.3. Over time, X and Y (& Z & ….) will be partially reinforced, at random and unpredictably.4. Result: the subject displays behaviors X and Y… with high frequency; difficult to extinguish the behaviors, behaviors can seem like a bizarre ritual.Does this explain superstitions and rituals of gamblers?Reinforcers:- PRIMARY REINFORCERS/REWARDS: innate, related to basic survival needsEx: food, water, love, statusSECONDARY (LEARNED) REINFORCERS/REWARDS: learned by association with primary reinforcersEx: negative reinforcers (electric shock, intense heat, failure, rejection, criticism)* Social acceptance and rejection is a basic primary reinforcerEx: sexually rejected flies turn to boozeNegative reinforcement example: Rat presses bar  Aversive stimulus (shock) is removed  Bar presses increasesPunishment example: rat presses bar  shock is delivered  bar pressing decreasesPunishment:- Response-contingent delivery of an aversive (noxious) stimulus.‘Positive punishment’ (aversive punishment).“If you do this right, no electric shock”- Removal or withholding of a positively reinforcing stimulus.‘Negative punishment’ (response cost).“No dessert if you keep doing this”- Punishers can be primary (unlearned, such as shock) or secondary (learned; such as verbal threats).Effects/Effectiveness of Punishment- Suppression of ongoing behavior. May be temporary; can be long-lasting.- Punishment can be effective in controlling behavior if it is strong, swift, contingent.Issues with punishment:1) Doesn’t directly teach a new behavior2) Many elicit emotional arousal, fear, anger, aggression3) Response to punishment may be incompatible with new learning4) Strong punishment: unpleasant to administer, legal and ethical constraints5) Not the same as extinction of the undesirable responseEscape and Avoidance behaviorESCAPE: perform an operant response that gets you away from an ongoing punishing stimulus.AVOIDANCE: perform an operant