A child's tablet is taken away for an evening each time she hits her brother, and the hitting decreases. This is:

Negative punishment (response cost)

In Tolman and Honzik's (1930) experiment, rats that were first rewarded on day 11 immediately made as few errors as rats rewarded from the start. This demonstrated:

The partial-reinforcement extinction effect

According to Mowrer's two-factor theory, the avoidance response is maintained by:

Negative reinforcement through the reduction of classically conditioned fear

Positive reinforcement through food reward

Punishment of the avoidance response

Observational learning from other animals

Assertion (A): Monkeys given hundreds of different two-choice discrimination problems eventually solve a new problem after a single trial. Reason (R): Lashley showed that maze learning depends on the amount of cortex available rather than on its location. Choose the correct option.

Both A and R are true, but R is not the correct explanation of A

Both A and R are true, and R is the correct explanation of A

Both A and R are true, but R is not the correct explanation of A

Midbrain dopamine neurons fire strongly to an unexpected reward, shift their firing to the cue that predicts the reward as learning proceeds, and decrease their firing when an expected reward is omitted. These cells appear to signal:

The location of the animal in space

Habituation to repeated stimuli

The absolute amount of reward received

Learning II: Instrumental Learning — Phenomena, Paradigms and Theoretical Issues; Reinforcement Variables and Schedules; Behaviour Modification; Latent and Observational Learning; Discrimination Learning; and the Neurophysiology of Learning

The rest of Unit 5's learning block reads: "Instrumental learning: Phenomena, Paradigms and theoretical issues; Reinforcement: Basic variables and schedules; Behaviour modification and its applications. Cognitive approaches in learning: Latent learning, observational learning. Verbal learning and Discrimination learning. Recent trends in learning: Neurophysiology of learning." (Verbal learning is taken with memory in the next chapter.) This chapter sets out the four instrumental paradigms, works each reinforcement schedule with the response pattern it produces, takes up the classic theoretical problems of avoidance and punishment, surveys behaviour-modification techniques, and closes with Tolman, Bandura, discrimination learning and what the brain does when it learns.

1. Instrumental learning: paradigms and phenomena

In instrumental (operant) learning the response is instrumental in producing its consequence. Early research used discrete trials (Thorndike's puzzle box, runways, mazes); Skinner's operant chamber allowed free-operant responding measured as a rate on a cumulative record. The paradigms are defined by whether a stimulus is added or removed and whether the response then increases or decreases.

The four instrumental paradigms
Consequence	Response increases	Response decreases
Stimulus added	Positive reinforcement (reward training)	Positive punishment (an aversive stimulus follows the response)
Stimulus removed	Negative reinforcement — escape (ending an aversive stimulus) and avoidance (preventing it)	Negative punishment — omission training, time-out, response cost

Shaping by reinforcing successive approximations, and chaining of responses into sequences (each step's stimulus reinforcing the one before).
Extinction (often preceded by an extinction burst) and the partial-reinforcement extinction effect (Humphreys, 1939): responses learned under intermittent reinforcement resist extinction far longer than those learned under continuous reinforcement.
Stimulus control: a discriminative stimulus (Sᴰ) signals that responding will be reinforced, and S-delta that it will not.
Superstitious behaviour (Skinner, 1948: pigeons fed at fixed times repeated whatever they happened to be doing), instinctive drift (Breland and Breland, 1961: trained animals drift back toward species-typical behaviour), contrast effects (Crespi) and learned helplessness (Seligman and Maier, 1967: dogs exposed to inescapable shock later failed to escape escapable shock).

2. Reinforcement: basic variables and schedules

The basic variables of reinforcement are its amount (magnitude), delay (immediate reinforcement is far more effective), quality, and schedule. Primary reinforcers satisfy biological needs; secondary (conditioned) reinforcers acquire value by association, and generalised reinforcers such as money or tokens are backed by many primary ones. The Premack principle (1959) states that a more probable behaviour reinforces a less probable one ("finish your homework, then you may play"); Timberlake and Allison's response-deprivation hypothesis generalises it — any behaviour restricted below its baseline level becomes a reinforcer.

Schedules of reinforcement and the response patterns they produce
Schedule	Rule	Response pattern	Everyday example
Continuous (CRF)	Every response reinforced	Fast acquisition, fast extinction	A vending machine
Fixed ratio (FR)	After every n-th response	High rate with a post-reinforcement pause ("break and run")	Piece-rate pay
Variable ratio (VR)	After an unpredictable number of responses around an average	The highest, steadiest rate, no pause; the most resistant to extinction	Slot machines, lotteries
Fixed interval (FI)	First response after a fixed time	The scallop: little responding after reinforcement, accelerating as the interval ends	Studying just before a scheduled test; a monthly salary
Variable interval (VI)	First response after an unpredictable time around an average	A moderate, steady rate, resistant to extinction	Checking for messages; pop quizzes

Worked. On an FR-5 schedule, 30 responses earn 30 ÷ 5 = 6 reinforcers. On a VI-60-second schedule, reinforcement becomes available on average once a minute, however fast the animal responds, so rate hardly changes earnings. With two keys on concurrent schedules, Herrnstein's matching law (1961) states that the proportion of responses on a key matches the proportion of reinforcers it provides: B₁ ÷ (B₁ + B₂) = R₁ ÷ (R₁ + R₂). If key 1 pays 40 reinforcers an hour and key 2 pays 10, the pigeon makes 40 ÷ 50 = 80 per cent of its responses on key 1. Differential-reinforcement schedules — DRL (low rates), DRH (high rates), DRO (any behaviour other than the target) — shape rate itself.

🧠 Ratio = rate; variable = steady

Ratio schedules pay for work, so they produce higher rates than interval schedules, which pay for time. Variable schedules remove predictability, so they abolish pauses and resist extinction. Hence VR is highest and steadiest, and FI shows the scallop.

3. Theoretical issues, and behaviour modification

What makes avoidance persist? Mowrer's two-factor theory (1947): fear of a warning signal is acquired by classical conditioning, and the avoidance response is then negatively reinforced by fear reduction. The problem is that well-practised avoidance continues long after signs of fear disappear; one-factor accounts (Herrnstein; Sidman avoidance without any warning signal) treat reduced shock frequency itself as the reinforcer.
Does punishment work? Estes (1944) found that mild punishment only temporarily suppressed responding. Punishment is effective when it is immediate, consistent and sufficiently intense from the start, and when an alternative response is reinforced; its side effects are fear, aggression and avoidance of the punisher.
What is a reinforcer? Drive reduction (Hull), a transsituational stimulus (Meehl), a more probable response (Premack) or a restricted response (response deprivation); and is reinforcement needed for learning at all, or only for performance (Tolman)?

Behaviour modification applies learning principles to change behaviour; applied behaviour analysis begins with a functional analysis of antecedents, behaviour and consequences (the ABC model). Operant techniques include positive reinforcement and shaping, prompting and fading, chaining, the token economy (Ayllon and Azrin, 1968, on psychiatric wards), contingency contracts, differential reinforcement of other, alternative or incompatible behaviour (DRO, DRA, DRI), extinction, time-out, response cost and overcorrection. Respondent techniques include Wolpe's systematic desensitisation, flooding and aversion therapy. Applications range from intensive early intervention for autism (Lovaas, 1987) and classroom management to organisational behaviour management, health behaviour, self-management and biofeedback.

4. Cognitive approaches: latent and observational learning; discrimination learning

Latent learning: Tolman and Honzik (1930) ran three groups of rats in a maze — always rewarded, never rewarded, and rewarded only from day 11. The last group's errors dropped at once to the level of the always-rewarded group, showing that they had learned the maze without reward and simply had no reason to show it. Tolman concluded that rats form cognitive maps (1948) and that reinforcement affects performance rather than learning. Observational learning: in Bandura's Bobo doll studies (1961) children who watched an adult attack a doll imitated the aggression; in 1965, children who saw the model punished imitated less — until offered a reward, when all groups reproduced the behaviour equally, again separating learning from performance. Bandura named four processes: attention, retention, motor reproduction and motivation, with vicarious reinforcement influencing the last.

Discrimination learning is learning to respond to one stimulus (S+) and not another (S−). Spence's continuity theory held that each reinforced trial adds excitatory strength to S+ and each non-reinforced trial inhibitory strength to S−, and the net gradients explain transposition (animals trained to choose the larger of two squares choose the larger of a new pair) and the peak shift (Hanson, 1959: after discrimination training, peak responding moves away from S−). Krechevsky and Lashley's non-continuity view held that animals test hypotheses and learn only about the dimension they are attending to. Harlow (1949) showed learning sets — monkeys given hundreds of different discrimination problems came to solve a new one in a single trial ("learning to learn") — and Terrace (1963) produced errorless discrimination learning by introducing S− faintly and fading it in.

5. Recent trends: the neurophysiology of learning

The neurophysiology of learning
Finding	What it showed
Lashley's search for the engram (1929)	Maze learning in rats was impaired in proportion to how much cortex was removed, not where: mass action and equipotentiality.
Hebb (1949); long-term potentiation	Cells that fire together wire together; LTP in the hippocampus (Bliss and Lømo, 1973) depends on glutamate acting at NMDA receptors, which need both presynaptic release and postsynaptic depolarisation.
Kandel's Aplysia	Habituation and sensitisation of the sea slug's gill-withdrawal reflex arise from changes in transmitter release at identified synapses; long-term memory requires protein synthesis and new synapses (Nobel Prize, 2000).
Circuits for different kinds of learning	The cerebellum (interpositus nucleus) for eyeblink conditioning (Thompson); the amygdala for fear conditioning; the striatum for habits; the hippocampus for spatial learning — place cells (O'Keefe) and the Morris water maze.
Dopamine prediction error	Midbrain dopamine neurons fire to unexpected reward, shift their firing to the predicting cue as learning proceeds, and dip when an expected reward is omitted (Schultz, 1997) — the neural counterpart of the Rescorla–Wagner error term.

🎯 Why this matters for theory

The dopamine finding shows that the brain computes something very like prediction error, which supports the cognitive, information-based reading of conditioning; and the separate circuits for habits (striatum) and maps (hippocampus) give a biological basis to the old contest between S–R theorists and Tolman — both were describing real systems.

Key takeaways

Positive and negative reinforcement increase behaviour; positive and negative punishment decrease it; escape and avoidance are forms of negative reinforcement.
FR gives a high rate with post-reinforcement pauses, VR the highest steady rate and most resistance to extinction, FI the scallop, and VI a moderate steady rate; the matching law predicts choice between concurrent schedules.
Mowrer's two-factor theory explains avoidance through conditioned fear and fear reduction; punishment works when immediate, consistent and paired with reinforcing an alternative; the Premack principle and response deprivation define reinforcers relationally.
Tolman's latent learning and Bandura's observational learning separate learning from performance; discrimination learning shows transposition, peak shift, learning sets and errorless learning.
The neurophysiology of learning runs from Lashley's mass action through Hebb, LTP and NMDA receptors and Kandel's Aplysia to dedicated circuits and dopamine prediction-error signals.

Practice questions (10)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

Which schedule of reinforcement typically produces the highest, steadiest rate of responding and the greatest resistance to extinction?
1. Fixed ratio
2. Variable ratio
3. Fixed interval
4. Continuous reinforcement
Show answer
Answer: B — Variable ratio
Variable-ratio schedules reward work unpredictably, so there is no pause and every response might pay — as with gambling. Fixed ratio produces post-reinforcement pauses, fixed interval the scallop, and continuous reinforcement extinguishes fastest.
A cumulative record shows little responding just after each reinforcer, with the rate accelerating as the time for the next reinforcer approaches. The schedule is most likely:
1. Fixed interval
2. Variable interval
3. Continuous reinforcement
4. Variable ratio
Show answer
Answer: A — Fixed interval
This is the fixed-interval scallop: since no response can be reinforced until the interval elapses, the animal learns to wait and then speeds up. Variable schedules produce steady rates without such patterning.
A pigeon chooses between two keys on concurrent schedules. Key 1 provides 40 reinforcers per hour and key 2 provides 10. According to the matching law, what proportion of its responses should be made on key 1? Type a number.
Numerical answer — type the value.
Show answer
Answer: 0.8
B₁ ÷ (B₁ + B₂) = R₁ ÷ (R₁ + R₂) = 40 ÷ (40 + 10) = 0.8, so about 80 per cent of responses go to key 1.
On a fixed-ratio 5 (FR-5) schedule, how many reinforcers does a rat earn for 30 responses? Type a number.
Numerical answer — type the value.
Show answer
Answer: 6
FR-5 delivers one reinforcer after every fifth response, so 30 ÷ 5 = 6 reinforcers.
A child's tablet is taken away for an evening each time she hits her brother, and the hitting decreases. This is:
1. Positive punishment
2. Extinction
3. Negative reinforcement
4. Negative punishment (response cost)
Show answer
Answer: D — Negative punishment (response cost)
A desired stimulus is removed and the behaviour decreases, which defines negative punishment. Negative reinforcement removes an aversive stimulus and increases behaviour; positive punishment adds an aversive stimulus; extinction withholds the reinforcer that maintained the behaviour.
In Tolman and Honzik's (1930) experiment, rats that were first rewarded on day 11 immediately made as few errors as rats rewarded from the start. This demonstrated:
1. Instinctive drift
2. Learned helplessness
3. The partial-reinforcement extinction effect
4. Latent learning
Show answer
Answer: D — Latent learning
The sudden improvement shows that the maze had been learned during the unrewarded days and was only revealed when reward gave a reason to perform — latent learning, which Tolman used to argue that reinforcement affects performance rather than learning.
Which of the following are among the four processes Bandura identified in observational learning? Select all that apply.
1. Attention
2. Motivation
3. Drive reduction
4. Retention
Show answer
Answer: A — Attention; B — Motivation; D — Retention
Bandura's four processes are attention, retention, motor reproduction and motivation. Drive reduction is Hull's account of reinforcement, not part of Bandura's model.
According to Mowrer's two-factor theory, the avoidance response is maintained by:
1. Negative reinforcement through the reduction of classically conditioned fear
2. Positive reinforcement through food reward
3. Punishment of the avoidance response
4. Observational learning from other animals
Show answer
Answer: A — Negative reinforcement through the reduction of classically conditioned fear
Factor one: the warning signal comes to elicit fear through classical conditioning. Factor two: the avoidance response removes the signal, reduces fear and is thereby negatively reinforced.
Assertion (A): Monkeys given hundreds of different two-choice discrimination problems eventually solve a new problem after a single trial. Reason (R): Lashley showed that maze learning depends on the amount of cortex available rather than on its location. Choose the correct option.
1. Both A and R are true, and R is the correct explanation of A
2. Both A and R are true, but R is not the correct explanation of A
3. A is true, but R is false
4. A is false, but R is true
Show answer
Answer: B — Both A and R are true, but R is not the correct explanation of A
Both are true but unrelated. A describes Harlow's learning sets ("learning to learn"); R describes Lashley's mass action and equipotentiality, which says nothing about why experience with many problems speeds the solving of new ones.
Midbrain dopamine neurons fire strongly to an unexpected reward, shift their firing to the cue that predicts the reward as learning proceeds, and decrease their firing when an expected reward is omitted. These cells appear to signal:
1. The location of the animal in space
2. Reward prediction error
3. Habituation to repeated stimuli
4. The absolute amount of reward received
Show answer
Answer: B — Reward prediction error
Schultz's findings match the difference between reward received and reward expected — the prediction error at the heart of the Rescorla–Wagner model. Location is signalled by hippocampal place cells.