A single perceptron fails to learn the XOR function chiefly because:

XOR's four points are not linearly separable, and a single perceptron is a linear classifier

XOR has more than four possible inputs

Perceptrons cannot process binary inputs at all

The perceptron learning rule does not exist

The perceptron convergence theorem guarantees the perceptron learning rule finds separating weights in finite steps:

For every possible dataset, without exception

Only for datasets with exactly two points

If every layer of a multi-layer perceptron used a purely linear activation function, the network would:

Become strictly more powerful than any non-linear network

Be unable to compute anything at all

In supervised learning, a neural network is trained using:

Input-output pairs with the correct target output labelled

Only inputs, with no target output at all

A scalar reward received after each action, with no labelled examples

A self-organizing map (SOM), compared to a Hopfield network, is best described as solving:

Unsupervised, topology-preserving dimensionality reduction, not associative memory/recall

Exactly the same problem as a Hopfield network

Nothing related to neural networks at all

Which of the following are true of a Hopfield network? (More than one option may be correct.)

It is a fully-connected, recurrent, single-layer network

It can recall a complete stored pattern from a corrupted or partial version of it

It settles iteratively into a stable state rather than computing a single feedforward pass

It is a fully-connected, recurrent, single-layer network

It can recall a complete stored pattern from a corrupted or partial version of it

It settles iteratively into a stable state rather than computing a single feedforward pass

It requires labelled target outputs for every training example, exactly like an MLP

A self-organizing map updates not only the best-matching unit's weights but also its neighbours' weights, with the neighbourhood shrinking over training. This is chiefly why it:

Preserves topology, so similar inputs map to nearby neurons on the grid

Guarantees every neuron ends up with identical weights

Has nothing to do with the map's final structure

Only affects a single neuron in the entire network

Artificial Neural Networks

This chapter closes Unit 10 and the whole subject with artificial neural networks — the single/multi-layer perceptron, the learning paradigms they are trained under, self-organizing maps and the Hopfield network — grounded correctly rather than hand-waved, exactly as this unit's brief asks.

The Single Perceptron and Its Limits

A single perceptron computes a weighted sum of its inputs plus a BIAS term, then applies an ACTIVATION function (classically a step function) to produce a binary output: output = step(Σ wᵢxᵢ + b). Geometrically, this is a LINEAR classifier — it can only correctly separate two classes of input that are LINEARLY SEPARABLE (a single straight line/hyperplane can divide them). The PERCEPTRON LEARNING RULE adjusts weights after each misclassified example: wᵢ ← wᵢ + η(target − output)xᵢ, where η is the learning rate — this rule is guaranteed (the PERCEPTRON CONVERGENCE THEOREM) to find separating weights in a finite number of steps IF AND ONLY IF the data is actually linearly separable; if it is not, the rule never converges, oscillating forever.

⚠️ The single perceptron cannot learn XOR — the historical result that stalled the field

XOR's four points — (0,0)→0, (0,1)→1, (1,0)→1, (1,1)→0 — are NOT linearly separable: any single straight line drawn in the input plane leaves at least one point on the wrong side, since the two 'output 1' points (0,1) and (1,0) are diagonally opposite the two 'output 0' points (0,0) and (1,1). Minsky and Papert's 1969 book proved this limitation rigorously, and its impact (a widely publicised limitation of the single-layer model, arriving alongside reduced funding) is the standard historical reference for why neural network research slowed for over a decade — resolved only once MULTI-LAYER networks (with a hidden layer) and a working training algorithm for them (backpropagation) were developed.

The Multi-Layer Perceptron and Backpropagation

A multi-layer perceptron (MLP) stacks an input layer, one or more HIDDEN layers, and an output layer, with every neuron in one layer connected to every neuron in the next (feedforward, fully connected). Crucially, each hidden-layer neuron uses a NON-LINEAR activation function (sigmoid, tanh, or ReLU in modern practice — never a linear one) — if every layer's activation were linear, the entire multi-layer stack would collapse mathematically into an equivalent SINGLE linear layer, gaining nothing over a single perceptron; it is the non-linearity at each hidden layer that actually lets an MLP represent non-linearly-separable functions like XOR. Backpropagation is the algorithm that trains an MLP: it computes the output error, then propagates that error BACKWARD through the network layer by layer using the chain rule of calculus, computing how much each individual weight contributed to the error (its gradient), and updates every weight by GRADIENT DESCENT (moving it a small step in the direction that reduces the error) — this is what makes training a network with many hidden layers computationally tractable, rather than needing to guess weights or search them exhaustively.

Supervised, Unsupervised and Reinforcement Learning

The three learning paradigms named in this sub-item
Paradigm	What the network is given	Standard ANN example
Supervised	Input-output pairs with the correct (target) output labelled	MLP trained by backpropagation
Unsupervised	Only inputs, no labelled target — the network must discover structure on its own	Self-organizing map (SOM)
Reinforcement	No labelled examples at all — only a scalar REWARD signal received after acting, and the network must learn a policy (which action to take) that maximises cumulative reward over time through trial and error	Not itself named for an ANN architecture in this syllabus item, but the paradigm applies broadly across AI

🎯 Why this is the same axis clustering/classification used

This three-way split is the exact same conceptual axis already introduced for data mining in Unit 4 — classification needed labelled training data (supervised), clustering did not (unsupervised) — with reinforcement learning added as a third category defined by neither labelled examples NOR unlabelled structure-discovery, but by a delayed, scalar feedback signal earned through interaction with an environment. Recognising this as one recurring axis, rather than three unrelated vocabularies in two different units, is exactly the kind of connection this platform's chapters are written to surface.

Self-Organizing Maps and the Hopfield Network

A Self-Organizing Map (SOM, Kohonen network) trains UNSUPERVISED, projecting high-dimensional input data onto a low-dimensional (typically 2D) grid of neurons while preserving TOPOLOGICAL relationships — inputs that are similar in the original high-dimensional space are mapped to neurons that are near each other on the grid. Training repeatedly finds the BEST-MATCHING UNIT (the grid neuron whose weight vector is closest to the current input) and updates that neuron's weights AND its neighbours' weights to move them closer to the input, with the neighbourhood radius and update magnitude both shrinking over training — this neighbourhood-based update is precisely what produces the map's topology-preserving property, unlike ordinary clustering which has no notion of which clusters are 'near' each other.
A Hopfield network is a fully-connected, single-layer RECURRENT network (every neuron connects to every other, symmetric weights, no self-connections) used as ASSOCIATIVE (content-addressable) MEMORY: it can be trained (via a Hebbian-style rule) to store a set of patterns as STABLE STATES of its own dynamics, and when given a corrupted or partial version of a stored pattern as a starting state, its neurons update iteratively until the whole network settles into the nearest stored pattern — RECALLING the complete original from a noisy or partial cue, unlike an MLP, which computes a single feedforward pass with no memory of a settling process at all.

⚠️ SOM and Hopfield solve genuinely different problems — do not conflate them

A SOM's job is DIMENSIONALITY REDUCTION/visualisation with topology preservation — it answers 'which inputs are similar to each other, laid out on a map I can look at'. A Hopfield network's job is PATTERN COMPLETION/associative recall — it answers 'given a corrupted version of something I stored, what was the original'. Both are unsupervised in the sense of needing no labelled target, and both use iterative update rules, but they solve unrelated problems and neither is a substitute for the other — a question naming one architecture and asking about the other's job is testing exactly this distinction.

Key takeaways

A single perceptron is a linear classifier, provably unable to learn a non-linearly-separable function like XOR; the perceptron learning rule converges only if the data is linearly separable.
An MLP needs non-linear hidden-layer activation, or it collapses mathematically to a single linear layer; backpropagation trains it via the chain rule and gradient descent.
Supervised (labelled), unsupervised (unlabelled) and reinforcement (delayed reward) are the three learning paradigms — the same supervised/unsupervised axis as classification versus clustering.
A SOM does unsupervised, topology-preserving dimensionality reduction; a Hopfield network does associative memory/pattern completion — genuinely different jobs.
The single-perceptron/XOR limitation historically stalled neural network research until multi-layer networks and backpropagation resolved it.

Practice questions (8)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

A single perceptron fails to learn the XOR function chiefly because:
1. XOR's four points are not linearly separable, and a single perceptron is a linear classifier
2. XOR has more than four possible inputs
3. Perceptrons cannot process binary inputs at all
4. The perceptron learning rule does not exist
Show answer
Answer: A — XOR's four points are not linearly separable, and a single perceptron is a linear classifier
No single straight line can separate XOR's two output-1 points from its two output-0 points, and since a single perceptron can only represent a linear decision boundary, it fundamentally cannot represent this function — this is exactly what Minsky and Papert proved rigorously in 1969.
The perceptron convergence theorem guarantees the perceptron learning rule finds separating weights in finite steps:
1. If and only if the data is actually linearly separable
2. For every possible dataset, without exception
3. Only for datasets with exactly two points
4. Only when no bias term is used
Show answer
Answer: A — If and only if the data is actually linearly separable
If the data is not linearly separable (like XOR), the perceptron learning rule never converges and oscillates indefinitely — convergence is conditional on linear separability, not universal.
If every layer of a multi-layer perceptron used a purely linear activation function, the network would:
1. Mathematically collapse into an equivalent single linear layer, gaining nothing over a single perceptron
2. Become strictly more powerful than any non-linear network
3. Be unable to compute anything at all
4. Require no training whatsoever
Show answer
Answer: A — Mathematically collapse into an equivalent single linear layer, gaining nothing over a single perceptron
A composition of purely linear functions is itself linear, so stacking linear layers with no non-linearity adds depth without adding any representational power beyond a single linear layer — the non-linearity at hidden layers is what actually enables representing functions like XOR.
In supervised learning, a neural network is trained using:
1. Input-output pairs with the correct target output labelled
2. Only inputs, with no target output at all
3. A scalar reward received after each action, with no labelled examples
4. No data at all
Show answer
Answer: A — Input-output pairs with the correct target output labelled
Supervised learning needs labelled target outputs to compute an error signal for backpropagation; unsupervised learning uses only inputs (as in a SOM), and reinforcement learning uses a delayed scalar reward instead of labelled examples.
A self-organizing map (SOM), compared to a Hopfield network, is best described as solving:
1. Unsupervised, topology-preserving dimensionality reduction, not associative memory/recall
2. Exactly the same problem as a Hopfield network
3. Supervised classification only
4. Nothing related to neural networks at all
Show answer
Answer: A — Unsupervised, topology-preserving dimensionality reduction, not associative memory/recall
A SOM projects high-dimensional data onto a low-dimensional grid while preserving similarity relationships, answering a genuinely different question from a Hopfield network's pattern-completion/associative-recall job — the two should never be conflated.
Which of the following are true of a Hopfield network? (More than one option may be correct.)
1. It is a fully-connected, recurrent, single-layer network
2. It can recall a complete stored pattern from a corrupted or partial version of it
3. It settles iteratively into a stable state rather than computing a single feedforward pass
4. It requires labelled target outputs for every training example, exactly like an MLP
Show answer
Answer: A — It is a fully-connected, recurrent, single-layer network; B — It can recall a complete stored pattern from a corrupted or partial version of it; C — It settles iteratively into a stable state rather than computing a single feedforward pass
A Hopfield network is trained to store patterns as stable states via a Hebbian-style rule, not via labelled input-output supervision in the MLP sense — its recall by iterative settling toward a stable state is exactly what distinguishes it from a feedforward MLP.
Name the algorithm that trains a multi-layer perceptron by propagating the output error backward through the network using the chain rule, then applying gradient descent to each weight.
Numerical answer — type the value.
Show answer
Answer: backpropagation
Backpropagation is what made training networks with hidden layers computationally practical, resolving the historical single-perceptron/XOR limitation once combined with a multi-layer architecture.
A self-organizing map updates not only the best-matching unit's weights but also its neighbours' weights, with the neighbourhood shrinking over training. This is chiefly why it:
1. Preserves topology, so similar inputs map to nearby neurons on the grid
2. Guarantees every neuron ends up with identical weights
3. Has nothing to do with the map's final structure
4. Only affects a single neuron in the entire network
Show answer
Answer: A — Preserves topology, so similar inputs map to nearby neurons on the grid
Updating a neighbourhood rather than only the single best-matching neuron is precisely the mechanism that produces the SOM's topology-preserving property, unlike ordinary clustering which has no notion of clusters being 'near' each other.