Probability and Statistics

Probability is the topic where intuition is least reliable, so it repays being strict about definitions. Two distinctions carry most of the marks. The first is mutually exclusive versus independent — the two are often confused and are in fact almost opposites, since mutually exclusive events cannot happen together while independent events do not affect each other. The second is conditional probability, where the question is not what is likely but what is likely given that something else has already happened; and that shift, formalised in Bayes' theorem, produces answers that regularly surprise people. Statistics is then a short and formulaic appendix.

The Addition and Multiplication Rules

The rules and when each applies
SituationRule
Any two eventsP(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Mutually exclusive eventsP(A ∩ B) = 0, so P(A ∪ B) = P(A) + P(B)
Independent eventsP(A ∩ B) = P(A)·P(B)
Conditional probabilityP(A|B) = P(A ∩ B)/P(B), for P(B) ≠ 0
ComplementP(not A) = 1 − P(A)
"At least one"1 − P(none), almost always the faster route
🎯 Why mutually exclusive and independent are nearly opposite ideas
Mutually exclusive means the two events cannot both occur, so P(A ∩ B) = 0. Independent means knowing that one occurred tells you nothing about the other, so P(A ∩ B) = P(A)P(B). Those are not variations on a theme — they conflict. If two events with non-zero probabilities are mutually exclusive, then learning that A happened tells you B definitely did not, which is the strongest possible dependence. So two events with positive probability cannot be both mutually exclusive and independent, and a question asserting both is describing an impossibility. Testing the two conditions separately, rather than treating them as interchangeable, is what keeps this section reliable.
🧠 Turn "at least one" into one minus "none"
Computing the probability of at least one success directly means adding the cases for exactly one, exactly two, exactly three and so on — a long sum that grows with the number of trials. The complement is a single term: P(at least one) = 1 − P(none). For six dice thrown, the probability of at least one six is 1 − (5/6)⁶, which is one line instead of six. The same move handles "at least two" as 1 − P(none) − P(exactly one), and the general principle is worth carrying: whenever the wording is "at least", check whether the complement is shorter before starting.

Conditional Probability and Bayes' Theorem

The total probability theorem splits an event across a set of exhaustive alternatives: P(A) = ΣP(Eᵢ)P(A|Eᵢ). Bayes' theorem runs it backwards, asking which alternative was responsible given that A occurred: P(Eᵢ|A) = P(Eᵢ)P(A|Eᵢ) / ΣP(Eⱼ)P(A|Eⱼ). Read plainly, Bayes updates a prior belief in the light of evidence, and the denominator is just the total probability of the evidence.

📖 Why a 99 % accurate test can still be wrong most of the time
Suppose a disease affects 1 person in 1000 and a test detects it with 99 % accuracy in both directions. Someone tests positive — how likely is it that they are ill? Intuition says 99 %, and intuition is badly wrong. Out of 100 000 people, about 100 are ill and 99 of them test positive, while 99 900 are healthy and about 999 of them test positive too. So of roughly 1098 positives, only 99 are genuine — about 9 %. The reason is that the disease is rare, so the small false-positive rate applied to a very large healthy group swamps the true positives. This is exactly what Bayes' theorem computes, and it is why the prior matters as much as the accuracy: a question of this shape is testing whether you multiplied by the base rate rather than reading the accuracy as the answer.

Random Variables and Statistics

The binomial distribution and the summary measures
QuantityFormula
Binomial probability of r successesⁿCᵣ pʳ qn−r, with q = 1 − p
Mean of a binomialnp
Variance of a binomialnpq — always less than the mean
Expectation of any random variableE(X) = Σ xᵢpᵢ
Varianceσ² = E(X²) − [E(X)]²
Standard deviationσ = √variance, in the same units as the data
⚠️ Variance is unaffected by shifting the data but scales with the square of a multiplier
Adding a constant to every observation moves the mean by that constant and leaves the variance completely unchanged, because variance measures spread and shifting does not change how spread out the data are. Multiplying every observation by k multiplies the mean by k but the variance by k², since variance is built from squared deviations. So if a data set has mean 10 and variance 4, then adding 5 to each value gives mean 15 and variance 4, while doubling each value gives mean 20 and variance 16. Questions in this shape are common precisely because it is tempting to scale the variance linearly, and standard deviation — being a square root — scales by k rather than k².

Two closing observations. For a binomial distribution the variance npq is always smaller than the mean np, since q < 1, so a question reporting a variance larger than the mean has not described a binomial variable — a quick plausibility check. And the mean, median and mode coincide for a symmetric distribution but separate for a skewed one, with the empirical relation mode ≈ 3(median) − 2(mean) giving a serviceable estimate when only two of the three are known.

Key takeaways

  • Mutually exclusive means P(A ∩ B) = 0; independent means P(A ∩ B) = P(A)P(B) — nearly opposite ideas.
  • Two events of positive probability cannot be both mutually exclusive and independent.
  • For "at least one", compute 1 − P(none) — almost always far shorter than summing the cases.
  • Bayes updates a prior with evidence; the denominator is the total probability of the evidence.
  • For a rare condition, even a highly accurate test yields mostly false positives — the base rate dominates.
  • Binomial: P(r) = ⁿCᵣpʳqn−r, mean np, variance npq — the variance is always below the mean.
  • Shifting data leaves the variance unchanged; scaling by k multiplies the variance by k².
  • Variance is E(X²) − [E(X)]², and standard deviation carries the same units as the data.

Practice questions (8)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. Two events A and B, each of positive probability, are mutually exclusive. Then they are:

    1. Also independent
    2. Necessarily dependent
    3. Independent only if P(A) = P(B)
    4. Impossible to classify
    Show answer

    Answer: B — Necessarily dependent

    Mutual exclusivity gives P(A ∩ B) = 0, while independence would require P(A ∩ B) = P(A)P(B), which is positive when both probabilities are. The two conditions therefore cannot hold together — knowing A occurred tells you B certainly did not, which is the strongest possible dependence. Treating the two terms as interchangeable is the single most common conceptual error in probability.
  2. Six dice are thrown. The probability of getting at least one six is:

    1. 6 × (1/6)
    2. 1 − (5/6)⁶
    3. (1/6)⁶
    4. (5/6)⁶
    Show answer

    Answer: B — 1 − (5/6)⁶

    The complement of "at least one six" is "no six at all", which has probability (5/6)⁶ since each die independently avoids a six with probability 5/6. Subtracting from 1 gives the answer in one line, whereas summing the cases for exactly one through exactly six sixes would take six terms. The first option exceeds 1 for larger numbers of dice, which shows it cannot be a probability at all.
  3. For a binomial distribution with n = 10 and p = 0.4, the mean and variance are:

    1. 4 and 4
    2. 4 and 2.4
    3. 2.4 and 4
    4. 10 and 4
    Show answer

    Answer: B — 4 and 2.4

    The mean is np = 10 × 0.4 = 4 and the variance is npq = 10 × 0.4 × 0.6 = 2.4. Note that the variance is smaller than the mean, which is always true for a binomial since q < 1 — so an option offering a variance larger than the mean can be discarded immediately as not describing a binomial variable at all.
  4. If 5 is added to every observation of a data set, then:

    1. The mean and variance both increase by 5
    2. The mean increases by 5 and the variance is unchanged
    3. The mean is unchanged and the variance increases by 25
    4. Both are unchanged
    Show answer

    Answer: B — The mean increases by 5 and the variance is unchanged

    A shift moves every value together, so the average moves with them while the spread about the average is untouched — variance depends only on deviations from the mean, and those are unchanged. Multiplying instead by k would scale the mean by k and the variance by k², so it is worth keeping the two operations distinct: shifts affect location only, scalings affect both.
  5. A disease affects 1 in 1000 people and a test is 99 % accurate both ways. A person tests positive. The chance they have the disease is closest to:

    1. 99 %
    2. 50 %
    3. 9 %
    4. 1 %
    Show answer

    Answer: C — 9 %

    Out of 100 000 people about 100 are ill, of whom 99 test positive, while 99 900 are healthy and about 999 of those also test positive. So roughly 99 of about 1098 positives are genuine, giving about 9 %. Because the disease is rare, the small false-positive rate applied to a huge healthy population outnumbers the true positives — which is precisely why the prior probability matters as much as the test accuracy.
  6. P(A|B) is defined as:

    1. P(A ∩ B)/P(A)
    2. P(A ∩ B)/P(B), provided P(B) ≠ 0
    3. P(A) + P(B) − P(A ∩ B)
    4. P(A)·P(B)
    Show answer

    Answer: B — P(A ∩ B)/P(B), provided P(B) ≠ 0

    Conditioning on B restricts attention to the part of the sample space where B occurred, so the denominator must be P(B) and the numerator the overlap. Dividing by P(A) instead gives P(B|A), a different quantity — and the two are equal only in special cases, which is exactly the asymmetry Bayes' theorem exists to handle.
  7. The variance of a random variable X is computed as:

    1. E(X²) + [E(X)]²
    2. E(X²) − [E(X)]²
    3. [E(X)]² − E(X²)
    4. E(X) − E(X²)
    Show answer

    Answer: B — E(X²) − [E(X)]²

    Variance is the expected squared deviation from the mean, and expanding that expression gives E(X²) − [E(X)]². The order matters: the third option is the negative of the right answer and would always come out negative, which is impossible for a variance — a useful sign check if the arithmetic ever produces one.
  8. For a binomial distribution, the variance is:

    1. Always greater than the mean
    2. Always less than the mean
    3. Always equal to the mean
    4. Sometimes greater and sometimes less
    Show answer

    Answer: B — Always less than the mean

    The mean is np and the variance npq, and since q = 1 − p is strictly less than 1 for any genuine binomial, the variance is always the smaller of the two. That gives a fast plausibility check on any binomial question: if the stated variance exceeds the stated mean, the distribution described cannot be binomial and the data are inconsistent.