Probability and Statistics
Conditional Probability and Bayes' Theorem
The total probability theorem splits an event across a set of exhaustive alternatives: P(A) = ΣP(Eᵢ)P(A|Eᵢ). Bayes' theorem runs it backwards, asking which alternative was responsible given that A occurred: P(Eᵢ|A) = P(Eᵢ)P(A|Eᵢ) / ΣP(Eⱼ)P(A|Eⱼ). Read plainly, Bayes updates a prior belief in the light of evidence, and the denominator is just the total probability of the evidence.
Random Variables and Statistics
| Quantity | Formula |
|---|---|
| Binomial probability of r successes | ⁿCᵣ pʳ qn−r, with q = 1 − p |
| Mean of a binomial | np |
| Variance of a binomial | npq — always less than the mean |
| Expectation of any random variable | E(X) = Σ xᵢpᵢ |
| Variance | σ² = E(X²) − [E(X)]² |
| Standard deviation | σ = √variance, in the same units as the data |
Two closing observations. For a binomial distribution the variance npq is always smaller than the mean np, since q < 1, so a question reporting a variance larger than the mean has not described a binomial variable — a quick plausibility check. And the mean, median and mode coincide for a symmetric distribution but separate for a skewed one, with the empirical relation mode ≈ 3(median) − 2(mean) giving a serviceable estimate when only two of the three are known.
Key takeaways
- Mutually exclusive means P(A ∩ B) = 0; independent means P(A ∩ B) = P(A)P(B) — nearly opposite ideas.
- Two events of positive probability cannot be both mutually exclusive and independent.
- For "at least one", compute 1 − P(none) — almost always far shorter than summing the cases.
- Bayes updates a prior with evidence; the denominator is the total probability of the evidence.
- For a rare condition, even a highly accurate test yields mostly false positives — the base rate dominates.
- Binomial: P(r) = ⁿCᵣpʳqn−r, mean np, variance npq — the variance is always below the mean.
- Shifting data leaves the variance unchanged; scaling by k multiplies the variance by k².
- Variance is E(X²) − [E(X)]², and standard deviation carries the same units as the data.
Practice questions (8)
Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.
Two events A and B, each of positive probability, are mutually exclusive. Then they are:
Show answer
Answer: B — Necessarily dependent
Mutual exclusivity gives P(A ∩ B) = 0, while independence would require P(A ∩ B) = P(A)P(B), which is positive when both probabilities are. The two conditions therefore cannot hold together — knowing A occurred tells you B certainly did not, which is the strongest possible dependence. Treating the two terms as interchangeable is the single most common conceptual error in probability.Six dice are thrown. The probability of getting at least one six is:
Show answer
Answer: B — 1 − (5/6)⁶
The complement of "at least one six" is "no six at all", which has probability (5/6)⁶ since each die independently avoids a six with probability 5/6. Subtracting from 1 gives the answer in one line, whereas summing the cases for exactly one through exactly six sixes would take six terms. The first option exceeds 1 for larger numbers of dice, which shows it cannot be a probability at all.For a binomial distribution with n = 10 and p = 0.4, the mean and variance are:
Show answer
Answer: B — 4 and 2.4
The mean is np = 10 × 0.4 = 4 and the variance is npq = 10 × 0.4 × 0.6 = 2.4. Note that the variance is smaller than the mean, which is always true for a binomial since q < 1 — so an option offering a variance larger than the mean can be discarded immediately as not describing a binomial variable at all.If 5 is added to every observation of a data set, then:
Show answer
Answer: B — The mean increases by 5 and the variance is unchanged
A shift moves every value together, so the average moves with them while the spread about the average is untouched — variance depends only on deviations from the mean, and those are unchanged. Multiplying instead by k would scale the mean by k and the variance by k², so it is worth keeping the two operations distinct: shifts affect location only, scalings affect both.A disease affects 1 in 1000 people and a test is 99 % accurate both ways. A person tests positive. The chance they have the disease is closest to:
Show answer
Answer: C — 9 %
Out of 100 000 people about 100 are ill, of whom 99 test positive, while 99 900 are healthy and about 999 of those also test positive. So roughly 99 of about 1098 positives are genuine, giving about 9 %. Because the disease is rare, the small false-positive rate applied to a huge healthy population outnumbers the true positives — which is precisely why the prior probability matters as much as the test accuracy.P(A|B) is defined as:
Show answer
Answer: B — P(A ∩ B)/P(B), provided P(B) ≠ 0
Conditioning on B restricts attention to the part of the sample space where B occurred, so the denominator must be P(B) and the numerator the overlap. Dividing by P(A) instead gives P(B|A), a different quantity — and the two are equal only in special cases, which is exactly the asymmetry Bayes' theorem exists to handle.The variance of a random variable X is computed as:
Show answer
Answer: B — E(X²) − [E(X)]²
Variance is the expected squared deviation from the mean, and expanding that expression gives E(X²) − [E(X)]². The order matters: the third option is the negative of the right answer and would always come out negative, which is impossible for a variance — a useful sign check if the arithmetic ever produces one.For a binomial distribution, the variance is:
Show answer
Answer: B — Always less than the mean
The mean is np and the variance npq, and since q = 1 − p is strictly less than 1 for any genuine binomial, the variance is always the smaller of the two. That gives a fast plausibility check on any binomial question: if the stated variance exceeds the stated mean, the distribution described cannot be binomial and the data are inconsistent.