Axioms of Probability, the Normal Distribution, Correlation and Regression, Interpolation and Simpson's 3/8 Rule

This chapter belongs to Section XE0 (Engineering Mathematics) of the Engineering Sciences (XE) paper, the common 15-mark part. It completes XE0.7 and XE0.8: “Axioms of probability; … Binomial, Poisson and Normal distributions; Correlation and linear regression” and “Lagrange and Newton's interpolations; … Numerical integration by trapezoidal rule and Simpson's 1/3rd and 3/8th rules.” The shared probability chapters give conditional probability, total probability, Bayes, mean and variance, and the binomial and Poisson families, and the shared numerical chapter gives bisection, Newton-Raphson, the trapezoidal and 1/3 rules and Euler's method. This chapter adds what they leave out: the three axioms and what follows, standardisation and tables for the normal distribution with the normal approximation of a binomial, Pearson correlation and the two regression lines, Lagrange and Newton interpolation, and the 3/8 rule. The numericals are a union probability, a normal probability, a regression prediction, an interpolated value and a 3/8-rule integral.

1. The axioms of probability and their consequences

A probability P assigns to each event a number with three axioms (Kolmogorov):
(1) P(A) ≥ 0;
(2) P(S) = 1 for the sample space S;
(3) for mutually exclusive events A₁, A₂, …, P(∪Aᵢ) = ΣP(Aᵢ) (countable additivity). Everything else is derived from these.

  • P(∅) = 0, and P(Aᶜ) = 1 − P(A) (since A and Aᶜ are exclusive and exhaust S).
  • If A ⊂ B then P(A) ≤ P(B) (monotonicity), so every P(A) ≤ 1.
  • P(A ∪ B) = P(A) + P(B) − P(A ∩ B); for three events P(A ∪ B ∪ C) = ΣP(A) − ΣP(pairs) + P(A ∩ B ∩ C).
  • Union bound: P(∪Aᵢ) ≤ ΣP(Aᵢ). Bounds: max(0, P(A) + P(B) − 1) ≤ P(A ∩ B) ≤ min(P(A), P(B)).

Worked. (i) P(A) = 0.6, P(B) = 0.7: since P(A ∪ B) ≤ 1, P(A ∩ B) ≥ 0.6 + 0.7 − 1 = 0.3, and at most min = 0.6. (ii) P(A) = 0.5, P(B) = 0.4, P(C) = 0.3, P(A ∩ B) = 0.2, P(A ∩ C) = 0.1, P(B ∩ C) = 0.15, P(A ∩ B ∩ C) = 0.05: P(A ∪ B ∪ C) = 1.2 − 0.45 + 0.05 = 0.80. (iii) Exclusive events with P(A) = 0.3, P(B) = 0.5: P(neither) = 1 − 0.8 = 0.2.

⚠️ Exclusive is not independent
Exclusive events (A ∩ B = ∅) with positive probabilities are strongly dependent: knowing A occurred forces B not to. Independence means P(A ∩ B) = P(A)P(B) ≠ 0. Adding probabilities without subtracting the overlap is only valid for exclusive events.

2. The normal distribution

X ~ N(μ, σ²) has density (1/(σ√(2π))) exp(−(x − μ)²/(2σ²)), a bell centred on μ with spread σ. Standardise: Z = (X − μ)/σ ~ N(0, 1), and P(X ≤ x) = Φ((x − μ)/σ), where Φ is the tabulated standard normal CDF with Φ(−z) = 1 − Φ(z). Useful values: Φ(1) = 0.8413, Φ(1.645) = 0.95, Φ(1.96) = 0.975, Φ(2) = 0.9772. A linear function aX + b is N(aμ + b, a²σ²); the sum of independent normals is normal with means and variances added.

Worked. X ~ N(50, 16), σ = 4: P(46 < X < 58) = Φ(2) − Φ(−1) = 0.9772 − (1 − 0.8413) = 0.8185. For rod lengths N(100, 25) mm: P(95 < L < 110) is the same standardised interval (−1, 2), so also 0.8185.

Normal approximation to the binomial. If np and n(1 − p) are both at least about 5 (or 10), Bin(n, p) ≈ N(np, np(1 − p)). Apply the continuity correction: P(X ≥ k) ≈ P(Y > k − 0.5). For Bin(100, 0.5): μ = 50, σ = 5, so P(X ≥ 60) ≈ 1 − Φ((59.5 − 50)/5) = 1 − Φ(1.9) = 1 − 0.9713 = 0.0287 (the exact value is 0.0284; without the correction one gets 0.0228).

⚠️ Variance, not standard deviation
N(μ, σ²) is written with the variance. N(50, 16) has σ = 4, not 16. The probability of a single exact value is zero for a continuous variable, so P(X < a) = P(X ≤ a); the continuity correction applies only when a discrete variable is approximated.

3. Correlation

The covariance is Cov(X, Y) = E[(X − μₓ)(Y − μᵧ)] = E[XY] − E[X]E[Y]. The Pearson correlation coefficient is r = Cov(X, Y)/(σₓσᵧ) = Sₓᵧ/√(SₓₓSᵧᵧ) for data, with Sₓᵧ = Σ(xᵢ − x̄)(yᵢ − ȳ). Always −1 ≤ r ≤ 1; |r| = 1 means exact linear dependence, and r² is the fraction of the variance of y explained by the line.

  • r is unchanged by shifting the origin or by multiplying x and y by positive constants; multiplying one variable by a negative constant flips the sign of r.
  • Independent variables have r = 0, but r = 0 does not imply independence: for X symmetric about 0 and Y = X², Cov(X, Y) = E[X³] − 0 = 0, yet Y is determined by X.
  • Correlation measures linear association only, and it is not causation.

4. Linear regression

The regression line of y on x minimises Σ(yᵢ − a − bxᵢ)², giving ŷ = ȳ + b_yx(x − x̄) with b_yx = Sₓᵧ/Sₓₓ = r σᵧ/σₓ. The line of x on y is x̂ = x̄ + b_xy(y − ȳ) with b_xy = Sₓᵧ/Sᵧᵧ = r σₓ/σᵧ. Facts: both lines pass through (x̄, ȳ); b_yx·b_xy = r², so the two coefficients have the same sign as r, and the lines coincide only when |r| = 1; they become perpendicular as r → 0 (one horizontal and one vertical).

Worked. x = 1, 2, 3, 4, 5 and y = 2, 4, 5, 4, 5: x̄ = 3, ȳ = 4, Sₓₓ = 10, Sₓᵧ = (−2)(−2) + (−1)(0) + 0 + 0 + 2(1) = 6, Sᵧᵧ = 4 + 0 + 1 + 0 + 1 = 6. So b_yx = 0.6, ŷ = 4 + 0.6(x − 3) = 2.2 + 0.6x, the prediction at x = 6 is 5.8, and r = 6/√60 = 0.775. If b_yx = 0.4 and b_xy = 0.9 are both positive, r = +√(0.36) = 0.6. If the lines are 3x + 2y = 26 and 6x + y = 31, both pass through the means: solving gives x̄ = 4, ȳ = 7.

⚠️ Which variable is on which side
b_yx = Sₓᵧ/Sₓₓ predicts y from x, and b_xy = Sₓᵧ/Sᵧᵧ predicts x from y. They are not reciprocals unless |r| = 1: the slope of the x-on-y line drawn in the (x, y) plane is 1/b_xy, steeper than b_yx when |r| < 1. Predicting outside the range of the data (extrapolating) is unreliable.

5. Lagrange and Newton interpolation

Through n + 1 points with distinct x₀, …, xₙ there is exactly one polynomial of degree at most n. The Lagrange form is P(x) = Σ yᵢLᵢ(x) with Lᵢ(x) = Πj≠i(x − xⱼ)/(xᵢ − xⱼ), so Lᵢ(xⱼ) = 1 if i = j and 0 otherwise. The Newton divided-difference form is P(x) = f[x₀] + f[x₀, x₁](x − x₀) + f[x₀, x₁, x₂](x − x₀)(x − x₁) + …, where f[xᵢ, xᵢ₊₁] = (yᵢ₊₁ − yᵢ)/(xᵢ₊₁ − xᵢ) and each higher difference is built from the lower ones. They are the same polynomial written differently, but Newton's form is easy to extend by a new point.

Worked. Points (1, 1), (2, 8), (4, 64) (from y = x³). Lagrange at x = 3: L₀ = (3 − 2)(3 − 4)/((1 − 2)(1 − 4)) = −1/3, L₁ = (3 − 1)(3 − 4)/((2 − 1)(2 − 4)) = 1, L₂ = (3 − 1)(3 − 2)/((4 − 1)(4 − 2)) = 1/3, so P(3) = −1/3 + 8 + 64/3 = 29. Newton: f[1, 2] = 7, f[2, 4] = 28, f[1, 2, 4] = (28 − 7)/3 = 7, so P(x) = 1 + 7(x − 1) + 7(x − 1)(x − 2) and P(3) = 1 + 14 + 14 = 29. The true value is 27; the error formula f‴(ξ)/3!·(x − 1)(x − 2)(x − 4) = 1·(2)(1)(−1) = −2 agrees (27 − 29).

Equal spacing (Newton forward difference). With step h and s = (x − x₀)/h, P = y₀ + sΔy₀ + s(s − 1)Δ²y₀/2! + s(s − 1)(s − 2)Δ³y₀/3! + …, where Δyᵢ = yᵢ₊₁ − yᵢ. The error after n + 1 points is f⁽ⁿ⁺¹⁾(ξ)/(n + 1)! · Π(x − xᵢ), small near the middle of the points and large when extrapolating.

6. Simpson's 3/8 rule

Fit a cubic through four equally spaced points and integrate: ∫ f dx ≈ (3h/8)[f₀ + 3f₁ + 3f₂ + f₃] over one panel of three subintervals. The composite rule for n subintervals (n a multiple of 3) is (3h/8)[f₀ + 3f₁ + 3f₂ + 2f₃ + 3f₄ + 3f₅ + 2f₆ + … + fₙ]: weights 1, 3, 3, 2, 3, 3, 2, …, 3, 3, 1. It is exact for polynomials up to degree 3, and its error per panel is −(3/80)h⁵f⁗(ξ), the same order as the 1/3 rule, which needs an even n.

Worked. (i) ∫₀¹ x³ dx with n = 3, h = 1/3: (1/8)[0 + 3/27 + 24/27 + 1] = (1/8)(2) = 0.25, exact. (ii) ∫₀¹ dx/(1 + x²) = π/4 = 0.785398 with n = 3: f = 1, 0.9, 0.692308, 0.5, so I ≈ (1/8)[1 + 2.7 + 2.076923 + 0.5] = 6.276923/8 = 0.7846.

🧠 Which rule fits which n
n = 6 works for both the 1/3 rule (even) and the 3/8 rule (multiple of 3). For n = 9 only the 3/8 rule is usable alone; for n = 8 only the 1/3 rule. With an odd count such as 7, use the 3/8 rule on the first three intervals and the 1/3 rule on the remaining four.

Key takeaways

  • Axioms: P ≥ 0, P(S) = 1, additivity; from them P(Aᶜ) = 1 − P(A), inclusion-exclusion and max(0, P(A) + P(B) − 1) ≤ P(A ∩ B) ≤ min.
  • Normal: standardise Z = (X − μ)/σ and use Φ; σ is the root of the variance; Bin(n, p) ≈ N(np, npq) with continuity correction k ± 0.5.
  • r = Sₓᵧ/√(SₓₓSᵧᵧ) lies in [−1, 1], is invariant to positive rescaling and shifting; r = 0 does not mean independence.
  • Regression: b_yx = Sₓᵧ/Sₓₓ = rσᵧ/σₓ, b_xy = rσₓ/σᵧ, b_yx·b_xy = r², both lines pass through (x̄, ȳ).
  • n + 1 points give one polynomial of degree ≤ n; Lagrange and Newton divided-difference are the same polynomial; the error is f⁽ⁿ⁺¹⁾(ξ)Π(x − xᵢ)/(n + 1)!.
  • Simpson's 3/8: (3h/8)[f₀ + 3f₁ + 3f₂ + 2f₃ + … + fₙ] with n a multiple of 3; exact up to cubics; the 1/3 rule needs n even.

Practice questions (17)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. For events A and B with P(A) = 0.6 and P(B) = 0.7, which interval must contain P(A ∩ B)?

    1. [0.3, 0.6]
    2. [0, 0.6]
    3. [0.42, 0.42]
    4. [0.6, 0.7]
    Show answer

    Answer: A — [0.3, 0.6]

    P(A ∪ B) = 1.3 − P(A ∩ B) ≤ 1 gives P(A ∩ B) ≥ 0.3, and P(A ∩ B) ≤ min(0.6, 0.7) = 0.6. The value 0.42 would hold only if A and B were independent, which is not given, and the lower end 0 ignores that the union cannot exceed 1.

  2. Mutually exclusive events A and B have P(A) = 0.3 and P(B) = 0.5. The probability that neither occurs is:

    1. 0.35
    2. 0.65
    3. 0.15
    4. 0.2
    Show answer

    Answer: D — 0.2

    For exclusive events P(A ∪ B) = 0.3 + 0.5 = 0.8, so P(neither) = 1 − 0.8 = 0.2. The product 0.7 × 0.5 = 0.35 would be the answer for independent events, which exclusive events with positive probability are not.

  3. X ~ N(μ, σ²) and Φ(1.96) = 0.975. Then P(X > μ + 1.96σ) equals:

    1. 0.95
    2. 0.975
    3. 0.05
    4. 0.025
    Show answer

    Answer: D — 0.025

    Standardising gives P(Z > 1.96) = 1 − Φ(1.96) = 0.025. The value 0.95 is the two-sided probability P(|Z| < 1.96), and 0.975 is the lower tail P(Z < 1.96), both different events.

  4. X ~ Bin(100, 0.5). Using the normal approximation with continuity correction, and Φ(1.9) = 0.9713, Φ(2.0) = 0.9772, the value of P(X ≥ 60) is closest to:

    1. 0.0228
    2. 0.0287
    3. 0.0179
    4. 0.0359
    Show answer

    Answer: B — 0.0287

    μ = 50, σ = √25 = 5. P(X ≥ 60) = P(X > 59.5) ≈ 1 − Φ(1.9) = 0.0287. Without the correction, z = 2 gives 0.0228, the correction applied in the wrong direction (60.5) gives z = 2.1 and a smaller tail near 0.018.

  5. The correlation coefficient of X and Y is 0.8. For U = 2X + 3 and V = −5Y + 1, the correlation coefficient of U and V is:

    1. 0.8
    2. −0.8
    3. −4
    4. −10
    Show answer

    Answer: B — −0.8

    Shifts do not change r and scaling by 2 and by 5 does not change the magnitude, but the negative factor −5 flips the sign, so r = −0.8. Covariance would scale by 2 × (−5) = −10, but the correlation is normalised by the standard deviations.

  6. The two regression coefficients of a bivariate sample are b_yx = 0.4 and b_xy = 0.9. The correlation coefficient is:

    1. 0.36
    2. 0.6
    3. −0.6
    4. 0.65
    Show answer

    Answer: B — 0.6

    b_yx·b_xy = r², so r² = 0.36 and |r| = 0.6. Both coefficients are positive, so r has the same positive sign: r = +0.6. The value 0.36 is r², not r, and 0.65 is the mean of the coefficients, which has no meaning here.

  7. The two regression lines of a sample are 3x + 2y = 26 and 6x + y = 31. The means x̄ and ȳ are:

    1. x̄ = 5, ȳ = 1
    2. x̄ = 4, ȳ = 7
    3. x̄ = 7, ȳ = 4
    4. x̄ = 2, ȳ = 10
    Show answer

    Answer: B — x̄ = 4, ȳ = 7

    Both regression lines pass through the point of means, so solve them together: y = 31 − 6x gives 3x + 62 − 12x = 26, x = 4, y = 7. Check: 3(4) + 2(7) = 26 and 6(4) + 7 = 31. The point (5, 1) satisfies only the second line, and the pair (2, 10) satisfies only the first.

  8. For the points (1, 1), (2, 8), (4, 64), the second divided difference f[1, 2, 4] is:

    1. 28
    2. 21
    3. 7
    4. 3
    Show answer

    Answer: C — 7

    f[1, 2] = (8 − 1)/(2 − 1) = 7 and f[2, 4] = (64 − 8)/(4 − 2) = 28, so f[1, 2, 4] = (28 − 7)/(4 − 1) = 7. The numerator 21 without dividing by the span 4 − 1 = 3 is the usual slip.

  9. Simpson's 3/8 rule is applied in composite form to a function on [0, 9] with equal subintervals. Which number of subintervals can be used with the 3/8 rule alone?

    1. n = 4
    2. n = 7
    3. n = 9
    4. n = 10
    Show answer

    Answer: C — n = 9

    The composite 3/8 rule needs n to be a multiple of 3, so n = 9 (three panels) qualifies. n = 4 and n = 10 suit the 1/3 rule (even), and n = 7 needs a mixture of the two rules.

  10. Which statements about probabilities of events A, B, C are correct?

    1. P(A ∪ B ∪ C) ≤ P(A) + P(B) + P(C)
    2. P(A ∩ B) ≥ P(A) + P(B)
    3. P(A ∩ B) ≥ P(A) + P(B) − 1
    4. If A ⊂ B then P(A) ≥ P(B)
    Show answer

    Answer: A — P(A ∪ B ∪ C) ≤ P(A) + P(B) + P(C); C — P(A ∩ B) ≥ P(A) + P(B) − 1

    The union bound and the Bonferroni lower bound P(A ∩ B) ≥ P(A) + P(B) − 1 both follow from P(A ∪ B) ≤ 1. P(A ∩ B) ≥ P(A) + P(B) is false (the left side is at most min(P(A), P(B))), and monotonicity gives P(A) ≤ P(B) when A ⊂ B.

  11. Which statements about correlation and regression are correct?

    1. r is unchanged when the origin of x and y is shifted
    2. r = 0 implies that X and Y are independent
    3. The two regression lines coincide when |r| = 1
    4. b_yx and b_xy are always equal
    Show answer

    Answer: A — r is unchanged when the origin of x and y is shifted; C — The two regression lines coincide when |r| = 1

    Shifting the origin leaves deviations from the means unchanged, and perfect linear dependence makes the two lines identical (b_yx·b_xy = 1). r = 0 only means no linear association (X symmetric and Y = X² is a counter-example), and b_yx = rσᵧ/σₓ differs from b_xy = rσₓ/σᵧ unless σₓ = σᵧ.

  12. Which statements about interpolation and Simpson's rules are correct?

    1. The Lagrange and Newton divided-difference forms through the same points give the same polynomial
    2. Simpson's 3/8 rule is exact for every polynomial of degree at most 3
    3. Simpson's 3/8 rule is exact only for polynomials of degree at most 2
    4. The interpolating polynomial through n + 1 points always has degree exactly n
    Show answer

    Answer: A — The Lagrange and Newton divided-difference forms through the same points give the same polynomial; B — Simpson's 3/8 rule is exact for every polynomial of degree at most 3

    An interpolating polynomial of degree at most n through n + 1 distinct points is unique, so both forms coincide, and the 3/8 rule integrates cubics exactly (x³ over [0, 1] gives 0.25). The degree can be lower than n (three collinear points give a line), and exactness holds through degree 3, not only 2.

  13. For events A, B, C: P(A) = 0.5, P(B) = 0.4, P(C) = 0.3, P(A ∩ B) = 0.2, P(A ∩ C) = 0.1, P(B ∩ C) = 0.15 and P(A ∩ B ∩ C) = 0.05. Then P(A ∪ B ∪ C) = ____ (two decimal places).

    Numerical answer — type the value.

    Show answer

    Answer: 0.8

    Inclusion-exclusion: 0.5 + 0.4 + 0.3 − (0.2 + 0.1 + 0.15) + 0.05 = 1.2 − 0.45 + 0.05 = 0.80. Second check by Python with exact fractions gives 4/5.

  14. A shaft diameter is normal with mean 100 mm and variance 25 mm². Given Φ(1) = 0.8413 and Φ(2) = 0.9772, the probability that it lies between 95 mm and 110 mm, to two decimal places, is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 0.82

    σ = 5 mm. z-limits: (95 − 100)/5 = −1 and (110 − 100)/5 = 2. P = Φ(2) − Φ(−1) = 0.9772 − (1 − 0.8413) = 0.9772 − 0.1587 = 0.8185 ≈ 0.82. The exact error-function evaluation in Python gives 0.81859, which also rounds to 0.82.

  15. For the data x = 1, 2, 3, 4, 5 and y = 2, 4, 5, 4, 5, the least-squares regression line of y on x predicts y at x = 6 as ____ (one decimal place).

    Numerical answer — type the value.

    Show answer

    Answer: 5.8

    x̄ = 3, ȳ = 4, Sₓₓ = 10, Sₓᵧ = 6, so the slope is 0.6 and the intercept 4 − 0.6 × 3 = 2.2. At x = 6: 2.2 + 3.6 = 5.8. A Python normal-equation computation gives the same slope and intercept.

  16. The quadratic polynomial through (1, 1), (2, 8) and (4, 64) is used to estimate f(3). The estimate is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 29

    Newton form: f[1, 2] = 7, f[1, 2, 4] = 7, so P(x) = 1 + 7(x − 1) + 7(x − 1)(x − 2) and P(3) = 1 + 14 + 14 = 29. Lagrange weights −1/3, 1, 1/3 give −1/3 + 8 + 64/3 = 29 also.

  17. Using Simpson's 3/8 rule with n = 3 subintervals, evaluate ∫₀¹ dx/(1 + x²). Give the answer to four decimal places.

    Numerical answer — type the value.

    Show answer

    Answer: 0.7846

    h = 1/3, f(0) = 1, f(1/3) = 0.9, f(2/3) = 9/13 = 0.692308, f(1) = 0.5. I ≈ (3h/8)[1 + 3(0.9) + 3(0.692308) + 0.5] = (1/8)(6.276923) = 0.784615 ≈ 0.7846. The exact value π/4 = 0.7854 differs by 8 × 10⁻⁴, as expected for this rule.