Probability Theory, the Central Limit Theorem and Descriptive Statistics

Unit 3 turns from theory to the quantitative tools economists use to test it. This chapter covers the concepts of probability, common distributions, moments and the central limit theorem, then descriptive statistics — central tendency, dispersion, correlation and index numbers; the next two chapters cover sampling and inference/regression, and the identification problem through time series analysis.

1. Probability Theory: Concepts, Distributions and Moments

Kolmogorov's axioms define a probability measure P on events: P(A) ≥ 0 for every event A, P(the entire sample space) = 1, and for mutually exclusive events the probability of their union is the sum of their individual probabilities. A random variable's distribution is described by a probability mass function (discrete case) or probability density function (continuous case), and the cumulative distribution function F(x) = P(X ≤ x). Among the standard distributions, the Binomial counts successes in a fixed number of independent trials with a constant success probability, the Poisson models rare, independent events over a fixed interval, and the Normal is the bell-shaped, symmetric distribution fully characterised by its mean and variance alone.

The four standard moments and what each measures
MomentWhat it measures
First (mean)The central location of the distribution.
Second central (variance)Dispersion/spread around the mean.
Third standardised (skewness)Asymmetry — zero for a symmetric distribution like the Normal, positive for a long right tail, negative for a long left tail.
Fourth standardised (kurtosis)Peakedness/tail-thickness — the Normal distribution's kurtosis of 3 is the reference point (excess kurtosis = kurtosis − 3).

2. The Central Limit Theorem

The Central Limit Theorem states that, for a sequence of independent and identically distributed random variables with finite mean μ and finite variance σ², the standardised sample mean, √n(X̄ − μ)/σ, converges IN DISTRIBUTION to a standard normal random variable as the sample size n grows large — REGARDLESS of the shape of the underlying population distribution. This is precisely why the normal distribution is so pervasive in applied statistics: sample means (and sums) of even a badly-behaved, skewed underlying variable become approximately normal once the sample is reasonably large, which is what licenses much of classical hypothesis testing and interval estimation.

⚠️ The CLT is about the SAMPLE MEAN's distribution, not the population's
A common confusion is thinking the CLT says the underlying population becomes normal as n grows — it does not, and the population distribution never changes. The theorem is entirely about the SAMPLING distribution of the mean (or any similarly-constructed sum): however skewed or oddly-shaped the population is, the distribution of X̄ across repeated samples approaches normality as n increases.

3. Descriptive Statistics: Central Tendency and Dispersion

The mean uses every observation but is pulled toward extreme values (outliers); the median (the middle value when data is ordered) is robust to outliers and is the natural centre for skewed distributions such as income, where a small number of very high incomes would drag the mean well above what a 'typical' household earns; the mode is simply the most frequent value and is the only measure of central tendency meaningful for purely categorical (nominal) data. Dispersion is measured by the range (crude, sensitive only to the two extreme values), the variance and its square root the standard deviation (uses every observation's squared deviation from the mean), and the coefficient of variation (CV = standard deviation / mean, expressed as a percentage) — the only one of these that is UNIT-FREE and therefore the right choice for comparing the relative variability of two series measured in different units or at very different scales.

🎯 Mean-median-mode ordering under skewness
For a positively (right) skewed distribution, the ordering is typically mean > median > mode, since the long right tail of a few very large values pulls the mean up furthest while the mode (the most common value) stays near the bulk of the data; a negatively skewed distribution reverses this to mean < median < mode. This ordering is a frequently tested, easily verified quick check of skewness direction.

4. Correlation and Index Numbers

The Pearson correlation coefficient r measures the strength and direction of LINEAR association between two variables, bounded between −1 and +1; r near ±1 indicates strong linear association, r near 0 indicates weak (or non-linear) linear association — a low r does NOT rule out a strong non-linear relationship, and correlation, however strong, never by itself establishes causation. Index numbers measure relative change in a variable (typically prices or quantities) over time relative to a base period: a Laspeyres price index uses BASE-YEAR quantities as fixed weights, a Paasche price index uses CURRENT-YEAR quantities as weights, and Fisher's Ideal Index is defined as the geometric mean of the Laspeyres and Paasche indices, constructed specifically to satisfy both the time-reversal test and the factor-reversal test that neither index alone satisfies.

🧠 Laspeyres versus Paasche: which overstates and which understates
Because Laspeyres holds quantities fixed at their base-year (pre-substitution) levels, it tends to OVERSTATE inflation when prices rise, since it ignores that consumers substitute away from goods whose relative price has risen; Paasche, using current-year quantities that already reflect this substitution, tends to UNDERSTATE it for the same reason. A common NET item gives specific price/quantity data across two years and asks for a numeric Laspeyres or Paasche value, or asks which of the two is larger given a described substitution pattern.

Key takeaways

  • Kolmogorov's axioms define a valid probability measure; the four standard moments (mean, variance, skewness, kurtosis) capture location, spread, asymmetry and tail-thickness respectively.
  • The Central Limit Theorem says the sample mean's own distribution approaches normality as n grows, regardless of the population's shape — it says nothing about the population itself becoming normal.
  • The median is robust to outliers and the natural centre for skewed data (e.g. income); under positive skew, mean > median > mode, reversed under negative skew.
  • The coefficient of variation is the only unit-free dispersion measure among range/variance/SD/CV, making it the right tool to compare variability across differently-scaled series.
  • Correlation measures only LINEAR association and never establishes causation; Laspeyres (base-year weights) tends to overstate inflation and Paasche (current-year weights) to understate it, while Fisher's Ideal Index (their geometric mean) corrects both indices' failure of the time-reversal and factor-reversal tests.

Practice questions (9)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. Kolmogorov's axioms of probability require that, for the entire sample space S:

    1. P(S) = 1
    2. P(S) = 0
    3. P(S) can be any positive number
    4. P(S) is undefined
    Show answer

    Answer: A — P(S) = 1

    One of Kolmogorov's three axioms fixes the probability of the entire (certain) sample space at exactly 1, alongside non-negativity and countable additivity over disjoint events.

  2. The third standardised moment of a distribution measures:

    1. Skewness — asymmetry of the distribution
    2. Dispersion around the mean
    3. The central location of the distribution
    4. Peakedness/tail-thickness
    Show answer

    Answer: A — Skewness — asymmetry of the distribution

    The first moment is the mean (location), the second central moment is variance (spread), the third standardised moment is skewness (asymmetry), and the fourth is kurtosis (tail-thickness/peakedness) — a standard NET moment-identification item.

  3. The Central Limit Theorem states that, as sample size n grows large, the sampling distribution of the sample mean approaches normality:

    1. Regardless of the shape of the underlying population distribution
    2. Only if the population is already normal
    3. Only for discrete random variables
    4. Only when the population variance is infinite
    Show answer

    Answer: A — Regardless of the shape of the underlying population distribution

    The theorem's power is precisely that it holds for iid random variables with finite mean and variance, whatever the shape of the underlying (population) distribution — normal or not.

  4. For a positively (right) skewed income distribution, the usual ordering of central tendency measures is:

    1. Mean > Median > Mode
    2. Mode > Median > Mean
    3. Mean = Median = Mode always
    4. Median > Mean > Mode
    Show answer

    Answer: A — Mean > Median > Mode

    A long right tail (a few very high incomes) pulls the mean up furthest while the mode stays near the bulk of typical values and the median sits between — giving Mean > Median > Mode for positive skew.

  5. The coefficient of variation is preferred over the standard deviation when comparing variability across two series because:

    1. It is unit-free, unlike the standard deviation which carries the original units
    2. It ignores the mean entirely
    3. It is always smaller than the standard deviation
    4. It can never exceed 1
    Show answer

    Answer: A — It is unit-free, unlike the standard deviation which carries the original units

    CV = SD/mean is a ratio and hence unit-free, so it can meaningfully compare relative variability across series measured in different units or at very different scales — something the SD alone cannot do.

  6. A Pearson correlation coefficient close to zero between two variables implies:

    1. Weak linear association — a strong non-linear relationship is not ruled out
    2. No relationship of any kind exists between the variables
    3. One variable causes the other
    4. The variables are measured in the same units
    Show answer

    Answer: A — Weak linear association — a strong non-linear relationship is not ruled out

    Pearson's r captures only LINEAR association; two variables can have a strong (e.g. quadratic) non-linear relationship while r is close to zero, so a low r never by itself rules out dependence.

  7. Fisher's Ideal price index is constructed as:

    1. The geometric mean of the Laspeyres and Paasche indices
    2. The arithmetic mean of only base-year prices
    3. The Laspeyres index divided by two
    4. The Paasche index multiplied by the CPI
    Show answer

    Answer: A — The geometric mean of the Laspeyres and Paasche indices

    Fisher's Ideal Index is defined precisely as √(Laspeyres × Paasche), and this specific construction is what allows it to satisfy both the time-reversal and factor-reversal tests that neither Laspeyres nor Paasche satisfies alone.

  8. Which of the following are true of the Laspeyres price index relative to the Paasche price index?

    1. Laspeyres uses base-year quantities as fixed weights
    2. Laspeyres tends to overstate inflation because it ignores consumer substitution
    3. Paasche uses current-year quantities as weights
    4. Laspeyres and Paasche always give exactly the same numerical value
    Show answer

    Answer: A — Laspeyres uses base-year quantities as fixed weights; B — Laspeyres tends to overstate inflation because it ignores consumer substitution; C — Paasche uses current-year quantities as weights

    The first three describe the standard Laspeyres/Paasche construction and bias correctly; the two indices generally give DIFFERENT values precisely because they use different (base-year versus current-year) quantity weights, which is why Fisher's Ideal Index averages them.

  9. Name the measure of central tendency that is robust to outliers and is the natural centre for a skewed distribution such as income.

    Numerical answer — type the value.

    Show answer

    Answer: median

    The median (the middle value when data is ordered) is unaffected by how extreme the tail values are, unlike the mean, which is exactly why it is the preferred measure for skewed data such as income or wealth.