Sampling and Statistical Inference, and Linear Regression Models with the BLUE Properties

This chapter covers how economists draw a sample, use it to test hypotheses, and then use regression to estimate the relationships economic theory predicts — with the precise sense in which ordinary least squares is the 'best' estimator.

1. Sampling Methods and the Sampling Distribution

In simple random sampling, every member of the population has an equal and independent chance of selection; stratified sampling first divides the population into homogeneous strata (e.g. by state or income bracket) and samples within each, generally improving precision when the strata differ from each other more than units differ within a stratum; cluster sampling samples entire pre-existing groups (e.g. villages) rather than individuals, trading some precision for lower field cost; systematic sampling selects every kth unit from an ordered list after a random start. A statistic (e.g. the sample mean) computed from a sample is itself a random variable, since it varies from sample to sample — its own probability distribution across all possible samples of a given size is the sampling distribution, and the standard error is its standard deviation.

ℹ️ Hypothesis testing: two errors, one trade-off
A Type I error rejects a true null hypothesis (its probability is the significance level, α, chosen by the researcher); a Type II error fails to reject a false null hypothesis (its probability is β, and 1−β is the test's POWER). For a fixed sample size, lowering α (demanding stronger evidence to reject) mechanically raises β — the two error probabilities trade off against each other, and only a larger sample can reduce both simultaneously.

2. Statistical Inference and Hypothesis Testing

Estimation asks for the best single value (point estimate) or a range (confidence interval) for an unknown population parameter using sample data; a good point estimator should be unbiased (its expected value equals the true parameter) and, ideally, of minimum variance among unbiased estimators (efficient), while also being consistent (its distribution collapses onto the true parameter as sample size grows). A confidence interval of, say, 95%, means that if the same sampling-and-interval-construction procedure were repeated many times, 95% of the resulting intervals would contain the true parameter — it is a statement about the PROCEDURE across repeated samples, not a 95% probability that this one particular interval contains the true value.

3. Linear Regression Models and the BLUE Properties

The classical linear regression model Y = β0 + β1X + u estimates β0 and β1 by Ordinary Least Squares (OLS), which minimises the sum of squared residuals. The Gauss-Markov theorem states that, under the classical assumptions — linearity in parameters, the error term has zero mean and constant variance (homoskedasticity), errors are uncorrelated with each other (no autocorrelation) and with X (exogeneity), and X is non-stochastic (or independent of the error) — OLS is BLUE: Best (minimum variance) Linear Unbiased Estimator, among the class of all linear unbiased estimators, whether or not the error term is normally distributed. Normality of the error is a SEPARATE assumption, needed only for exact small-sample hypothesis testing (t- and F-tests), not for the BLUE property itself.

What each letter of BLUE requires
LetterRequirement
B — BestMinimum variance among all linear unbiased estimators.
L — LinearThe estimator is a linear function of the sample observations (of Y).
U — UnbiasedIts expected value equals the true population parameter.
E — EstimatorA rule/formula computed from sample data used to infer the unknown parameter.
⚠️ Violated assumptions do not make OLS wrong to compute — they make it unreliable
Heteroskedasticity (non-constant error variance) leaves OLS still unbiased but no longer BLUE (no longer minimum variance, and standard errors are wrong, so hypothesis tests are unreliable); autocorrelated errors have the same consequence in time-series regressions. NET frequently tests which Gauss-Markov assumption a described violation breaks and which specific property (unbiasedness versus efficiency versus valid standard errors) it costs — the three are not lost together.

4. Reading and Testing a Regression

  • R² and adjusted R² — R² is the proportion of variation in Y explained by X, and mechanically rises (or stays the same) with every additional regressor added, however irrelevant; adjusted R² penalises added regressors, making it the fairer basis for comparing models with a different number of explanatory variables.
  • The t-test on a single coefficient — tests whether an individual β differs significantly from a hypothesised value (usually zero); the F-test on overall significance tests whether the regressors, taken together, explain a significant share of Y's variation.
📖 Why NET keeps returning to BLUE rather than to R²
A high R² tells you the fitted line tracks the data well IN-SAMPLE; it says nothing about whether the estimated coefficients are unbiased or efficient, which is exactly what the Gauss-Markov assumptions and the BLUE property are about. A regression can have a very high R² and still be built on a mis-specified or endogenous model whose coefficients are badly biased — which is why the syllabus places 'linear regression models and their properties — BLUE' as a distinct, separately-tested item rather than folding it into goodness-of-fit.

Key takeaways

  • Simple random, stratified, cluster and systematic sampling trade off precision against field cost differently; a sample statistic's own probability distribution across repeated samples is the sampling distribution, whose standard deviation is the standard error.
  • Type I error (rejecting a true null, probability α) and Type II error (failing to reject a false null, probability β) trade off for a fixed sample size; a 95% confidence interval is a statement about the repeated-sampling procedure, not this one interval's probability.
  • The Gauss-Markov theorem makes OLS BLUE under its classical assumptions, with no need for normality of the error term — normality is a separate assumption needed only for exact small-sample hypothesis testing.
  • Heteroskedasticity and autocorrelation leave OLS unbiased but no longer minimum-variance (no longer BLUE), and make standard errors — hence hypothesis tests — unreliable.
  • R² mechanically rises with added regressors regardless of relevance; adjusted R² penalises this, and neither R² measure by itself says anything about whether the coefficients are unbiased or efficient.

Practice questions (9)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. Stratified sampling generally improves precision over simple random sampling when:

    1. The strata differ from each other more than units differ within a stratum
    2. Every stratum is identical to every other
    3. The sample size is very small
    4. The population has no natural groupings at all
    Show answer

    Answer: A — The strata differ from each other more than units differ within a stratum

    Stratification gains precision by exploiting between-strata heterogeneity while sampling within relatively homogeneous strata; if strata do not differ from each other, stratifying buys nothing over simple random sampling.
  2. A Type II error in hypothesis testing is:

    1. Failing to reject a false null hypothesis
    2. Rejecting a true null hypothesis
    3. Choosing too large a sample size
    4. Using a biased estimator
    Show answer

    Answer: A — Failing to reject a false null hypothesis

    Type I error rejects a true null (probability α); Type II error fails to reject a false null (probability β, with power = 1−β) — the two are distinct and trade off against each other for a fixed sample size.
  3. A 95% confidence interval for a population parameter means:

    1. If the sampling-and-interval procedure were repeated many times, 95% of the resulting intervals would contain the true parameter
    2. There is a 95% probability that this specific interval contains the true parameter
    3. 95% of the sample observations fall inside the interval
    4. The true parameter changes 95% of the time
    Show answer

    Answer: A — If the sampling-and-interval procedure were repeated many times, 95% of the resulting intervals would contain the true parameter

    The correct frequentist interpretation is about the long-run performance of the PROCEDURE across repeated samples, not a probability statement about one already-computed interval, which either does or does not contain the fixed true parameter.
  4. The Gauss-Markov theorem states that, under its classical assumptions, OLS is:

    1. The Best (minimum variance) Linear Unbiased Estimator
    2. The only consistent estimator possible
    3. Unbiased only if the errors are normally distributed
    4. Guaranteed to have an R² of 1
    Show answer

    Answer: A — The Best (minimum variance) Linear Unbiased Estimator

    BLUE holds under the classical (Gauss-Markov) assumptions WITHOUT needing the error term to be normally distributed — normality is a separate, additional assumption used only for exact small-sample inference.
  5. In the acronym BLUE, the 'B' stands for:

    1. Best — minimum variance among all linear unbiased estimators
    2. Biased
    3. Bayesian
    4. Bounded
    Show answer

    Answer: A — Best — minimum variance among all linear unbiased estimators

    BLUE stands for Best Linear Unbiased Estimator; 'Best' specifically means minimum variance among the class of all linear, unbiased estimators of the parameter.
  6. If the error term in a regression is heteroskedastic, OLS remains:

    1. Unbiased, but no longer minimum variance (no longer BLUE), and standard errors are unreliable
    2. Biased and inconsistent
    3. Still BLUE with no consequence at all
    4. Undefined
    Show answer

    Answer: A — Unbiased, but no longer minimum variance (no longer BLUE), and standard errors are unreliable

    Heteroskedasticity violates the constant-variance assumption of Gauss-Markov: OLS coefficients stay unbiased, but they are no longer minimum-variance and the usual standard error formulas are wrong, making hypothesis tests unreliable unless corrected (e.g. robust standard errors).
  7. Adjusted R² is preferred over plain R² for comparing models because:

    1. It penalises the addition of irrelevant regressors, unlike plain R² which mechanically rises (or stays the same) with every added regressor
    2. It is always equal to 1
    3. It ignores the number of observations
    4. It measures unbiasedness directly
    Show answer

    Answer: A — It penalises the addition of irrelevant regressors, unlike plain R² which mechanically rises (or stays the same) with every added regressor

    Plain R² cannot fall when a regressor is added, however irrelevant, so it favours larger models automatically; adjusted R² includes a penalty for the number of regressors relative to sample size, giving a fairer basis for model comparison.
  8. Which of the following are among the classical (Gauss-Markov) assumptions needed for OLS to be BLUE?

    1. The error term has zero mean and constant variance
    2. Errors are uncorrelated with each other and with the regressor X
    3. The model is linear in parameters
    4. The error term must be normally distributed
    Show answer

    Answer: A — The error term has zero mean and constant variance; B — Errors are uncorrelated with each other and with the regressor X; C — The model is linear in parameters

    The first three are Gauss-Markov assumptions required for BLUE; normality is a separate, additional assumption needed only for exact small-sample hypothesis testing, not for the BLUE property itself.
  9. Name the theorem that establishes OLS as the Best Linear Unbiased Estimator under the classical regression assumptions.

    Numerical answer — type the value.

    Show answer

    Answer: Gauss-Markov theorem

    The Gauss-Markov theorem is the standard name for this result, and it holds without any assumption of normality on the error term.