Sampling and Statistical Inference, and Linear Regression Models with the BLUE Properties
2. Statistical Inference and Hypothesis Testing
Estimation asks for the best single value (point estimate) or a range (confidence interval) for an unknown population parameter using sample data; a good point estimator should be unbiased (its expected value equals the true parameter) and, ideally, of minimum variance among unbiased estimators (efficient), while also being consistent (its distribution collapses onto the true parameter as sample size grows). A confidence interval of, say, 95%, means that if the same sampling-and-interval-construction procedure were repeated many times, 95% of the resulting intervals would contain the true parameter — it is a statement about the PROCEDURE across repeated samples, not a 95% probability that this one particular interval contains the true value.
3. Linear Regression Models and the BLUE Properties
The classical linear regression model Y = β0 + β1X + u estimates β0 and β1 by Ordinary Least Squares (OLS), which minimises the sum of squared residuals. The Gauss-Markov theorem states that, under the classical assumptions — linearity in parameters, the error term has zero mean and constant variance (homoskedasticity), errors are uncorrelated with each other (no autocorrelation) and with X (exogeneity), and X is non-stochastic (or independent of the error) — OLS is BLUE: Best (minimum variance) Linear Unbiased Estimator, among the class of all linear unbiased estimators, whether or not the error term is normally distributed. Normality of the error is a SEPARATE assumption, needed only for exact small-sample hypothesis testing (t- and F-tests), not for the BLUE property itself.
| Letter | Requirement |
|---|---|
| B — Best | Minimum variance among all linear unbiased estimators. |
| L — Linear | The estimator is a linear function of the sample observations (of Y). |
| U — Unbiased | Its expected value equals the true population parameter. |
| E — Estimator | A rule/formula computed from sample data used to infer the unknown parameter. |
4. Reading and Testing a Regression
- R² and adjusted R² — R² is the proportion of variation in Y explained by X, and mechanically rises (or stays the same) with every additional regressor added, however irrelevant; adjusted R² penalises added regressors, making it the fairer basis for comparing models with a different number of explanatory variables.
- The t-test on a single coefficient — tests whether an individual β differs significantly from a hypothesised value (usually zero); the F-test on overall significance tests whether the regressors, taken together, explain a significant share of Y's variation.
Key takeaways
- Simple random, stratified, cluster and systematic sampling trade off precision against field cost differently; a sample statistic's own probability distribution across repeated samples is the sampling distribution, whose standard deviation is the standard error.
- Type I error (rejecting a true null, probability α) and Type II error (failing to reject a false null, probability β) trade off for a fixed sample size; a 95% confidence interval is a statement about the repeated-sampling procedure, not this one interval's probability.
- The Gauss-Markov theorem makes OLS BLUE under its classical assumptions, with no need for normality of the error term — normality is a separate assumption needed only for exact small-sample hypothesis testing.
- Heteroskedasticity and autocorrelation leave OLS unbiased but no longer minimum-variance (no longer BLUE), and make standard errors — hence hypothesis tests — unreliable.
- R² mechanically rises with added regressors regardless of relevance; adjusted R² penalises this, and neither R² measure by itself says anything about whether the coefficients are unbiased or efficient.
Practice questions (9)
Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.
Stratified sampling generally improves precision over simple random sampling when:
Show answer
Answer: A — The strata differ from each other more than units differ within a stratum
Stratification gains precision by exploiting between-strata heterogeneity while sampling within relatively homogeneous strata; if strata do not differ from each other, stratifying buys nothing over simple random sampling.A Type II error in hypothesis testing is:
Show answer
Answer: A — Failing to reject a false null hypothesis
Type I error rejects a true null (probability α); Type II error fails to reject a false null (probability β, with power = 1−β) — the two are distinct and trade off against each other for a fixed sample size.A 95% confidence interval for a population parameter means:
Show answer
Answer: A — If the sampling-and-interval procedure were repeated many times, 95% of the resulting intervals would contain the true parameter
The correct frequentist interpretation is about the long-run performance of the PROCEDURE across repeated samples, not a probability statement about one already-computed interval, which either does or does not contain the fixed true parameter.The Gauss-Markov theorem states that, under its classical assumptions, OLS is:
Show answer
Answer: A — The Best (minimum variance) Linear Unbiased Estimator
BLUE holds under the classical (Gauss-Markov) assumptions WITHOUT needing the error term to be normally distributed — normality is a separate, additional assumption used only for exact small-sample inference.In the acronym BLUE, the 'B' stands for:
Show answer
Answer: A — Best — minimum variance among all linear unbiased estimators
BLUE stands for Best Linear Unbiased Estimator; 'Best' specifically means minimum variance among the class of all linear, unbiased estimators of the parameter.If the error term in a regression is heteroskedastic, OLS remains:
Show answer
Answer: A — Unbiased, but no longer minimum variance (no longer BLUE), and standard errors are unreliable
Heteroskedasticity violates the constant-variance assumption of Gauss-Markov: OLS coefficients stay unbiased, but they are no longer minimum-variance and the usual standard error formulas are wrong, making hypothesis tests unreliable unless corrected (e.g. robust standard errors).Adjusted R² is preferred over plain R² for comparing models because:
Show answer
Answer: A — It penalises the addition of irrelevant regressors, unlike plain R² which mechanically rises (or stays the same) with every added regressor
Plain R² cannot fall when a regressor is added, however irrelevant, so it favours larger models automatically; adjusted R² includes a penalty for the number of regressors relative to sample size, giving a fairer basis for model comparison.Which of the following are among the classical (Gauss-Markov) assumptions needed for OLS to be BLUE?
Show answer
Answer: A — The error term has zero mean and constant variance; B — Errors are uncorrelated with each other and with the regressor X; C — The model is linear in parameters
The first three are Gauss-Markov assumptions required for BLUE; normality is a separate, additional assumption needed only for exact small-sample hypothesis testing, not for the BLUE property itself.Name the theorem that establishes OLS as the Best Linear Unbiased Estimator under the classical regression assumptions.
Numerical answer — type the value.
Show answer
Answer: Gauss-Markov theorem
The Gauss-Markov theorem is the standard name for this result, and it holds without any assumption of normality on the error term.