Which assumption is specific to ANCOVA?

Homogeneity of regression slopes

Normality of the dependent variable

A therapist wants to show that a token programme controls a child's self-injury but judges it unethical to withdraw treatment once it works. The most suitable design is:

Multiple-baseline design across settings

Assertion (A): Cross-sectional comparisons of different age groups can mistake cohort differences for age differences. Reason (R): A longitudinal design follows a single cohort and therefore cannot separate the effects of age from those of the time of measurement. Choose the correct option.

Both A and R are true, but R is not the correct explanation of A

Both A and R are true, and R is the correct explanation of A

Both A and R are true, but R is not the correct explanation of A

Research Methodology IV: Experimental Designs — Validity and its Threats, One-Way and Factorial ANOVA, Randomized Block, Repeated Measures and Latin Square Designs, Cohort and Time-Series Studies, MANOVA, ANCOVA and Single-Subject Designs

The last block of Unit 2 reads: "Experimental Designs: ANOVA [One-way, Factorial], Randomized Block Designs, Repeated Measures Design, Latin Square, Cohort studies, Time series, MANOVA, ANCOVA. Single-subject designs." A design is a plan for ruling out rival explanations, so this chapter begins with validity and the classic threats to it, then works a one-way ANOVA from sums of squares to F and effect size, reads main effects and interactions in a factorial design, shows how blocking, repeated measures and the Latin square each control a different source of error, places cohort and time-series studies, distinguishes MANOVA from ANCOVA, and ends with the logic of the A-B-A and multiple-baseline designs.

1. Validity and the threats to it

R. A. Fisher's three principles of experimental design are randomisation, replication and local control (blocking). Cook and Campbell distinguish four kinds of validity: internal (did the IV cause the change?), external (does it generalise?), construct (do the operations represent the intended constructs?) and statistical conclusion validity (were the statistics appropriate and powerful enough?). Campbell and Stanley (1963) listed eight threats to internal validity:

Campbell and Stanley's threats to internal validity
Threat	What happens
History	An outside event between pretest and posttest affects the DV.
Maturation	Participants change naturally with time — growing, tiring, becoming hungry.
Testing	Taking the pretest itself changes posttest scores (practice, sensitisation).
Instrumentation	The measuring instrument or observers change between measurements.
Statistical regression	Groups selected for extreme scores drift toward the mean on retest.
Selection	Groups differ before the treatment because they were not randomly formed.
Experimental mortality	Differential dropout from the groups.
Selection–maturation interaction	Non-equivalent groups mature at different rates.

Their designs fall into three families. Pre-experimental: the one-shot case study, the one-group pretest–posttest design and the static-group comparison — all vulnerable to most threats. True experimental: the pretest–posttest control-group design, the posttest-only control-group design and the Solomon four-group design, which adds unpretested groups to measure the effect of pretesting itself. Quasi-experimental: time-series and non-equivalent control-group designs, used when random assignment is impossible.

2. One-way and factorial ANOVA

Analysis of variance (Fisher) compares three or more means in one test, avoiding the inflation of Type I error that many t-tests produce: with three t-tests at α = 0.05, the chance of at least one false positive is 1 − 0.95³ ≈ 0.14. The total sum of squares is partitioned into between-groups and within-groups parts, SS_T = SS_B + SS_W, each divided by its degrees of freedom to give a mean square, and F = MS_B ÷ MS_W. If H₀ is true F is near 1. A significant F says only that some means differ; post hoc tests (Tukey's HSD, Scheffé, Bonferroni) say which. ANOVA assumes normality, homogeneity of variance (Levene's test) and independence.

Worked one-way ANOVA: three groups of five (N = 15). Critical F(2, 12) at 0.05 is 3.89, so F = 4.0 is significant; η² = 40 ÷ 100 = 0.40
Source	SS	df	MS	F
Between groups	40	k − 1 = 2	40 ÷ 2 = 20	20 ÷ 5 = 4.0
Within groups	60	N − k = 12	60 ÷ 12 = 5	—
Total	100	N − 1 = 14	—	—

A factorial design crosses two or more IVs so that every level of one occurs with every level of the other: a 2 × 3 design has six cells. It yields a main effect for each factor and an interaction — the effect of one factor depends on the level of the other, seen as non-parallel lines in a plot. When an interaction is significant, main effects must be interpreted with caution. Degrees of freedom: A has a − 1, B has b − 1, the A × B interaction has (a − 1)(b − 1), and within-cells error has N − ab. In a 2 × 3 design with 10 per cell (N = 60): df_A = 1, df_B = 2, df_A×B = 2 and df_within = 54.

⚠️ Interaction trumps main effect

If a drug helps men and harms women equally, there may be no main effect of drug at all — yet a large interaction. Reporting "the drug had no effect" would be wrong. Always read the interaction first.

3. Randomized block, repeated measures and Latin square designs

Three ways of controlling error
Design	How it works	What it controls
Randomized block	Participants are first grouped into homogeneous blocks on a nuisance variable (for example IQ bands), then randomly assigned to treatments within each block.	Removes the block variable's variance from the error term, increasing precision — Fisher's "local control".
Repeated measures (within-subjects)	The same participants experience every condition.	Individual differences, since each person is their own control; needs fewer participants and has more power. Its costs are order effects (practice, fatigue, carry-over), controlled by counterbalancing, and the sphericity assumption (Mauchly's test; Greenhouse–Geisser correction).
Latin square	A k × k arrangement in which each treatment appears exactly once in each row (for example participant group) and once in each column (for example order position).	Two nuisance variables at once, and order effects with only k sequences instead of k! — for four conditions, 4 orders rather than 24.

A 4 × 4 Latin square for conditions A, B, C and D can be written with rows ABCD, BCDA, CDAB and DABC: each letter occurs once in each position. A balanced Latin square additionally ensures that each condition immediately precedes every other condition equally often, which controls simple carry-over.

4. Cohort studies, time series, MANOVA and ANCOVA

Cohort studies follow a group sharing a defining experience (usually birth year) over time, prospectively. Cross-sectional comparisons of age groups confound age with cohort effects; longitudinal studies confound age with time of measurement. Schaie's cohort-sequential (sequential) designs follow several cohorts longitudinally to separate age, cohort and time effects.
Time-series designs take many measurements before and after an intervention; an interrupted time series looks for a change in level or slope at the point of intervention. It is quasi-experimental, and its main weakness is history.
MANOVA (multivariate ANOVA) tests group differences on two or more dependent variables at once, taking their intercorrelations into account and controlling the Type I error that separate ANOVAs would inflate; its test statistics are Wilks's lambda, Pillai's trace, Hotelling's trace and Roy's largest root.
ANCOVA (analysis of covariance) compares group means on a DV after statistically adjusting for a covariate — usually a pretest — measured before treatment. It reduces error variance (more power) and adjusts for pre-existing group differences. It assumes a linear covariate–DV relation and homogeneity of regression slopes across groups.

🧠 M before or C inside?

MANOVA: Many outcomes (several DVs). ANCOVA: a Covariate is taken out (one DV, adjusted). MANCOVA does both.

5. Single-subject designs

Single-subject (small-N) designs, rooted in Skinner's experimental analysis of behaviour, study one participant intensively with repeated measurement. The participant serves as their own control, comparing a stable baseline phase (A) with a treatment phase (B). Results are usually judged by visual analysis of changes in level, trend and variability across phases.

Single-subject designs
Design	Logic
A-B	Baseline then treatment; weak, because any change could be due to history.
A-B-A (reversal/withdrawal)	Treatment is withdrawn; if behaviour returns to baseline, the treatment is shown to be responsible.
A-B-A-B	Reinstates treatment, ending on a treatment phase — ethically preferable and a double demonstration.
Multiple baseline	Treatment is introduced at staggered times across behaviours, settings or participants; change that occurs only when treatment begins in each is the evidence. Used when a behaviour cannot or should not be reversed.
Changing criterion	The performance criterion for reinforcement is raised in steps, and behaviour tracks each new criterion.
Alternating treatments	Two or more treatments are rapidly alternated to compare them within one participant.

Key takeaways

Fisher's principles are randomisation, replication and local control; Campbell and Stanley's eight threats to internal validity are history, maturation, testing, instrumentation, regression, selection, mortality and selection–maturation interaction.
One-way ANOVA: F = MS_B ÷ MS_W with df k − 1 and N − k; η² = SS_B ÷ SS_T; post hoc tests locate the differences.
Factorial designs give main effects and interactions (df (a − 1)(b − 1)); interpret the interaction before the main effects.
Randomized blocks remove a nuisance variable, repeated measures remove individual differences at the cost of order effects, and a Latin square counterbalances k conditions in k orders.
MANOVA handles several DVs, ANCOVA adjusts one DV for a covariate, cohort-sequential designs separate age from cohort, and single-subject designs (A-B-A-B, multiple baseline) let one person serve as their own control.

Practice questions (10)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

In a one-way ANOVA with three groups of five participants each, SS between = 40 and SS within = 60. What is the F ratio? Type a number.
Numerical answer — type the value.
Show answer
Answer: 4
df between = k − 1 = 2 and df within = N − k = 15 − 3 = 12. MS between = 40 ÷ 2 = 20 and MS within = 60 ÷ 12 = 5, so F = 20 ÷ 5 = 4. It exceeds the critical F(2, 12) of 3.89 at the 0.05 level.
In the same ANOVA (SS between = 40, SS within = 60), what is the effect size eta squared (η²)? Type a number.
Numerical answer — type the value.
Show answer
Answer: 0.4
η² = SS between ÷ SS total = 40 ÷ (40 + 60) = 0.4: group membership accounts for 40 per cent of the variance in the DV, a large effect by Cohen's benchmark of 0.14.
In a 3 × 4 factorial ANOVA, how many degrees of freedom does the interaction term have? Type a number.
Numerical answer — type the value.
Show answer
Answer: 6
The interaction has (a − 1)(b − 1) = (3 − 1)(4 − 1) = 2 × 3 = 6 degrees of freedom; the main effects have 2 and 3.
A researcher assesses the same participants' anxiety, depression and stress (three dependent variables) and compares three therapy groups on all three at once. The appropriate analysis is:
1. MANOVA
2. ANCOVA
3. Kruskal–Wallis test
4. Chi-square test
Show answer
Answer: A — MANOVA
Several correlated dependent variables compared across groups in one analysis call for MANOVA. ANCOVA adjusts a single DV for a covariate, Kruskal–Wallis is a non-parametric one-DV test, and chi-square tests frequencies.
Which assumption is specific to ANCOVA?
1. Sphericity
2. Independence of observations
3. Homogeneity of regression slopes
4. Normality of the dependent variable
Show answer
Answer: C — Homogeneity of regression slopes
ANCOVA adjusts every group with one pooled regression line, so the covariate–DV slope must be the same in each group. Sphericity belongs to repeated-measures ANOVA; independence and normality apply to ANOVA in general.
A design in which each of four treatments appears once in every row and once in every column, so that order position and participant group are both controlled, is a:
1. Static-group comparison
2. Solomon four-group design
3. Multiple-baseline design
4. Latin square design
Show answer
Answer: D — Latin square design
The Latin square arranges k treatments in a k × k grid with each treatment once per row and column, controlling two nuisance variables. The Solomon design controls pretest sensitisation, multiple baseline is a single-subject design, and static-group comparison is pre-experimental.
Which of the following are threats to internal validity listed by Campbell and Stanley? Select all that apply.
1. Experimental mortality
2. Statistical regression
3. Maturation
4. Random assignment
Show answer
Answer: A — Experimental mortality; B — Statistical regression; C — Maturation
Regression toward the mean, maturation and differential dropout (mortality) are all on the list of eight. Random assignment is the main remedy for threats such as selection, not a threat.
A therapist wants to show that a token programme controls a child's self-injury but judges it unethical to withdraw treatment once it works. The most suitable design is:
1. A-B-A reversal design
2. Multiple-baseline design across settings
3. One-shot case study
4. Posttest-only control-group design
Show answer
Answer: B — Multiple-baseline design across settings
A multiple-baseline design demonstrates control by introducing treatment at staggered times without ever withdrawing it. The A-B-A design depends on withdrawal, the one-shot case study has no comparison, and a group design does not fit a single child.
Assertion (A): Cross-sectional comparisons of different age groups can mistake cohort differences for age differences. Reason (R): A longitudinal design follows a single cohort and therefore cannot separate the effects of age from those of the time of measurement. Choose the correct option.
1. Both A and R are true, and R is the correct explanation of A
2. Both A and R are true, but R is not the correct explanation of A
3. A is true, but R is false
4. A is false, but R is true
Show answer
Answer: B — Both A and R are true, but R is not the correct explanation of A
Both are true, but R does not explain A. A is about the cross-sectional design (age confounded with cohort); R describes a different weakness of the longitudinal design (age confounded with time of measurement). Schaie's sequential designs address both.
Which design adds two unpretested groups to the classic pretest–posttest control-group design in order to measure the effect of pretesting itself?
1. Changing-criterion design
2. Randomized block design
3. Interrupted time-series design
4. Solomon four-group design
Show answer
Answer: D — Solomon four-group design
The Solomon four-group design crosses pretest (yes/no) with treatment (yes/no), so the testing effect and its interaction with treatment can be estimated. The others do not manipulate pretesting.