Research Methodology III: Product-Moment, Rank-Order, Partial and Multiple Correlation; the Special Correlations — Biserial, Point-Biserial, Tetrachoric and Phi; Simple and Multiple Regression; and Factor Analysis — Worked
2. Partial and multiple correlation
Partial correlation is the correlation between two variables with the influence of a third held constant (removed from both): r₁₂.₃ = (r₁₂ − r₁₃r₂₃) ÷ √[(1 − r₁₃²)(1 − r₂₃²)]. Worked: if intelligence (1) and achievement (2) correlate 0.6, and each correlates 0.6 with socio-economic status (3), then r₁₂.₃ = (0.6 − 0.36) ÷ √(0.64 × 0.64) = 0.24 ÷ 0.64 = 0.375 — much of the original 0.6 was carried by SES. (A semi-partial or part correlation removes the third variable from only one of the two.)
Multiple correlation R is the correlation between one criterion and the best linear combination of two or more predictors; it runs from 0 to 1 and R² is the proportion of criterion variance the predictors jointly explain. For two predictors, R₁.₂₃ = √[(r₁₂² + r₁₃² − 2r₁₂r₁₃r₂₃) ÷ (1 − r₂₃²)]. Worked: if the criterion correlates 0.3 and 0.4 with two predictors that are uncorrelated with each other (r₂₃ = 0), R² = 0.09 + 0.16 = 0.25 and R = 0.5. When predictors are correlated, R² is less than the sum of the squared correlations because they share predictive variance.
3. The special correlations: which fits which variables
The special correlations exist because variables are not always continuous. The decisive distinction is between a true (genuine) dichotomy — a variable that really has only two values, such as sex recorded as male/female or an item answered right/wrong — and an artificial dichotomy, a continuous and normally distributed variable that has been cut into two, such as pass/fail on an underlying score or "tall/short".
| Coefficient | Variable 1 | Variable 2 | Typical use |
|---|---|---|---|
| Point-biserial (r_pb) | True dichotomy | Continuous | Item (right/wrong) with total test score — item discrimination; it is simply Pearson's r with one variable coded 0/1 |
| Biserial (r_b) | Artificial dichotomy (underlying normal continuum) | Continuous | Pass/fail on a cut-off with anxiety scores; estimates the r that would be found if the dichotomised variable were measured continuously; always larger than r_pb for the same data |
| Tetrachoric (r_t) | Artificial dichotomy | Artificial dichotomy | Above/below median on two tests; estimates the correlation of the two underlying normal variables from a 2 × 2 table |
| Phi (φ) | True dichotomy | True dichotomy | Sex with yes/no answers; two items scored right/wrong — Pearson's r on two 0/1 variables |
Phi worked. In a 2 × 2 table with cells a = 30, b = 10 (top row) and c = 10, d = 30 (bottom row), φ = (ad − bc) ÷ √[(a + b)(c + d)(a + c)(b + d)] = (900 − 100) ÷ √(40 × 40 × 40 × 40) = 800 ÷ 1,600 = 0.5. φ is related to chi-square by χ² = Nφ², here 80 × 0.25 = 20. Unlike r, the maximum value of φ is less than 1 unless both variables split in the same proportions.
4. Simple and multiple regression
Simple linear regression predicts Y from X with the least-squares line Y′ = a + bX, where the slope b = r(s_Y ÷ s_X) and the intercept a = M_Y − bM_X. The accuracy of prediction is given by the standard error of estimate, s_Y√(1 − r²). Worked: with r = 0.6, s_Y = 10, s_X = 5, M_X = 20 and M_Y = 50, b = 0.6 × 2 = 1.2 and a = 50 − 1.2 × 20 = 26, so Y′ = 26 + 1.2X; a person with X = 25 is predicted to score 26 + 30 = 56. The standard error of estimate is 10 × √0.64 = 8.
Multiple regression predicts one criterion from several predictors: Y′ = a + b₁X₁ + b₂X₂ + … . Standardised coefficients (β weights) show each predictor's unique contribution with the others held constant, and R² the variance explained jointly. Predictors can be entered all at once (simultaneous), in an order set by theory (hierarchical, testing the change in R²), or by a statistical rule (stepwise). Multicollinearity — predictors highly correlated with each other — makes the weights unstable, and is checked with tolerance and the variance inflation factor.
5. Factor analysis: assumptions, methods, rotation and interpretation
| Stage | What is done |
|---|---|
| Assumptions | Interval-level variables, linear relations, an adequate sample (a common rule is at least 5–10 cases per variable and 200 or more overall), enough correlation to factor — Bartlett's test of sphericity significant and the Kaiser–Meyer–Olkin measure of sampling adequacy above about 0.6 — and no singularity. |
| Extraction methods | Principal components (analyses total variance), principal-axis factoring (common variance only, communalities on the diagonal), maximum likelihood, and Thurstone's historical centroid method. |
| How many factors | Kaiser's criterion (eigenvalues greater than 1), Cattell's scree test (the "elbow"), and parallel analysis. |
| Rotation | To reach Thurstone's simple structure (each variable loading highly on one factor). Orthogonal rotation — varimax (Kaiser), quartimax, equamax — keeps factors uncorrelated; oblique rotation — direct oblimin, promax — allows them to correlate, which is often more realistic for psychological constructs. |
| Interpretation | A loading is the correlation of a variable with a factor (|0.30| or |0.40| and above is usually treated as salient); a variable's communality h² is the sum of its squared loadings across factors; a factor's eigenvalue is the sum of squared loadings down the variables; and the percentage of variance a factor explains is its eigenvalue ÷ the number of variables × 100. |
Worked: a test loads 0.6 on a verbal factor and 0.5 on a reasoning factor after orthogonal rotation, so its communality is 0.36 + 0.25 = 0.61 — 61 per cent of its variance is common variance, and 39 per cent is unique (specific plus error). If the first of two factors in a ten-variable analysis has an eigenvalue of 3.0, it explains 3.0 ÷ 10 = 30 per cent of the total variance. Exploratory factor analysis discovers structure; confirmatory factor analysis tests a structure specified in advance and is judged by fit indices within structural equation modelling.
Key takeaways
- Pearson r = Σxy ÷ √(Σx²Σy²) measures linear association and r² the shared variance; Spearman ρ = 1 − 6Σd² ÷ [n(n² − 1)] works on ranks.
- Partial correlation removes a third variable from both of two others; multiple R correlates a criterion with the best combination of predictors, and with uncorrelated predictors R² is the sum of the squared r's.
- Point-biserial: one true dichotomy and one continuous variable; biserial: one artificial dichotomy; tetrachoric: two artificial dichotomies; phi: two true dichotomies, with φ = (ad − bc) ÷ √[(a + b)(c + d)(a + c)(b + d)].
- In regression b = r(s_Y ÷ s_X), a = M_Y − bM_X, and the standard error of estimate is s_Y√(1 − r²); multiple regression weights predictors and suffers from multicollinearity.
- Factor analysis needs adequate correlations (Bartlett, KMO) and sample size, extracts factors by eigenvalue or scree, rotates orthogonally (varimax) or obliquely (oblimin) to simple structure, and reads loadings, communalities and eigenvalues.
Practice questions (10)
Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.
A researcher wants to correlate students' sex (recorded as male or female) with their scores on a continuous aptitude test. The appropriate coefficient is:
Show answer
Answer: B — Point-biserial correlation
Sex recorded this way is a true dichotomy, and the other variable is continuous: that combination calls for the point-biserial r. Biserial is for an artificial dichotomy, and tetrachoric and phi need two dichotomies.Candidates are classified as "pass" or "fail" by a cut-off on a normally distributed ability score, and this classification is correlated with a continuous anxiety score. The appropriate coefficient is:
Show answer
Answer: C — Biserial correlation
Pass/fail created by cutting a continuous, normal variable is an artificial dichotomy; with a continuous second variable the biserial r applies, estimating the correlation the underlying continuum would show. Point-biserial is for true dichotomies.Which of the following coefficients are computed from a 2 × 2 table of two dichotomised variables? Select all that apply.
Show answer
Answer: A — Phi coefficient; B — Tetrachoric correlation
Tetrachoric (two artificial dichotomies) and phi (two true dichotomies) both use a four-cell table. Biserial pairs a dichotomy with a continuous variable, and Spearman's rho uses ranks.In a 2 × 2 table the cell frequencies are a = 30, b = 10, c = 10 and d = 30. Compute the phi coefficient. Type a number.
Numerical answer — type the value.
Show answer
Answer: 0.5
φ = (ad − bc) ÷ √[(a + b)(c + d)(a + c)(b + d)] = (900 − 100) ÷ √(40 × 40 × 40 × 40) = 800 ÷ 1,600 = 0.5.Six students are ranked by two judges, and the sum of squared differences between their ranks is Σd² = 14. What is Spearman's rank-order correlation? Type a number.
Numerical answer — type the value.
Show answer
Answer: 0.6
ρ = 1 − 6Σd² ÷ [n(n² − 1)] = 1 − (6 × 14) ÷ (6 × 35) = 1 − 84 ÷ 210 = 1 − 0.4 = 0.6.Variables 1 and 2 correlate 0.6, and each correlates 0.6 with variable 3. Compute the partial correlation r₁₂.₃. Type the value to three decimal places.
Numerical answer — type the value.
Show answer
Answer: 0.375
r₁₂.₃ = (r₁₂ − r₁₃r₂₃) ÷ √[(1 − r₁₃²)(1 − r₂₃²)] = (0.6 − 0.36) ÷ √(0.64 × 0.64) = 0.24 ÷ 0.64 = 0.375. Holding variable 3 constant reduces the association from 0.6 to 0.375.In a regression of Y on X, r = 0.6, the SD of Y is 10, the SD of X is 5, the mean of X is 20 and the mean of Y is 50. What is the predicted Y for a person with X = 25? Type a number.
Numerical answer — type the value.
Show answer
Answer: 56
b = r(s_Y ÷ s_X) = 0.6 × (10 ÷ 5) = 1.2; a = M_Y − bM_X = 50 − 1.2 × 20 = 26; Y′ = 26 + 1.2 × 25 = 26 + 30 = 56.After orthogonal rotation, a test loads 0.6 on Factor I and 0.5 on Factor II and negligibly on all others. What is its communality (h²)? Type a number.
Numerical answer — type the value.
Show answer
Answer: 0.61
Communality is the sum of a variable's squared loadings across the factors: 0.6² + 0.5² = 0.36 + 0.25 = 0.61. The remaining 0.39 is unique variance (specific plus error).Which rotation method allows the extracted factors to be correlated with one another?
Show answer
Answer: D — Direct oblimin
Direct oblimin (like promax) is an oblique rotation, so factors may correlate. Varimax, quartimax and equamax are orthogonal and keep the factors uncorrelated.Assertion (A): Kaiser's criterion retains only factors with eigenvalues greater than 1. Reason (R): With standardised variables each contributes a variance of 1, so a factor with an eigenvalue below 1 explains less variance than a single original variable. Choose the correct option.
Show answer
Answer: A — Both A and R are true, and R is the correct explanation of A
Both are true and R explains A: a factor is worth keeping only if it summarises more than one variable's worth of variance, which is exactly an eigenvalue above 1.