Vector Spaces, Rank–Nullity, Decompositions and Quadratic Forms
Check your paper before you study this one. The engineering papers — Mechanical, Civil, Electrical, Instrumentation, Chemical — stop their linear algebra at eigenvalues and ask none of what follows. Electronics asks the first half: vector spaces, basis, linear dependence and independence, rank, and the existence and uniqueness of solutions. Computer Science adds LU decomposition. Data Science and AI asks all of it — vector spaces and subspaces, projection, orthogonal and idempotent matrices, quadratic forms, rank and nullity, LU and the singular value decomposition — and it is the highest-weighted mathematics topic in that paper. The organising idea is one number. Rank is the count of genuinely independent columns, and once you have it almost everything else follows without further work: nullity is columns minus rank, so a wide matrix always has a non-trivial null space; the columns are independent exactly when the rank equals their number; a square matrix is invertible exactly when its rank is full; and a homogeneous system has only the trivial solution exactly when the rank equals the number of unknowns. Rank + nullity = the number of COLUMNS, not rows, and reversing that is the single commonest error in the topic — it is a statement about the domain of the map, and the domain has one dimension per column. The special matrices are each defined by one equation and give up their eigenvalues immediately: an orthogonal matrix has QᵀQ = I, so its inverse is its transpose and its determinant is ±1; an idempotent matrix has A² = A, so its eigenvalues can only be 0 or 1, and therefore its trace equals its rank — a fact that answers most questions about projections in one line. A projection is the symmetric idempotent case, and projecting (3, 1) onto the line through (1, 1) lands on (2, 2) rather than anywhere on the original vector. Finally, a quadratic form is read through the symmetric matrix behind it, and the trap is in the writing rather than the mathematics: in 2x² + 2xy + 2y² the off-diagonal entries are half the cross coefficient, so the matrix is [[2, 1], [1, 2]] and not [[2, 2], [2, 2]]. Get that halving right and definiteness is just the sign of the eigenvalues, or equivalently of the leading principal minors.
LU, and the Matrices Defined by One Equation
LU decomposition writes A = LU with L unit lower-triangular and U upper-triangular, and it is nothing more than Gauss elimination with the multipliers kept. For A = [[2, 1], [4, 3]] the elimination subtracts twice the first row from the second, so L = [[1, 0], [2, 1]] and U = [[2, 1], [0, 1]] — and multiplying them back returns A, which is the check to run. Two consequences are what GATE actually asks: the multipliers are the entries of L, and the product of U's diagonal is the determinant, since det A = det L · det U = 1 · (product of pivots).
| Kind | Defined by | What it gives you free |
|---|---|---|
| Orthogonal | QᵀQ = I | Q⁻¹ = Qᵀ, det Q = ±1, columns orthonormal, and lengths preserved: |Qx| = |x|. |
| Idempotent | A² = A | Eigenvalues are 0 or 1 only, so trace = rank. [[2, −2], [1, −1]] is idempotent with trace 1 and rank 1. |
| Projection | A² = A and Aᵀ = A | The symmetric idempotent case. Onto the line through (1, 1) it is ½[[1, 1], [1, 1]], and it sends (3, 1) to (2, 2). |
🧠 Singular values, without computing an SVD
The singular values of A are the square roots of the eigenvalues of AᵀA, listed largest first, and they are always real and non-negative even when A has complex eigenvalues or is not square. For A = [[3, 0], [0, −2]], AᵀA = [[9, 0], [0, 4]] and the singular values are 3 and 2 — note that they are not the eigenvalues 3 and −2, and the difference is exactly the point: a singular value measures stretching, which has no sign. The count of non-zero singular values is the rank, so a rank-deficient matrix has a zero among them: [[2, 0], [0, 0]] has singular values 2 and 0, and rank 1.Quadratic Forms and Definiteness
Every quadratic form q(x) = xᵀAx is written with a symmetric A, and building that A is where the marks are lost. The diagonal entries are the coefficients of the squares; each off-diagonal entry is half the coefficient of the corresponding cross term, because the term appears twice in the product. So 2x² + 2xy + 2y² has A = [[2, 1], [1, 2]], and x² + 4xy + y² has A = [[1, 2], [2, 1]].
- By eigenvalues. All positive → positive definite; all negative → negative definite; mixed signs → indefinite; some zero and the rest of one sign → semidefinite. [[2, 1], [1, 2]] has eigenvalues 3 and 1, so 2x² + 2xy + 2y² is positive definite.
- By leading principal minors, which is faster for a 2×2 or 3×3: positive definite exactly when every leading minor is positive. For [[2, 1], [1, 2]] they are 2 and 3 — both positive. For [[1, 2], [2, 1]] they are 1 and −3, so it is indefinite, and its eigenvalues 3 and −1 agree.
- A real symmetric matrix always has real eigenvalues and an orthogonal set of eigenvectors, which is why definiteness is a well-posed question at all — the eigenvalues cannot be complex, so their signs are always comparable.
Key takeaways
- Check your paper first: the engineering papers stop at eigenvalues, EC asks the vector-space half, CS adds LU, and DA asks all of it at the highest mark weight in that paper.
- Rank + nullity = the number of COLUMNS. A 3×4 matrix of rank 2 has nullity 2, and using the row count is the error that survives longest because it is right for every square matrix.
- One rank answers five questions: independence, nullity, non-trivial solutions, invertibility, and the number of non-zero singular values.
- LU is Gauss elimination with the multipliers kept: they are the entries of L, and the product of U’s diagonal is the determinant.
- Idempotent means A² = A, so the eigenvalues are 0 or 1 and trace = rank. A projection is the symmetric case, and it lands a vector somewhere new rather than on itself.
- Singular values are the square roots of the eigenvalues of AᵀA — always real and non-negative, so 3 and 2 for a matrix whose eigenvalues are 3 and −2.
- In a quadratic form the off-diagonal entry is HALF the cross coefficient. Get the halving right and definiteness is just the sign of the eigenvalues, or of the leading principal minors.
Practice questions (9)
Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.
A matrix A of size 3×4 has rank 2. Its nullity is:
Show answer
Answer: B — 2
Rank + nullity = the number of columns, so nullity = 4 − 2 = 2. Option A is what the row count gives — 3 − 2 = 1 — and it is the answer a candidate reaches by remembering the theorem as being about the matrix rather than about the map. The map sends 4-vectors to 3-vectors, and the identity is a statement about its domain, so the four columns are what must be accounted for. As a check that costs nothing: a 3×4 matrix can never have rank above 3, so its nullity is at least 1 whatever else is true, and a wide matrix always has a non-trivial null space.The vectors (1, 2, 3), (4, 5, 6) and (7, 8, 9) are:
Show answer
Answer: B — Linearly dependent, spanning a 2-dimensional subspace
Stack them as rows and take the determinant: it is 0, so they are dependent and option A is out. The rank tells you how dependent — row-reducing gives rank 2, so they span a plane, not a line. The dependence is visible without any arithmetic once you look for it: the third vector is twice the second minus the first, and equally the three are in arithmetic progression, which always forces the middle one to be the average of the other two. Option C would need rank 1, which happens only when every vector is a multiple of one other — true of (1, 2, 3), (2, 4, 6), (3, 6, 9) and not of these. Option D fails on the first dot product, 1·4 + 2·5 + 3·6 = 32.For A = [[2, 1], [4, 3]], the LU decomposition A = LU with unit lower-triangular L gives:
Show answer
Answer: A — L = [[1, 0], [2, 1]], U = [[2, 1], [0, 1]]
Eliminate: the multiplier that clears the 4 below the pivot 2 is 4/2 = 2, and subtracting twice the first row from the second gives U = [[2, 1], [0, 1]]. The multiplier is the entry of L, so L = [[1, 0], [2, 1]]. Multiply back to check: [[1, 0], [2, 1]]·[[2, 1], [0, 1]] = [[2, 1], [4, 3]] = A. Option B uses the entry 4 itself instead of the multiplier 4/2, which is the standard slip. Option C breaks the "unit" condition — L must have 1s on its diagonal, and the question said so. Two things worth carrying: the product of U's diagonal, 2 × 1 = 2, is det A; and an LU decomposition can genuinely fail to exist without row interchanges, which is why partial pivoting exists.A is an idempotent matrix (A² = A) with trace 1. Its rank is:
Show answer
Answer: B — 1
If A² = A then every eigenvalue λ satisfies λ² = λ, so λ is 0 or 1 — no other value is possible. The trace is the sum of the eigenvalues, which is therefore just the count of 1s; and an idempotent matrix is diagonalisable, so that count is also the rank. Trace 1 gives rank 1. Option D is the tempting answer because the trace does not usually determine the rank — it is exactly idempotence that makes it do so here, and noticing which hypothesis is doing the work is the skill being tested. [[2, −2], [1, −1]] is a concrete instance: it squares to itself, has trace 1, and has rank 1.The quadratic form q(x, y) = 2x² + 2xy + 2y² is:
Show answer
Answer: A — Positive definite
Build the symmetric matrix, halving the cross coefficient: A = [[2, 1], [1, 2]], not [[2, 2], [2, 2]]. Its leading principal minors are 2 and det = 4 − 1 = 3, both positive, so the form is positive definite; the eigenvalues 3 and 1 agree. The halving is the whole question. With the cross coefficient left unhalved the matrix would be [[2, 2], [2, 2]], whose determinant is 0 and whose minors would suggest semidefinite — option D, which is there precisely to catch that. A quick sanity check that costs nothing: q(1, −1) = 2 − 2 + 2 = 2 > 0, and a form that is positive at the point most likely to make it negative is unlikely to be indefinite.Q is a real orthogonal matrix. Which of the following must be true?
Show answer
Answer: B — det Q = ±1
From QᵀQ = I take determinants: det(Qᵀ)·det(Q) = 1, and det(Qᵀ) = det(Q), so (det Q)² = 1 and det Q = ±1. Option A is too strong — a reflection is orthogonal with determinant −1, and only the rotations have +1. Option C is false: [[0, 1], [−1, 0]] is orthogonal and not symmetric. Option D is false for the same matrix, whose eigenvalues are ±i — an orthogonal matrix preserves lengths, so its eigenvalues have modulus 1, but they need not be real at all. The property worth carrying beyond the determinant is that Q⁻¹ = Qᵀ, which makes an orthogonal matrix the cheapest inverse in linear algebra.A is a real m×n matrix. Which of the following are always true? (More than one option may be correct.)
Show answer
Answer: A — rank(A) + nullity(A) = n; B — The number of non-zero singular values of A equals rank(A); C — rank(A) ≤ min(m, n)
Three hold for every real matrix. A is the rank–nullity theorem, counting columns. B is the definition of rank read off a singular value decomposition, and it is the reason an SVD is used to estimate rank numerically. C holds because the rank cannot exceed either the number of independent rows or the number of independent columns. Only D fails, and it fails backwards: m < n means fewer equations than unknowns, so the rank is at most m < n, the nullity is at least n − m > 0, and the system must have a non-trivial solution. A wide matrix always has one, which is the useful form of the statement and the exact opposite of what option D claims.The largest singular value of A = [[3, 0], [0, −2]] is _____.
Numerical answer — type the value.
Show answer
Answer: 3
Singular values are the square roots of the eigenvalues of AᵀA. Here AᵀA = [[9, 0], [0, 4]], whose eigenvalues are 9 and 4, so the singular values are 3 and 2 and the largest is 3. Note what did not happen: the eigenvalues of A itself are 3 and −2, and the singular values are 3 and 2. A singular value measures how much the matrix stretches a direction, and stretching has no sign, so singular values are always real and non-negative even when the eigenvalues are negative or complex or the matrix is not square. For this diagonal matrix the two happen to agree in magnitude; for a general matrix they need not be related at all.The nullity of the matrix [[1, 2, 1], [2, 4, 2], [3, 6, 3]] is _____.
Numerical answer — type the value.
Show answer
Answer: 2
Every row is a multiple of the first, so the rank is 1 — spotted rather than computed. The matrix has 3 columns, so the nullity is 3 − 1 = 2. Two checks that cost nothing and catch a slip: the null space really is two-dimensional, since the single equation x + 2y + z = 0 defines a plane through the origin in ℝ³; and the eigenvalues must be 0, 0 and the trace 1 + 4 + 3 = 8, which is consistent with a rank-1 matrix having exactly one non-zero eigenvalue. Recognising proportional rows before reducing is worth the two seconds it takes — it turns the whole question into subtraction.