Rank, Nullity and Column Space: Understanding What a Matrix Preserves and Loses
A matrix transforms input vectors into output vectors. Some independent directions survive, some are combined, and some may disappear completely into the zero vector. Span, dimension, column space, rank, null space and nullity describe this process precisely.
input space → matrix transformation → column space
1. Span: all reachable combinations
Given vectors v₁, v₂, …, vₖ, their span is the set of every vector that can be made by scaling and adding them:
Span{v₁, v₂, …, vₖ}
= {c₁v₁ + c₂v₂ + ⋯ + cₖvₖ : c₁, c₂, …, cₖ ∈ ℝ}.
One non-zero direction spans a line through the origin. Two independent directions span a plane through the origin. Three independent directions in ℝ³ span all of ℝ³.
2. Independence and dimension
Vectors are linearly independent when none of them can be constructed from the others. Each independent vector contributes a genuinely new direction. A dependent vector contributes no new direction because it already lies in the span of the others.
dimension = number of independent directions
A line has dimension 1, a plane has dimension 2, and ordinary three-dimensional space has dimension 3. Dimension counts directions, not the infinitely many individual vectors lying along those directions.
3. Matrix columns generate every output
Let an m × n matrix M have the columns c₁, c₂, …, cₙ. An input vector therefore has n components:
M = [c₁ c₂ ⋯ cₙ],
x = (x₁, x₂, …, xₙ)ᵀ.
Matrix multiplication forms the output by using the input components as weights on the columns:
Mx = x₁c₁ + x₂c₂ + ⋯ + xₙcₙ.
This identity is the key. Every output of M is a linear combination of its columns. Conversely, every linear combination of the columns is produced by some choice of x₁, x₂, …, xₙ.
4. Column space: the complete output territory
The column space of M is the set of every output vector that M can produce. Since every output is a combination of the columns,
Col(M) = Span{c₁, c₂, …, cₙ}.
The column space lies inside the output space ℝᵐ. If the columns produce only one independent direction, the column space is a line. If they produce two independent directions, it is a plane. Dependent columns do not enlarge it.
5. Rank: the dimension of the output territory
Rank counts the independent directions in the column space:
rank(M) = dim(Col(M)).
Rank 1 means all outputs lie along one independent direction. Rank 2 means the outputs occupy two independent directions. Rank 3 means three independent output directions are available. Rank counts independent columns, not merely the total number of columns.
6. The null equation
The equation
Mx = 0
asks which input vectors disappear under the transformation. The zero vector is always one solution. Depending on M, there may also be infinitely many non-zero solutions.
7. Row reduction, pivots and free variables
Row reduction replaces equations with simpler but equivalent equations, so the solution set remains unchanged. It uses three reversible operations: swapping two rows, multiplying a row by a non-zero number, and adding a multiple of one row to another row.
After row reduction, the variables separate into two groups:
| Variable type | Meaning |
|---|---|
| Pivot variable | Controlled by an equation and determined by the free variables. |
| Free variable | May be chosen freely and creates one independent solution direction. |
number of pivots = rank(M)
number of free variables = nullity(M)
8. Null space and nullity
The null space is the complete set of input vectors sent to zero:
Null(M) = {x : Mx = 0}.
If the independent null-space directions are v₁, v₂, …, vₖ, then every null vector is a linear combination of them:
Null(M) = Span{v₁, v₂, …, vₖ}.
Nullity is not the number of individual vectors sent to zero; that set can be infinite. Nullity is the number of independent directions within that set:
nullity(M) = dim(Null(M)).
- Nullity 0: only the zero vector is sent to zero.
- Nullity 1: a line of input vectors is sent to zero.
- Nullity 2: a plane of input vectors is sent to zero.
9. One example connecting every idea
Consider a transformation from ℝ³ to ℝ² defined by
M(x, y, z)ᵀ = (x + 2y + z, z)ᵀ.
Reading off the coefficients gives the three columns
c₁ = (1, 0)ᵀ, c₂ = (2, 0)ᵀ, c₃ = (1, 1)ᵀ.
The second column is twice the first, so it adds no new direction. The first and third columns are independent and span ℝ². Therefore,
Col(M) = Span{(1, 0)ᵀ, (1, 1)ᵀ} = ℝ²,
rank(M) = 2.
To find which inputs disappear, set the output equal to zero:
The pivot variables are x and z. The variable y is free. Let y = t. Then z = 0 and x = −2t, so
(x, y, z)ᵀ = t(−2, 1, 0)ᵀ.
Thus the null space is a line with one independent direction:
Null(M) = Span{(−2, 1, 0)ᵀ},
nullity(M) = 1.
10. Why rank plus nullity equals the number of columns
If M has n columns, its input vector contains n variables. After row reduction, every input variable is either a pivot variable or a free variable. There is no third category.
pivot variables + free variables = all input variables.
The number of pivots is the rank. The number of free variables is the nullity. The total number of input variables is the number n of columns. Therefore,
rank(M) + nullity(M) = n.
In the example, the input has three variables. Two independent input directions survive as independent output directions, while one independent input direction disappears into zero:
2 + 1 = 3.
Summary
| Concept | Meaning |
|---|---|
| Span | Every linear combination of a set of vectors. |
| Dimension | The number of independent directions in a space. |
| Column space | Every output obtainable by combining the columns of M. |
| Rank | The dimension of the column space. |
| Pivot variable | A variable controlled by an equation. |
| Free variable | A variable that may be chosen freely. |
| Null space | Every input vector satisfying Mx = 0. |
| Nullity | The dimension of the null space. |
Rank measures the independent directions that appear in the output. Nullity measures the independent directions that vanish. Together, they account for the entire input space.
