Understanding Matrices 3: A Few Words on Determinants

《新理解矩阵3》：行列式的点滴 苏剑林 · 2012-11-04 · Mathematics algebra matrix linear

The latest version of this article is at: http://kexue.fm/archives/2208/

Dear readers, the "Understanding Matrices" series on Scientific Spaces has now reached what I, BoJone, consider the most exciting part: the discussion of determinants. This material was not covered in Meng Yan's article; it is a set of views I arrived at by drawing on various books and online resources. The main book involved was The Mathematical Bridge, and tracing things back to their origin, what actually pushed me to look into this topic was matrix67's essay "How Should Textbooks Be Written." This post contains quite a lot of intuitive material — views that may not be the orthodox ones, but which, I believe, can at least to some extent help deepen our understanding of linear algebra.

Most linear algebra courses introduce determinants via systems of linear equations. This approach lets students quickly master the computation and gives the most practical application right away (namely, solving equations). But it can easily lead readers into a trap, making them think that linear algebra is fundamentally about solving systems of equations. This does not let readers truly grasp its essence — and it's only once we have an intuitive, well-practiced feel for the subject that we can really put it to use.

Determinants actually arose in order to determine whether a matrix is invertible. They construct, by some means, a "relatively simple" function to serve this purpose, and that function is the determinant of the matrix. Let's reflect for a moment on what it means for a matrix to be invertible. As mentioned before, a matrix is a transformation from one point to another, so the inverse matrix is clearly meant to transform it back. We also said that "motion is relative," and that the transformation of points can equally be realized as a transformation of coordinate systems. But intuitively, apart from differences in computational complexity (for instance, computing a dot product in a general affine coordinate system is more complicated than in a Cartesian one), different coordinate systems shouldn't differ from one another in any other way. In physical language, all coordinate systems are on equal footing. So, given one coordinate system, we should naturally be able to transform to another one, and just as naturally transform back. Since a matrix is a description of such a coordinate system, the only possible reason a matrix could fail to be invertible is:

The $n$ column vectors of this $n$-order matrix simply fail to form a coordinate system for a $n$-dimensional space.

In other words, they aren't qualified to be a coordinate system! (In technical terms, these $n$ vectors are linearly dependent.)

So we want to find a way to construct, for some $n$-order matrix $A$, a function $|A|$ such that when $|A|$ equals zero, it means this matrix is not qualified to be a coordinate system — that is, it is not invertible.

Let $A=[\vec{c_1},\vec{c_2},...,\vec{c_n}]$, where $\vec{v_i}$ are the individual column vectors. Let's think about which cases obviously disqualify matrix A from being a coordinate system.

1. Clearly, whether or not a set of vectors can form a coordinate system depends on every single vector $\vec{v_i}$, so $|A|$ needs to be a function of each vector $\vec{v_i}$ in order to satisfy this requirement.
2. For simplicity, we'd like our function to be linear, that is, $A=[\vec{c_1},...,\vec{v_i},...,\vec{c_n}]$, $A'=[\vec{c_1},...,\alpha \vec{v_i},...,\vec{c_n}]$, so $|A'|=\alpha |A|$.
3. Forming an n-dimensional coordinate system requires $n$ independent vectors; if, among the $n$ vectors of matrix $A$, two happen to coincide, then it obviously cannot form a coordinate system, meaning $|A|=0$ in this case;
4. To fix an appropriate scaling factor, and to match our intuition, we require $|I|=1$.

Remarkably, these three properties alone are enough to pin down a single unique function $|A|$ — and that function is exactly the determinant formula given in our textbooks! The derivation is admittedly a bit involved, so I won't give it here; readers who are interested can work it out by hand for 2×2 and 3×3 matrices, and then generalize by analogy.

Let's recall how textbooks usually introduce determinants: first the computation formula for the determinant is given via systems of linear equations, and then various properties are proved — for instance, "multiplying one row by 5 multiplies the value of the determinant by 5" and so on. Compared to the account above (which, in fact, is the standard way linear algebra is presented), this feels rather like putting the cart before the horse. The determinant itself is defined by these properties, so why on earth would we want to derive those properties backwards from the computation formula? Aside from giving students practice in arithmetic, I honestly can't see what purpose this serves.

There's also a highly intuitive geometric way to understand all of this.

What is the necessary and sufficient condition for three vectors in three-dimensional space to form an affine coordinate system? Very simple: the three vectors must not be coplanar! Generalizing to n-dimensional space, we clearly have:

$n$ vectors forming a coordinate system means that these vectors are not "coplanar"!

Of course, in $n$-dimensional space, "plane" here doesn't mean a two-dimensional plane, but rather something $n-1$-dimensional, standing in contrast to the $n$-dimensional volume.

We can also phrase it another way:

$n$ vectors forming a coordinate system means that the $n$-dimensional parallelepiped (think of our familiar parallelepiped) spanned by these vectors has nonzero volume!

Note carefully: this brings in volume, and whether it's zero or not is tied to the invertibility of the matrix — exactly the same story as with the determinant. Could this n-dimensional volume be related to the determinant? Let's analyze some properties of the volume $V$ of the $n$-dimensional solid spanned by matrix $A$:

1. Obviously, each vector $\vec{v_i}$ is also one of the edge lengths, so V should be a function of every vector $\vec{v_i}$;
2. When some vector is scaled to $\alpha$ times its original length, the volume should also scale to $|\alpha|$ times its original value — the absolute value sign is there because vectors can extend in the opposite direction, but volume must always be non-negative;
3. If two of the vectors coincide, it means the vectors are "coplanar," in which case the volume is naturally 0;
4. The volume of the unit n-dimensional cube is naturally 1.

Surprisingly, these properties correspond almost exactly to those of the determinant, except that in property 2 the factor $\alpha$ needs an absolute value. So these properties ultimately force the volume of this n-dimensional solid to equal $V=ABS(|A|)$ — I use ABS here to denote absolute value, just to avoid confusion.

This really is a wonderful fact: the determinant has such a clear geometric meaning that it can be used to compute volume (in the 2×2 case, it's the area of a parallelogram)! This immediately explains why the formula for the volume of a parallelepiped, which we first learn in analytic geometry, takes the form it does. It can be obtained entirely without any reference to dot products or cross products! There are many more examples like this that readers will naturally encounter as they continue learning. But on the other hand, this also reflects rather poorly on our textbooks: if we stick to the conventional way of presenting determinants, how much extra effort would it take just to prove that the determinant is the volume?

These are just my own shallow views — I welcome anyone to point out mistakes.