Revisiting the Determinant of Non-Square Matrices

再谈非方阵的行列式 苏剑林 · 2018-10-16 · Mathematics vector matrix determinant

A few years ago, based on my own rudimentary understanding of matrices, I wrote a series called "Understanding Matrices." One post in that series, Why Do Only Square Matrices Have Determinants?, discussed the question of determinants for non-square matrices, and concluded that "the determinant of a non-square matrix isn't very elegant" and "the determinant of a square matrix is enough." This post revisits that question.

Let's first recall the determinant of a square matrix. The most important value of the determinant really lies in its geometric meaning:

The absolute value of the determinant of an n-dimensional square matrix equals the n-dimensional hyper-volume spanned by its row (or column) vectors.

This geometric meaning is the source of everything important about determinants; for related discussion see Bits and Pieces on Determinants, which also serves as the foundation for our discussion of non-square determinants here.

Analysis

For a square matrix $\boldsymbol{A}_{n\times n}$, we can view it either as a combination of $n$ row vectors or as a combination of $n$ column vectors. Either way, the absolute value of the determinant equals the hyper-volume of the $n$-dimensional solid spanned by these $n$ vectors. In other words, for a square matrix, the distinction between rows and columns doesn't change the determinant.

For a non-square matrix $\boldsymbol{B}_{n \times k}$, things are different. Without loss of generality, assume $n > k$. We can view it either as a combination of $n$ row vectors of dimension $k$, or as a combination of $k$ column vectors of dimension $n$. The determinant of a non-square matrix should carry the same meaning: the hyper-volume of the solid these vectors span.

Let's look at the first case: if we regard it as $n$ row vectors of dimension $k$, then we'd need to treat it as the hyper-volume of the $n$-dimensional solid spanned by these $n$ vectors. But note that $n > k$, so these $n$ vectors must be linearly dependent — they simply cannot span a $n$-dimensional solid at all. At best they span a $n-1$-dimensional solid or something of even lower dimension, and consequently the $n$-dimensional hyper-volume is naturally zero.

The second case, however, is not so trivial. If we regard it as $k$ column vectors of dimension $n$, then although these $k$ vectors live in $n$-dimensional space, what they span is a $k$-dimensional solid, and the hyper-volume of this $k$-dimensional solid need not be zero. Let's just take this non-trivial volume as our definition of the determinant of a non-square matrix.

Definition

For this second case, there's a rather clever definition that leverages the determinant of a square matrix:

\begin{equation}|\det \boldsymbol{B}| = \sqrt{\det (\boldsymbol{B}^{\top}\boldsymbol{B})}\label{eq:dingyi}\end{equation}

Of course, this only defines the absolute value of the determinant, but that's already sufficient in practice — in most cases we only ever need the absolute value of the determinant anyway.

Notice that this definition is compatible with the result for square matrices, and later we'll further prove that this definition does indeed preserve the geometric meaning of the determinant.

Let's now work through two examples. First, consider a $n\times 1$ matrix

\begin{equation}\boldsymbol{X} = \begin{pmatrix}x_1 \\ x_2 \\ \vdots \\ x_n\end{pmatrix}\end{equation}

By definition $\eqref{eq:dingyi}$, we compute

\begin{equation}|\det \boldsymbol{X}| = \sqrt{x_1^2 + x_2^2 + \dots + x_n^2}\end{equation}

According to our definition, this should represent the "1-dimensional volume" of $1$ column vectors of dimension $n$. By analogy, this "1-dimensional volume" is just length, and the expression above is exactly the formula for the norm of a vector. That is, in the $n\times 1$ case, definition $\eqref{eq:dingyi}$ agrees with what we expect.

The second example is a $n\times 2$ matrix

\begin{equation}\boldsymbol{Z} = \begin{pmatrix}x_1 & y_1 \\ x_2 & y_2 \\ \vdots & \vdots \\ x_n & y_n\end{pmatrix}=(\boldsymbol{x}, \boldsymbol{y})\end{equation}

Computing according to definition $\eqref{eq:dingyi}$, we eventually get

\begin{equation}|\det \boldsymbol{Z}| = \sqrt{\boldsymbol{x}^{\top}\boldsymbol{x}\boldsymbol{y}^{\top}\boldsymbol{y} - (\boldsymbol{x}^{\top}\boldsymbol{y})^2}\end{equation}

It's not hard to see that this result is exactly the square of the area of the parallelogram spanned by $\boldsymbol{x},\boldsymbol{y}$, since computing that area directly gives

\begin{equation}\begin{aligned}|\boldsymbol{x}|\cdot|\boldsymbol{y}|\cdot\sin\theta =& |\boldsymbol{x}|\cdot|\boldsymbol{y}|\cdot\sqrt{1-\cos^2\theta}\\ =&|\boldsymbol{x}|\cdot|\boldsymbol{y}|\cdot\sqrt{1-\left(\frac{\boldsymbol{x}^{\top}\boldsymbol{y}}{|\boldsymbol{x}|\cdot|\boldsymbol{y}|}\right)^2} \end{aligned}\end{equation}

That is, for a $n\times 2$ matrix, definition $\eqref{eq:dingyi}$ again matches our expectations.

Proof

Now let's consider the general proof. For a $\boldsymbol{B}_{n \times k}$ matrix

\begin{equation}\boldsymbol{B}_{n \times k} = \begin{pmatrix}b_{11} & \dots & b_{1k}\\ b_{21} & \dots & b_{2k}\\ \vdots & \ddots & \vdots\\ b_{n1} & \dots & b_{nk}\end{pmatrix} = (\boldsymbol{b}_1,\dots,\boldsymbol{b}_k)\end{equation}

with $n > k$, we first invoke the familiar Gram–Schmidt orthogonalization process, which tells us that there exists a $n\times k$ orthogonal matrix $\boldsymbol{U}_{n\times k}$ (consisting of $k$ pairwise orthogonal $n$-dimensional unit column vectors) and a $k\times k$ lower-triangular matrix $\boldsymbol{C}_{k\times k}$, such that

\begin{equation}\boldsymbol{B}_{n \times k}=\boldsymbol{U}_{n\times k}\boldsymbol{C}_{k\times k}\end{equation}

This is known in mathematics as the "QR decomposition." We know that orthogonal transformations don't change any geometric properties, so the determinant of $\boldsymbol{B}_{n \times k}$ should equal (the absolute value of) the determinant of $\boldsymbol{C}_{k\times k}$, i.e., $|\det \boldsymbol{C}_{k\times k}|$.

It follows that

\begin{equation}\begin{aligned}|\det \boldsymbol{B}_{n\times k}| =& |\det \boldsymbol{C}_{k\times k}|\\ =& \sqrt{\det\left(\boldsymbol{C}_{k\times k}^{\top}\boldsymbol{C}_{k\times k}\right)}\\ =& \sqrt{\det\left[\left(\boldsymbol{U}_{n\times n}^{\top}\boldsymbol{B}_{n \times k}\right)^{\top}\left(\boldsymbol{U}_{n\times n}^{\top}\boldsymbol{B}_{n \times k}\right)\right]}\\ =& \sqrt{\det\left(\boldsymbol{B}_{n \times k}^{\top}\boldsymbol{B}_{n \times k}\right)} \end{aligned}\end{equation}

So, for a matrix $\boldsymbol{B}$ satisfying $n\times k$ and $n > k$, a non-trivial and reasonable definition of the determinant is $\sqrt{\det (\boldsymbol{B}^{\top}\boldsymbol{B})}$. Clearly, if $n < k$, then the definition is $\sqrt{\det (\boldsymbol{B}\boldsymbol{B}^{\top})}$.

Conclusion

Starting from the geometric meaning of the determinant, we set out to discuss the determinant of non-square matrices, and ultimately showed that expression $\eqref{eq:dingyi}$ can serve as a reasonably sensible definition of the determinant for non-square matrices. Although in theory $\eqref{eq:dingyi}$ can only define the absolute value of the determinant, this is already sufficient for most purposes.

As for applications of the determinant of non-square matrices: we know that in integral transforms, the Jacobian determinant guarantees that a transformation is non-trivial. Similarly, perhaps the determinant of a non-square matrix could be used to guarantee the non-triviality of dimension-raising or dimension-reducing transformations. This is, of course, just an idea for now — I'm still thinking through these questions, and readers who are interested are welcome to discuss them with me.