"A New Understanding of Matrices, Part 1": What Is a Matrix?

《新理解矩阵1》：矩阵是什么？ 苏剑林 · 2012-10-29 · Mathematics algebra matrix linear

I promised earlier that I would write down some of my own understanding of matrices. Actually, Meng Yan, in his three-part series Understanding Matrices, has already given us a very intuitive account of matrices and linear algebra, along with some of the ideas behind them. Most of my own understanding of matrices originally came from his articles. Of course, in order to get a better grasp of linear algebra, I have also read a fair number of related books, in search of a way of understanding that agrees with intuition. Meng Yan's blog hasn't been updated for a long time, so here I borrow his title in order to lay out my own understanding of matrices.

That said, I have no intention of chasing after the highly abstract issues of spaces, operators, and the like. I just want to share my own views on some of the commonly used tools of linear algebra, such as matrices and determinants. Also, since the piece is titled "Understanding Matrices," this is not meant to be an introductory tutorial on matrices, but rather a discussion, together with readers who already have some background in linear algebra, of other ways of understanding matrices — nothing more. I expect that basically any reader who has studied linear algebra will be able to follow this article.

First, let's go back to a fundamental question: what is a matrix? more

Let's recall how matrices came about in the first place. A matrix can be seen as an ordered collection of vectors, which suggests that a matrix is analogous to a vector; but then, how did vectors themselves come about? A vector is an ordered collection of numbers, which points our inquiry toward the question "what is a number?" For instance, what is the number 1? It can represent 1 meter, 1 kilogram, 1 minute, 1 degree Celsius, or even 1 apple. Why can it carry so many different meanings? The answer is simple: fundamentally, it is nothing at all — it is just the number 1, a symbol, an abstract concept. Precisely because it is abstract, it can be endowed with all sorts of concrete meanings! Coming back to matrices, we can now see why they are so powerful: it's precisely because of that rather dry textbook definition — a matrix is simply an array of numbers with m rows and n columns! This abstraction is what let the matrix "evolve." It is a more general concept: a vector can be regarded as a matrix, and even a single number can be regarded as a matrix, and so on.

An Algebraic Understanding

Of course, the above statements are still vague; we need to pin down exactly what a matrix is good for. This can be analyzed from both an algebraic and a geometric point of view, since combining algebra with geometry is really the ideal approach. To begin with, we know that the most basic elements of mathematics are numbers — strictly speaking, the natural numbers 0, 1, 2, .... Once we have numbers, we can accomplish a great deal. But numbers are singular, whereas very often we need to process a batch of similar operations at once. Suppose, for example, we want to compute the four sums 1+2, 1+3, 2+3, and 4+5 simultaneously. Whether from the standpoint of record-keeping or of study, treating them one by one is rather tedious. Hence a kind of "batch" notation was born: we write (1,1,2,4)+(2,3,3,5), using two distinct symbols to record them, say $\vec{A}=(1,1,2,4),\vec{B}=(2,3,3,5)$, and we can then write this as $\vec{A}+\vec{B}$. This gives us convenience both for study and for record-keeping. This is how the thing we call a "vector" came into being — that is, from an algebraic point of view, vectors arose in order to study batch operations. But vectors do not solve every problem involving batch operations. Consider, for instance, a system of three linear equations in three unknowns:

$$\begin{aligned} a_{11}x_{1}+a_{12}x_{2}+a_{13}x_{3}=b_{1}\\ a_{21}x_{1}+a_{22}x_{2}+a_{23}x_{3}=b_{2}\\ a_{31}x_{1}+a_{32}x_{2}+a_{33}x_{3}=b_{3} \end{aligned}$$

Using vectors alone, we still cannot study this class of problems very well. So we need to devise some new notation. Since the coefficients on the left-hand side follow a certain fixed arrangement and a uniform form, we might as well write them out separately as

$$\begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix}$$

and represent them with a single simple symbol $A$, then write the unknowns and the constants on the right-hand side as vectors $x=[x_{1},x_{2},x_{3}]^{\top}$ and $b=[b_{1},b_{2},b_{3}]^{T}$ respectively (adding the superscript T to indicate column vectors). We would like the equation system above to be written in a simple form as

$$Ax=b$$

This lets us define the multiplication of a 3rd-order square matrix by a 3-dimensional column vector — a purely definitional move, made purely for our convenience in recording and studying things. Building on this, we can go on to study more: for instance, what should matrix-times-matrix multiplication give us?

Likewise, the matrices we study here refer primarily to the core case of n×n square matrices — we should first get a thorough grasp of this core case, since jumping straight into all the messy general cases risks leaving us bewildered and at a loss.

Before studying general multiplication, let's first think about the laws of operation. We know that among real numbers, addition satisfies associativity and commutativity, and multiplication satisfies associativity, commutativity, and distributivity. Which of these laws can be carried over to matrix multiplication? Commutativity cannot be settled a priori — it's a matter of definition; I could define it to hold or not to hold. But for the sake of computational convenience, we would like it to satisfy as many of these laws as possible, so let's first consider associativity, and hope that it can be made to hold. That is, we want

$$(AB)x=A(Bx)$$

where $Bx$ is already an operation we're familiar with (arising from our definition), which yields a column vector, so we can also easily compute $A(Bx)$. Intuitively, $AB$ ought to be an n×n square matrix as well; we can posit it as such, then carry out the operation with the column vector x, and finally match up the results on both sides term by term, giving us an expression for each entry of the n×n matrix $AB$. We'll eventually find that this is exactly the expression defined in textbooks.

Take 2×2 matrices as an example. Let

$$A= \begin{pmatrix} a & b\\ c & d \end{pmatrix},\quad B=\begin{pmatrix} e & f\\ g & h \end{pmatrix},\quad AB=\begin{pmatrix} p & q\\ r & s \end{pmatrix}$$

and set $x=[x,y]^{\top}$; then $(AB)x$ equals

$$[px+qy,rx+sy]^{\top}$$

while $Bx=[ex+fy,gx+hy]^{\top}$, so $A(Bx)=[aex+afy+bgx+bhy,cex+cfy+dgx+dhy]^{\top}$

Then, by matching up the corresponding entries, we get $p=ae+bg,q=af+bh,r=ce+dg,s=cf+dh$. This completes the definition of 2×2 matrix multiplication.

We can now say, from an algebraic point of view, that a matrix is the "ultimate weapon" for simplifying batch linear operations! This is a fairly intuitive and useful algebraic meaning of the matrix.

If we examine commutativity under this definition, we find that matrices generally do not satisfy it. This is, admittedly, something of a pity. But no matter — the fact that matrix multiplication obeys associativity already endows this tool with tremendous power. Consider, for example, the linear system $Ax=y$, for which we have $By=B(Ax)=(BA)x$; if we can somehow find a matrix $B$ such that $BA=I$, that would be wonderful, because then we could simply apply the matrix $B$ to the vector $y$ to obtain the solution of the system. And indeed, such a matrix $B$ exists — this is the inverse matrix. Without associativity, none of this would be possible!

Since matrix multiplication is a "batch version" of the basic operations (linear operations) on real numbers, we can quite naturally extend some formulas from real numbers to their matrix counterparts (as long as they don't involve commutativity). For instance, in the real numbers we have the formula

$$\frac{1}{1-x}=1+x+x^{2}+x^{3}+...\approx 1+x$$

so, when computing the inverse of a matrix, we have an analogous formula

$$(I-A)^{-1}=I+A+A^{2}+A^{3}+...\approx I+A$$

where I is the identity matrix and A is a "sufficiently small" matrix. Exactly how "small" should be defined here is not yet clear — one can think of it, roughly, as the determinant of the matrix being small. Similarly, based on

$$\sqrt{1+x}\approx 1+\frac{1}{2}x$$

we can correspondingly derive $(I+\frac{1}{2}A)^{2}\approx I+A$, which is an approximation formula for the "square root" of a matrix.

Thanks to the matrix multiplication we've defined, batch operations can be carried out directly using the formulas for single-quantity operations, without our having to rack our brains devising brand-new formulas each time. This is precisely where the power of matrices lies! They work miraculously well in solving many linear problems — the simplest example being that the solution to the linear system $Ax=y$ is $x=A^{-1} y$: solving a system of equations takes on the same simple form as solving a single-variable equation! There are also definitions involving matrix exponentials and the like, which I'll introduce later when we come to applications. All of these are like exquisite works of art!

Next time, we'll turn to a geometric understanding of matrices. Of course, most of what's covered here has already been touched on in Meng Yan's articles — I'm merely retelling an old story, and I hope readers won't find it tiresome.