Appreciating the Identity det(exp(A)) = exp(Tr(A))

恒等式 det(exp(A)) = exp(Tr(A)) 赏析 苏剑林 · 2019-02-18 · Mathematics analysis matrix determinant

The subject of this post is an interesting identity for matrix determinants:

\begin{equation}\det(\exp(\boldsymbol{A})) = \exp(\text{Tr}(\boldsymbol{A}))\label{eq:main}\end{equation}

This identity has shown up in quite a few mathematical and physical calculations, and I've come across it several times in different pieces of literature.

Notice that the left-hand side involves the exponential of a matrix followed by taking a determinant — both extremely expensive operations; the right-hand side is merely the trace of the matrix (a scalar), followed by a scalar exponential. The two sides differ enormously in computational cost, and yet they turn out to be equal! You have to admit this is a remarkable fact.

So in this post, let's take a proper look and enjoy this identity. more

Matrix exponentials

Of course, to appreciate this identity we need to do some groundwork first. To begin with, how should we understand $\exp(\boldsymbol{A})$? Generally speaking, it is defined via the ordinary Taylor series expansion of $e^x$:

\begin{equation}\exp(\boldsymbol{A})=\sum_{n=0}^{\infty}\frac{\boldsymbol{A}^n}{n!}\end{equation}

Here $\boldsymbol{A}$ is a $k\times k$ matrix. One can show that this definition converges for any matrix, so it is a well-posed definition.

With this definition in hand, we can immediately write down the solution to a constant-coefficient system of linear differential equations:

\begin{equation}\frac{d}{dt}\boldsymbol{x}=\boldsymbol{A}\boldsymbol{x}\quad\Rightarrow \quad \boldsymbol{x}=\exp(t\boldsymbol{A})\boldsymbol{x}_0\end{equation}

Of course, this result is only of theoretical value at best, because when it comes to actual computation, you're still stuck laboriously computing $\boldsymbol{A}^2,\boldsymbol{A}^3,\dots$ term by term.

Is there a simpler way to compute it? Yes — things become simpler when $\boldsymbol{A}$ can be diagonalized, since diagonalizability means

\begin{equation}\boldsymbol{A}=\boldsymbol{P}\boldsymbol{\Lambda}\boldsymbol{P}^{-1}\end{equation}

Here $\boldsymbol{P}$ is naturally an invertible matrix, and $\boldsymbol{\Lambda}=\text{diag}(\lambda_1,\dots,\lambda_k)$ is a diagonal matrix. This case is simple precisely because

\begin{equation}\boldsymbol{A}^n=\boldsymbol{P}\boldsymbol{\Lambda}^n\boldsymbol{P}^{-1}\end{equation}

and since $\boldsymbol{\Lambda}$ is diagonal, computing $\boldsymbol{\Lambda}^n$ just means raising each entry on the diagonal to the $n$-th power. Hence

\begin{equation}\begin{aligned}\exp(\boldsymbol{A})=&\exp(\boldsymbol{P}\boldsymbol{\Lambda}\boldsymbol{P}^{-1})\\ =&\boldsymbol{P}\left(\sum_{n=0}^{\infty}\frac{\boldsymbol{\Lambda}^n}{n!}\right)\boldsymbol{P}^{-1}\\ =&\boldsymbol{P}\exp(\boldsymbol{\Lambda})\boldsymbol{P}^{-1} \end{aligned}\end{equation}

where $\exp(\boldsymbol{\Lambda})=\text{diag}\left(e^{\lambda_1},\dots,e^{\lambda_k}\right)$.

Note, however, that although the definition of the matrix exponential simply copies the exponential series for real numbers, in general for two arbitrary matrices $\boldsymbol{A},\boldsymbol{B}$,

\begin{equation}\exp(\boldsymbol{A}+\boldsymbol{B})\neq \exp(\boldsymbol{A})\exp(\boldsymbol{B})\end{equation}

A sufficient condition for the two sides to be equal is $\boldsymbol{A}\boldsymbol{B}=\boldsymbol{B}\boldsymbol{A}$, i.e., that multiplication commutes. In other words, when we want to carry over combined operation rules for multiple matrices from the real-number formulas, in many cases we need to add the condition of commutativity for them to hold.

Since $\boldsymbol{A}$ and $-\boldsymbol{A}$ obviously commute, we have

\begin{equation}\boldsymbol{I}=\exp(\boldsymbol{A}-\boldsymbol{A})=\exp(\boldsymbol{A})\exp(-\boldsymbol{A})\end{equation}

which means $\exp(\boldsymbol{A})$ is always invertible, with inverse $\exp(-\boldsymbol{A})$.

Matrix functions

In fact, by using series, many real-valued series can be carried over to matrices, turning them into matrix functions. For example,

\begin{equation}\begin{aligned}\sin(\boldsymbol{A})=&\boldsymbol{A}-\frac{\boldsymbol{A}^3}{3!}+\frac{\boldsymbol{A}^5}{5!}-\frac{\boldsymbol{A}^7}{7!}+\dots\\ \cos(\boldsymbol{A})=&\boldsymbol{I}-\frac{\boldsymbol{A}^2}{2!}+\frac{\boldsymbol{A}^4}{4!}-\frac{\boldsymbol{A}^6}{6!}+\dots \end{aligned}\end{equation}

Likewise, if $\boldsymbol{A}\boldsymbol{B}=\boldsymbol{B}\boldsymbol{A}$, then

\begin{equation}\sin(\boldsymbol{A}+\boldsymbol{B})=\sin(\boldsymbol{A})\cos(\boldsymbol{B})+\sin(\boldsymbol{B})\cos(\boldsymbol{A})\end{equation}

Having discussed the exponential function above, where there's an exponential there's naturally a logarithm. There are generally two definitions of the matrix logarithm. The first is: if a matrix $\boldsymbol{B}$ satisfies $\exp(\boldsymbol{B})=\boldsymbol{A}$, then $\boldsymbol{B}$ is called the logarithm of the matrix $\boldsymbol{A}$.

However, under this definition, the logarithm of a matrix is not unique — even restricting to real matrices. For example, for $\boldsymbol{A}=\begin{pmatrix}\cos\alpha & -\sin\alpha \\ \sin\alpha & \cos\alpha\end{pmatrix}$, any $(\alpha + 2\pi n)\begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix}$ is a logarithm of it, where $n$ is any integer.

The other definition simply copies the logarithmic series expansion for real numbers:

\begin{equation}\ln (\boldsymbol{I}+\boldsymbol{A}) = \sum_{n=1}^{\infty}(-1)^{n-1}\frac{\boldsymbol{A}^n}{n}\end{equation}

This definition is simple and gives a unique result, but it only converges under the condition $\Vert \boldsymbol{A}\Vert_2 < 1$, where $\Vert\cdot\Vert_2$ is the matrix 2-norm (see the section "Matrix Norms" in Lipschitz Constraints in Deep Learning: Generalization and Generative Models). When this constraint is satisfied, the logarithm defined this way satisfies

\begin{equation}\exp(\ln (\boldsymbol{I}+\boldsymbol{A})) = \boldsymbol{I}+\boldsymbol{A}\end{equation}

That is, the definition is self-consistent.

The diagonalization technique discussed earlier for the matrix exponential applies equally well to any matrix function defined via a series, for example

\begin{equation}\ln \left(\boldsymbol{I}+\boldsymbol{P}\boldsymbol{\Lambda}^n\boldsymbol{P}^{-1}\right) = \boldsymbol{P}\ln(\boldsymbol{I}＋\boldsymbol{\Lambda})\boldsymbol{P}^{-1}\end{equation}

where $\ln(\boldsymbol{I}＋\boldsymbol{\Lambda})=\text{diag}\big(\ln(1+\lambda_1),\dots,\ln(1+\lambda_k)\big)$.

det(exp(A)) = exp(Tr(A))[ #](#det(exp(A)) = exp(Tr(A)))

After all this preparation, we can finally get to the main topic. For the identity $\eqref{eq:main}$, if the matrix is diagonalizable, the proof isn't too difficult. Because

\begin{equation}\begin{aligned}\text{left end}=&\det(\exp(\boldsymbol{A}))\\ =&\det\big(\boldsymbol{P}\exp(\boldsymbol{\Lambda})\boldsymbol{P}^{-1}\big)\\ =&\det(\boldsymbol{P}) \det(\exp(\boldsymbol{\Lambda})) \underbrace{\det(\boldsymbol{P}^{-1})}_{=1/\det(\boldsymbol{P})}\\ =&\det(\exp(\boldsymbol{\Lambda}))\\ =&e^{\lambda_1 + \dots + \lambda_k} \end{aligned}\end{equation}

and

\begin{equation}\begin{aligned}\text{right side}=&\exp(\text{Tr}(\boldsymbol{A}))\\ =&\exp(\text{Tr}(\boldsymbol{P}\boldsymbol{\Lambda}\boldsymbol{P}^{-1}))\\ =&\exp(\text{Tr}(\boldsymbol{P}^{-1}\boldsymbol{P}\boldsymbol{\Lambda}))\quad [\text{swap order because for square matrix}\boldsymbol{A},\boldsymbol{B}\text{, with}\text{Tr}(\boldsymbol{A}\boldsymbol{B})=\text{Tr}(\boldsymbol{B}\boldsymbol{A})]\\ =&\exp(\text{Tr}(\boldsymbol{\Lambda}))\\ =&e^{\lambda_1 + \dots + \lambda_k} \end{aligned}\end{equation}

But what about non-diagonalizable matrices? If we could show that the set of all diagonalizable matrices is dense in the set of all matrices, we could use a limiting argument to cover the remaining cases — but that's clearly rather cumbersome. Is there a proof that works in one clean sweep? Yes, there is! And it's remarkably elegant — indeed, it's the main reason I wrote this post.

This elegant proof asks us to consider, with a parameter $t$,

\begin{equation}f(t)=\det(\exp(t\boldsymbol{A}))\label{eq:detexp}\end{equation}

and then take its derivative (this involves the derivative of a determinant; for the method, see The Derivative of a Determinant):

\begin{equation}\begin{aligned}\frac{d}{dt}f(t)=&f(t)\text{Tr}\left(\exp(-t\boldsymbol{A})\underbrace{\frac{d}{dt}\exp(t\boldsymbol{A})}_{=\exp(t\boldsymbol{A})\boldsymbol{A}}\right)\\ =&f(t)\text{Tr}(\boldsymbol{A})\end{aligned}\end{equation}

Note that $\text{Tr}(\boldsymbol{A})$ is just a scalar, so we've obtained an ordinary differential equation in $f(t)$!! Its solution is

\begin{equation}f(t)=C\exp(t\,\text{Tr}(\boldsymbol{A}))\end{equation}

From equation $\eqref{eq:detexp}$ we can see that $f(0)=1$, hence $C=1$, i.e., $f(t)=\exp(t\,\text{Tr}(\boldsymbol{A}))$, and so we have proven that

\begin{equation}\det(\exp(t\boldsymbol{A}))=\exp(t\,\text{Tr}(\boldsymbol{A}))\end{equation}

Setting $t=1$ gives us the identity $\eqref{eq:main}$.

Isn't that elegant?

Taking the logarithm of both sides of $\eqref{eq:main}$ and writing $\exp(\boldsymbol{A})=\boldsymbol{B}$, we obtain another commonly seen form of it:

\begin{equation}\ln\det(\boldsymbol{B}) = \text{Tr}(\ln (\boldsymbol{B}))\end{equation}

That's all for this appreciation post — thanks for reading!

Reference link:

https://math.stackexchange.com/questions/1487773/the-identity-deta-exptrlna-for-a-general