GAN Models from the Energy Perspective (Part 2): GAN = "Analysis" + "Sampling"
In this series, we try to understand GANs from the energy perspective. We'll find that this perspective is so elegant and intuitive that it's genuinely delightful.
In the previous post, we sketched a plain and forceful energy-based picture that let us easily understand a lot about GANs — in other words, a colloquial explanation already got us most of the way there, and we had already written down the final conclusions. In this post, we continue understanding GANs from the energy perspective, but this time we'll try to derive, with reasonably rigorous mathematical language, what we previously described in simple, informal terms.
As with the first article, this derivation is directly inspired by Bengio's team's new paper, Maximum Entropy Generators for Energy-Based Models.
The original authors' open-source implementation: https://github.com/ritheshkumar95/energy_based_generative_models
The rough outline of this post is as follows:
1. Derive the update formula for the "positive-negative phase" adversarial process under an energy distribution;
2. Compare theoretical analysis with experimental sampling, and show how combining the two gives us the GAN framework;
3. Derive an auxiliary loss for the generator that can, in theory, prevent mode collapse;
4. Briefly touch on MCMC sampling based on the energy function.
Energy from a Mathematical Perspective
In this section, we briefly introduce the energy model and derive its theoretical update formula, pointing out that it exhibits an adversarial structure between a "positive phase" and a "negative phase".
The Energy Distribution Model
First, suppose we have a batch of data $x_1,x_2,\dots,x_n\sim p(x)$, which we want to fit with a probability model. We choose the model
\begin{equation}q_{\theta}(x) = \frac{e^{-U_{\theta}(x)}}{Z_{\theta}}\end{equation}
where $U_{\theta}$ is an undetermined function with parameters $\theta$, which we call the "energy function", and $Z_{\theta}$ is the normalizing constant (partition function)
\begin{equation}Z_{\theta} = \int e^{-U_{\theta}(x)}dx\label{eq:z}\end{equation}
Such a distribution can be called an "energy distribution"; in physics it's also known as the "Boltzmann distribution".
There are many ways to justify choosing this kind of energy distribution — you could say it's inspired by physics, or by the maximum entropy principle, or you could simply say it's chosen because it happens to be relatively tractable. But regardless, this distribution is extremely common and useful; the softmax activation we use so often is, in fact, precisely an instance of this distribution.
The difficulty now is how to find the parameters $\theta$, and the source of the difficulty is that the partition function $\eqref{eq:z}$ usually cannot be computed explicitly. Of course, despite this practical difficulty, nothing stops us from carrying on with the derivation.
The Adversarial Interplay of Positive and Negative Phases
To find the parameters $\theta$, we first define the log-likelihood function:
\begin{equation}\mathbb{E}_{x\sim p(x)} \big[\log q_{\theta}(x)\big]\end{equation}
We want this to be as large as possible, which means we want
\begin{equation}L_{\theta}=\mathbb{E}_{x\sim p(x)} \big[-\log q_{\theta}(x)\big]\end{equation}
to be as small as possible. To that end, we apply gradient descent to $L_{\theta}$. We have
\begin{equation}\begin{aligned}\nabla_{\theta}\log q_{\theta}(x)=&\nabla_{\theta}\log e^{-U_{\theta}(x)}-\nabla_{\theta}\log Z_{\theta}\\ =&-\nabla_{\theta} U_{\theta}(x)-\frac{1}{Z_{\theta}}\nabla_{\theta} Z_{\theta}\\ =&-\nabla_{\theta} U_{\theta}(x)-\frac{1}{Z_{\theta}}\nabla_{\theta} \int e^{-U_{\theta}(x)}dx\\ =&-\nabla_{\theta} U_{\theta}(x)+\frac{1}{Z_{\theta}} \int e^{-U_{\theta}(x)}\nabla_{\theta} U_{\theta}(x) dx\\ =&-\nabla_{\theta} U_{\theta}(x)+\int \frac{e^{-U_{\theta}(x)}}{Z_{\theta}}\nabla_{\theta} U_{\theta}(x) dx\\ =&-\nabla_{\theta} U_{\theta}(x)+\mathbb{E}_{x\sim q_{\theta}(x)}\big[\nabla_{\theta} U_{\theta}(x)\big] \end{aligned}\end{equation}
and therefore
\begin{equation}\nabla_{\theta} L_{\theta} = \mathbb{E}_{x\sim p(x)}\big[\nabla_{\theta} U_{\theta}(x)\big] - \mathbb{E}_{x\sim q_{\theta}(x)}\big[\nabla_{\theta} U_{\theta}(x)\big]\label{eq:q-grad}\end{equation}
which means the gradient descent update rule is
\begin{equation}\theta \leftarrow \theta - \varepsilon \Big(\mathbb{E}_{x\sim p(x)}\big[\nabla_{\theta} U_{\theta}(x)\big] - \mathbb{E}_{x\sim q_{\theta}(x)}\big[\nabla_{\theta} U_{\theta}(x)\big]\Big)\label{eq:q-grad-gd}\end{equation}
Notice the structure of equation $\eqref{eq:q-grad}$: it is the difference between the mean of $\nabla_{\theta} U_{\theta}(x)$ under the true distribution and under the fitted distribution — this is exactly the famous decomposition into "positive phase" and "negative phase" in machine learning. Equation $\eqref{eq:q-grad}$ embodies the adversarial interplay between these two phases; some people even liken this to the process of dreaming.
Playing to Each Model's Strengths ⇒ GAN
In this section, we show that "easy to analyze" and "easy to sample from" are hard to have simultaneously: models that are easy to analyze theoretically are hard to sample from experimentally, while models that are easy to sample from are hard to analyze cleanly in theory. Trying to combine the strengths of both leads us to the GAN model.
Theoretical Analysis vs. Experimental Sampling
In fact, equations $\eqref{eq:q-grad}$ and $\eqref{eq:q-grad-gd}$ show that the theoretical analysis of the energy distribution model we started with isn't actually difficult. But when we try to put it into practice, we find that we must be able to sample from $q_{\theta}$: $\mathbb{E}_{x\sim q_{\theta}(x)}$. That is, given a specific $U_{\theta}(x)$, we need some way of drawing a batch of samples $x$ from $q_{\theta}(x)=e^{-U_{\theta}(x)}/Z_{\theta}$.
However, at this point we have no experience whatsoever sampling from $q_{\theta}(x)=e^{-U_{\theta}(x)}/Z_{\theta}$. What we do find convenient to sample from is a process of the following form:
\begin{equation}z\sim q(z),\quad x = G_{\varphi}(z)\end{equation}
Here $q(z)$ denotes the standard normal distribution. That is, we can sample a $z$ from the standard normal distribution, and then transform it into the $x$ we want via a fixed model $G_{\varphi}$. This implies that the theoretical expression for this distribution is:
\begin{equation}q_{\varphi}(x) = \int \delta\big(x - G_{\varphi}(z)\big)q(z)dz\label{eq:q-varphi}\end{equation}
The problem is: if we replace the original $q_{\theta}(x)$ with $q_{\varphi}(x)$, sampling becomes convenient, but the analogous theoretical derivation becomes hard — in other words, we simply cannot derive anything like $\eqref{eq:q-grad-gd}$.
The Birth of GAN
So here's an audacious idea: can we combine the two, letting each play to its own strengths where it excels?
Isn't the $\mathbb{E}_{x\sim q_{\theta}(x)}$ in equation $\eqref{eq:q-grad-gd}$ hard to realize? Then let's just replace that part with $\mathbb{E}_{x\sim q_{\varphi}(x)}$:
\begin{equation}\theta \leftarrow \theta - \varepsilon \Big(\mathbb{E}_{x\sim p(x)}\big[\nabla_{\theta} U_{\theta}(x)\big] - \mathbb{E}_{x\sim q_{\varphi}(x)}\big[\nabla_{\theta} U_{\theta}(x)\big]\Big)\end{equation}
that is
\begin{equation}\theta \leftarrow \theta - \varepsilon \Big(\mathbb{E}_{x\sim p(x)}\big[\nabla_{\theta} U_{\theta}(x)\big] - \mathbb{E}_{x=G_{\varphi}(z),z\sim q(z)}\big[\nabla_{\theta} U_{\theta}(x)\big]\Big)\label{eq:q-grad-gd-new}\end{equation}
Now sampling is convenient, but this is only valid provided $q_{\varphi}(x)$ is sufficiently close to $q_{\theta}(x)$ (since $q_{\theta}(x)$ is the standard, correct one). So we use the KL divergence to measure how far apart the two are:
\begin{equation}\begin{aligned}KL\big(q_{\varphi}(x)\big\Vert q_{\theta}(x)\big)=&\int q_{\varphi}(x) \log \frac{q_{\varphi}(x)}{q_{\theta}(x)}dx \\ =& - H_{\varphi}(X) + \mathbb{E}_{x\sim q_{\varphi}(x)}\big[U_{\theta}(x)\big]+\log Z_{\theta}\end{aligned}\end{equation}
Equation $\eqref{eq:q-grad-gd-new}$ is valid on the premise that $q_{\varphi}(x)$ and $q_{\theta}(x)$ are sufficiently close, i.e., that the above quantity is sufficiently small. For fixed $q_{\theta}(x)$, $Z_{\theta}$ is a constant, so the optimization objective for $\varphi$ is:
\begin{equation}\varphi =\mathop{\text{argmin}}_{\varphi} - H_{\varphi}(X) + \mathbb{E}_{x\sim q_{\varphi}(x)}\big[U_{\theta}(x)\big]\label{eq:varphi-gd}\end{equation}
Here $H_{\varphi}(X) = - \int q_{\varphi}(x) \log q_{\varphi}(x) dx$ denotes the entropy of $q_{\varphi}(x)$. $- H_{\varphi}(X)$ wants the entropy to be as large as possible, which corresponds to diversity; $\mathbb{E}_{x\sim q_{\varphi}(x)}[U_{\theta}(x)]$ wants the potential energy of the images to be as small as possible, which corresponds to realism.
On the other hand, notice that equation $\eqref{eq:q-grad-gd-new}$ is in fact the gradient descent update rule for the objective
\begin{equation}\theta =\mathop{\text{argmin}}_{\theta} \mathbb{E}_{x\sim p(x)}\big[U_{\theta}(x)\big] - \mathbb{E}_{x=G_{\varphi}(z),z\sim q(z)}\big[U_{\theta}(x)\big]\label{eq:theta-gd}\end{equation}
So we find that the whole process is, in fact, alternating gradient descent on $\eqref{eq:theta-gd}$ and $\eqref{eq:varphi-gd}$. And as noted in the first post, the objective for $\theta$ may cause numerical instability; based on the reasoning given there, since real samples should lie near local minima, we can add a gradient penalty term into $\eqref{eq:theta-gd}$, giving us the final procedure:
\begin{equation}\begin{aligned}\theta =&\,\mathop{\text{argmin}}_{\theta} \mathbb{E}_{x\sim p(x)}\big[U_{\theta}(x)\big] - \mathbb{E}_{x=G_{\varphi}(z),z\sim q(z)}\big[U_{\theta}(x)\big] + \lambda \mathbb{E}_{x\sim p(x)}\big[\Vert \nabla_x U_{\theta}(x)\Vert^2\big]\\ \varphi =&\,\mathop{\text{argmin}}_{\varphi} - H_{\varphi}(X) + \mathbb{E}_{x=G_{\varphi}(z),z\sim q(z)}\big[U_{\theta}(x)\big] \end{aligned}\label{eq:gan-energy}\end{equation}
This is exactly the gradient-penalty-based GAN model, which we already "brainstormed" our way to in GAN Models from the Energy Perspective (Part 1) — and now we have derived it rigorously from the mathematical analysis of the energy model.
So, in essence, GAN is the result of the energy model and the sampling model each playing to its own strengths while avoiding its own weaknesses.
Attacking $H(X)$ Head-On! #
We're now just $H_{\varphi}(X)$ away from a complete implementation of the whole model. We've already said that
\begin{equation}H_{\varphi}(X) = - \int q_{\varphi}(x) \log q_{\varphi}(x) dx\end{equation}
denotes the entropy of $q_{\varphi}(x)$, and the theoretical expression for $q_{\varphi}(x)$ is $\eqref{eq:q-varphi}$, whose integral is intractable — so $H_{\varphi}(X)$ is likewise intractable.
The way to break out of this bind is to convert entropy into mutual information, and then estimate the mutual information. There are two ways to do this: an (in principle exact) estimate via f-divergence, or an estimate via an information lower bound.
Maximum Entropy and Mutual Information
First, we can make use of the fact that $x=G_{\varphi}(z)$: $x=G_{\varphi}(z)$ means the conditional probability $q_{\varphi}(x|z) = \delta\big(x - G(z)\big)$ is deterministic, which we can also think of as a Gaussian distribution $\mathcal{N}(x;G_{\varphi}(z),0)$ with mean $G(z)$ and variance 0.
Now let's consider the mutual information $I(X,Z)$:
\begin{equation}\begin{aligned}I_{\varphi}(X,Z)=&\iint q_{\varphi}(x|z)q(z)\log \frac{q_{\varphi}(x|z)}{q_{\varphi}(x)}dxdz\\ =&\iint q_{\varphi}(x|z)q(z)\log q_{\varphi}(x|z) dxdz - \iint q_{\varphi}(x|z)q(z) \log q_{\varphi}(x)dxdz\\ =&\int q(z)\left(\int q_{\varphi}(x|z)\log q_{\varphi}(x|z) dx\right)dz + H(X) \end{aligned}\end{equation}
We now have a relation between $I_{\varphi}(X,Z)$ and $H_{\varphi}(X)$: their difference is
\begin{equation}\int q(z)\left(\int q_{\varphi}(x|z)\log q_{\varphi}(x|z) dx\right)dz\triangleq -H_{\varphi}(X|Z)\end{equation}
In fact $H_{\varphi}(X|Z)$ is called the "conditional entropy."
If we're working with a discrete distribution, then since $x=G_{\varphi}(z)$ is deterministic, we have $q_{\varphi}(x|z)\equiv 1$, so $H_{\varphi}(X|Z)$ is 0, i.e. $I_{\varphi}(X,Z)=H_{\varphi}(X)$. If it's a continuous distribution, as noted above we can think of it as a Gaussian with variance 0 $\mathcal{N}(x;G_{\varphi}(z),0)$; we can first consider the case of a constant variance $\mathcal{N}(x;G(z),\sigma^2)$, and computation shows that $H_{\varphi}(X|Z)\sim \log \sigma^2 $ is a constant, and then $\sigma \to 0$ — but we find the result is infinite. Strictly speaking, infinities can't be computed with, but in fact the variance doesn't actually need to be exactly 0, only small enough to be visually indistinguishable.
So, in general, we can be confident that the mutual information $I_{\varphi}(X,Z)$ and the entropy $H_{\varphi}(X)$ differ only by an irrelevant constant, so in equation $\eqref{eq:gan-energy}$ we can replace $H_{\varphi}(X)$ with $I_{\varphi}(X,Z)$:
\begin{equation}\begin{aligned}\theta =&\,\mathop{\text{argmin}}_{\theta} \mathbb{E}_{x\sim p(x)}\big[U_{\theta}(x)\big] - \mathbb{E}_{x=G_{\varphi}(z),z\sim q(z)}\big[U_{\theta}(x)\big] + \lambda \mathbb{E}_{x\sim p(x)}\big[\Vert \nabla_x U_{\theta}(x)\Vert^2\big]\\ \varphi =&\,\mathop{\text{argmin}}_{\varphi} - I_{\varphi}(X,Z) + \mathbb{E}_{x=G_{\varphi}(z),z\sim q(z)}\big[U_{\theta}(x)\big] \end{aligned}\label{eq:gan-energy-2}\end{equation}
Now we want to minimize $- I_{\varphi}(X,Z)$, which is to say, maximize the mutual information $I_{\varphi}(X,Z)$. Intuitively this also makes sense, because this term is meant to prevent mode collapse — and if mode collapse does occur, then almost any $z$ generates the same $x$, so the mutual information $X,Z$ can't possibly be large.
But changing the objective from $H_{\varphi}(X)$ to $I_{\varphi}(X,Z)$ looks like it's merely a formal rewrite, and it doesn't seem to have actually solved anything. Fortunately, though, we've already done the work of estimating mutual information before, in the section "The Essence of Mutual Information" of Mutual Information in Deep Learning: Unsupervised Feature Extraction — that is, there's already a solution for directly estimating mutual information, and readers can just go read that post; I won't repeat the argument here.
Mutual Information and the Information Lower Bound
If we don't need an exact estimate of the mutual information, we can instead borrow the idea from InfoGAN: obtain a lower bound on the mutual information, and optimize that bound instead.
Starting from the definition of mutual information:
\begin{equation}I_{\varphi}(X,Z)=\iint q_{\varphi}(x|z)q(z)\log \frac{q_{\varphi}(x|z)q(z)}{q_{\varphi}(x)q(z)}dxdz\end{equation}
Let $q_{\varphi}(z|x) = q_{\varphi}(x|z)q(z)/q_{\varphi}(x)$ denote the exact posterior distribution; then for any approximate posterior distribution $p(z|x)$, we have
\begin{equation}\begin{aligned}I_{\varphi}(X,Z)=&\iint q_{\varphi}(x|z)q(z)\log \frac{q_{\varphi}(z|x)}{q(z)}dxdz\\ =&\iint q_{\varphi}(x|z)q(z)\log \frac{p(z|x)}{q(z)}dxdz + \iint q_{\varphi}(x|z)q(z)\log \frac{q_{\varphi}(z|x)}{p(z|x)}dxdz\\ =&\iint q_{\varphi}(x|z)q(z)\log \frac{p(z|x)}{q(z)}dxdz + \int q_{\varphi}(x)KL\Big(q_{\varphi}(z|x) \Big\Vert p(z|x)\Big)dz\\ \geq &\iint q_{\varphi}(x|z)q(z)\log \frac{p(z|x)}{q(z)}dxdz\\ =& \iint q_{\varphi}(x|z)q(z)\log p(z|x) - \underbrace{\iint q_{\varphi}(x|z)q(z)\log q(z) dxdz}_{=\int q(z)\log q(z)dz\,\,\text{is a const}} \end{aligned}\end{equation}
That is to say, the mutual information is greater than or equal to $\iint q_{\varphi}(x|z)q(z)\log p(z|x)$ plus a constant. If we want to maximize the mutual information, we can instead maximize this lower bound. Since $p(z|x)$ is arbitrary, we can simply take $p(z|x)=\mathcal{N}\left(z;E(x),\sigma^2\right)$, where $E(x)$ is a parameterized encoder. Substituting this in and dropping the irrelevant constants, we find this amounts to adding an extra loss term to the generator:
\begin{equation}\mathbb{E}_{z\sim q(z)} \big[\Vert z - E(G(z))\Vert^2\big]\end{equation}
So, based on the InfoGAN information-lower-bound approach, equation $\eqref{eq:gan-energy}$ becomes:
\begin{equation}\begin{aligned}\theta =&\,\mathop{\text{argmin}}_{\theta} \mathbb{E}_{x\sim p(x)}\big[U_{\theta}(x)\big] - \mathbb{E}_{z\sim q(z)}\big[U_{\theta}(G_{\varphi}(z))\big] + \lambda_1 \mathbb{E}_{x\sim p(x)}\big[\Vert \nabla_x U_{\theta}(x)\Vert^2\big]\\ \varphi,E =&\,\mathop{\text{argmin}}_{\varphi,E} \mathbb{E}_{z\sim q(z)}\big[U_{\theta}(G_{\varphi}(z)) + \lambda_2 \Vert z - E(G_{\varphi}(z))\Vert^2\big] \end{aligned}\label{eq:gan-energy-3}\end{equation}
At this point, we've dealt with $H_{\varphi}(X)$ from two different angles, thereby completing the whole derivation connecting GANs and energy models.
Improving Results with MCMC
Looking back to the beginning, we derived the GAN model starting from the energy distribution, and the energy function $U(x)$ is precisely the discriminator in the GAN model. Since $U(x)$ carries the meaning of an energy function, once training is complete we can exploit properties of the energy function to do something even more useful — for instance, introducing MCMC to improve the results.
A Brief Introduction to MCMC
To be honest, I only have a rough grasp of what MCMC means, and I don't really understand its methods and inner workings; so this "brief introduction" is really just a basic sketch of the concept. MCMC stands for "Markov Chain Monte Carlo." As I understand it, roughly speaking it works like this: we can't directly sample from some given distribution $q(x)$, but we can construct the following random process:
\begin{equation}x_{n+1} = f(x_n, \alpha)\label{eq:suijidigui}\end{equation}
where $\alpha$ is some easy-to-implement random process, e.g., sampling from a binary distribution or a normal distribution. In this way, starting from some $x_0$, the resulting sequence $\{x_1,x_2,\dots,x_n,\dots\}$ is itself random.
If we can further show that the stationary distribution of process $\eqref{eq:suijidigui}$ is exactly $q(x)$, then this means the sequence $\{x_1,x_2,\dots,x_n,\dots\}$ is in fact a batch of samples drawn from $q(x)$ — thus we've achieved sampling from $q(x)$, except the samples come out in a particular sequential order.
The Langevin Equation
A special case of equation $\eqref{eq:suijidigui}$ is the Langevin equation:
\begin{equation}x_{t+1} = x_t - \frac{1}{2}\varepsilon \nabla_x U(x_t) + \sqrt{\varepsilon}\alpha,\quad \alpha \sim \mathcal{N}(\alpha;0,1)\label{eq:sde}\end{equation}
which is also called a stochastic differential equation. When $\varepsilon\to 0$, its stationary distribution is exactly the energy distribution
\begin{equation}p(x) = \frac{e^{-U(x)}}{Z}\end{equation}
That is, given the energy function $U(x)$, equation $\eqref{eq:sde}$ lets us sample from the energy distribution — this is the original idea behind MCMC sampling for energy distributions.
Of course, sampling $x$ directly from the energy function via equation $\eqref{eq:sde}$ may not be very practical, because the dimensionality of $x$ (in the common case, $x$ represents an image) is too large, making it hard to keep things under control. What's more, the last term in equation $\eqref{eq:sde}$ is Gaussian noise, so as long as $\varepsilon\neq 0$, the result will inevitably be noisy, which also makes it hard to guarantee that the image looks real.
An interesting trick is: instead of directly considering MCMC sampling over $x$, we can instead consider sampling over $z$. Because in the model derived earlier, we end up with both an energy function $U_{\theta}(x)$ and a generative model $G_{\varphi}(z)$, which means the energy function of $z$ is
\begin{equation}U_{\theta}(G_{\varphi}(z))\end{equation}
Note:
This result doesn't hold strictly — it should really be regarded as an empirical formula, and rigorously speaking it only holds when the Jacobian determinant of $G$ equals 1. I actually discussed this with the original authors on GitHub, and they also pointed out that there's no rigorous theoretical justification for it — it's just intuition. See here for details:
https://github.com/ritheshkumar95/energy_based_generative_models/issues/4
Once we have the energy function for $z$, we can use equation $\eqref{eq:sde}$ to perform MCMC sampling over $z$:
\begin{equation}z_{t+1} = z_t - \frac{1}{2}\varepsilon \nabla_z U_{\theta}(G_{\varphi}(z_t)) + \sqrt{\varepsilon}\alpha,\quad \alpha \sim \mathcal{N}(\alpha;0,1)\label{eq:sde-2}\end{equation}
This does away with all the problems mentioned above, since the dimensionality of $z$ is generally much smaller than that of $x$, and we no longer need to worry about $\varepsilon\neq 0$ introducing noise, because $z$ is noise to begin with.
A Better Truncation Trick
At this point, readers whose heads aren't yet spinning might pause and think: isn't the distribution of $z$ just the standard normal distribution? Isn't that easy enough to sample from already? Why bother with all this MCMC machinery?
Ideally, the energy distribution
\begin{equation}q_{\theta,\varphi}(z)=\frac{e^{-U_{\theta}(G_{\varphi}(z))}}{Z}\end{equation}
corresponding to the energy function $U_{\theta}(G_{\varphi}(z))$ of $z$ should indeed just be the standard normal distribution $q(z)$ we originally fed into it. But of course, there's always some gap between the ideal and the reality: once we've trained a generative model using the standard normal distribution, the noise that actually produces realistic samples tends to occupy a narrower range — which calls for some truncation, or filtering, tricks.
For example, flow-based generative models often use an "annealing" trick after training: reducing the variance of the noise at generation time to produce more reliably good samples — see Slow-Flowing Streams — NICE: Basic Concepts and Implementation of Flow Models. And BigGAN, released last year, also discussed truncation tricks for noise in GANs.
If we trust our model — if we believe that both the energy function $U_{\theta}(x)$ and the generative model $G_{\varphi}(z)$ carry genuine value — then we have good reason to believe that $e^{-U_{\theta}(G_{\varphi}(z))}/Z$ is a better distribution over $z$ than the standard normal (a distribution over $x$ $z$ that can produce more realistic samples, since it incorporates $G_{\varphi}(z)$ into the very definition of the distribution). So sampling from $e^{-U_{\theta}(G_{\varphi}(z))}/Z$ should outperform sampling from $q(z)$ — that is, MCMC sampling via $\eqref{eq:sde-2}$ can improve the quality of the resulting generated samples, and the original paper confirms exactly this. We can think of this as a superior version of the truncation trick.
A More Efficient Approach: MALA
The sampling process $\eqref{eq:sde-2}$ is actually still fairly inefficient in practice; what the original paper actually uses is an improved version called MALA (Metropolis-adjusted Langevin algorithm), which introduces an additional filtering step on top of $\eqref{eq:sde-2}$:
\begin{equation}\begin{aligned}\tilde{z}_{t+1} =& z_t - \frac{1}{2}\varepsilon \nabla_z U_{\theta}(G_{\varphi}(z_t)) + \sqrt{\varepsilon}\alpha,\quad \alpha \sim \mathcal{N}(\alpha;0,1)\\ \\ z_{t+1} =& \left\{\begin{aligned}&\tilde{z}_{t+1}, \quad \text{if}\beta < \gamma\\ &z_t, \quad \text{other cases}\end{aligned}\right.,\quad \beta \sim U[0,1]\\ \\ \gamma =& \min\left\{1, \frac{q(\tilde{z}_{t+1})q(z_t|\tilde{z}_{t+1})}{q(\tilde{z}_{t})q(\tilde{z}_{t+1}|z_t)}\right\} \end{aligned}\end{equation}
where
\begin{equation}\begin{aligned}q(z)\propto&\, \exp\Big(-U_{\theta}(G_{\varphi}(z))\Big)\\ q(z'|z)\propto&\, \exp\left(-\frac{1}{2\varepsilon}\Vert z' - z + \varepsilon \nabla_z U_{\theta}(G_{\varphi}(z))\Vert^2\right) \end{aligned}\end{equation}
That is, we accept $z_{t+1}=\tilde{z}_{t+1}$ with probability $\gamma$, and keep the value unchanged with probability $1-\gamma$. According to the Wikipedia article on this, this improvement gives the sampling process a better chance of landing on high-probability samples, which in turn means it can generate more realistic samples. (I don't really understand the theory behind this myself, so I'm just taking it on faith here.)
A Powerful Perspective
Another long, formula-heavy post, but we've finally worked our way through the mathematical derivation of GANs under the energy distribution — GAN turns out to be the product of reconciling the tension between "theoretical analysis" and "experimental sampling." All in all, I think the whole derivation is quite illuminating, and it helps us understand both the key insight behind GANs and where the difficulties lie.
The energy perspective leans toward mathematical physics, and once we manage to connect machine learning with mathematical physics, we can draw inspiration directly from physics and mathematics — even making the corresponding machine learning models feel less like a "black box." This kind of perspective tends to be intoxicating, and it gives one a real sense of power.
Translated automatically with claude-sonnet-5; all equations are reproduced verbatim from the source. Copyright remains with the original author.