f-GAN Introduction: A Production Workshop for GAN Models

f-GAN简介：GAN模型的生产车间 苏剑林 · 2018-09-29 · Machine Learning variational GAN inference generative model

Today I'll introduce a fairly classic piece of work, which the author named f-GAN. In this paper, he presents a scheme for constructing general GANs via a general $f$ divergence. It's no exaggeration to say that this paper is essentially a "production workshop" for GAN models—it generalizes and encompasses many GAN variants, and can inspire us to quickly construct new GAN variants (whether they're actually worth anything is another matter, but in principle this is possible).

Local Variational Methods

The treatment of the $f$ divergence throughout this paper is, in machine learning terminology, called the "local variational method"—a very classic and useful estimation technique. In fact, this article will spend most of its length introducing the results of applying this estimation technique to the $f$ divergence. As for GANs, they're merely a basic application of this result.

The f-divergence

Let's start with a basic introduction to the $f$ divergence. The so-called $f$ divergence is a generalization of the KL divergence:

$$\begin{equation}\mathcal{D}_f(P\Vert Q) = \int q(x) f\left(\frac{p(x)}{q(x)}\right)dx\label{eq:f-div}\end{equation}$$

Note that, following the standard convention, the argument inside the parentheses is $p/q$ rather than $q/p$—don't naturally assume based on the form of the KL divergence that it should be $q/p$. more

You'll find that this form covers many of the measures between probability distributions we've seen before. Here I'll directly reproduce (part of) the table from the paper:

$$\begin{array}{c|c|c}\hline \textbf{distance name} & \textbf{formula} & \textbf{corresponding}f\\ \hline \text{total variation} & \frac{1}{2}\int | p(x) - q(x)| dx & \frac{1}{2}|u - 1|\\ \hline \text{KL divergence} & \int p(x)\log \frac{p(x)}{q(x)} dx & u \log u\\ \hline \text{reverse KL divergence} & \int q(x)\log \frac{q(x)}{p(x)} dx & - \log u\\ \hline \text{Pearson }\chi^2 & \int \frac{(q(x) - p(x))^{2}}{p(x)} dx & \frac{(1 - u)^{2}}{u}\\ \hline \text{Neyman }\chi^2 & \int \frac{(p(x) - q(x))^{2}}{q(x)} dx & (u - 1)^{2}\\ \hline \text{Hellinger distance} & \int \left(\sqrt{p(x)} - \sqrt{q(x)}\right)^{2} dx & (\sqrt{u} - 1)^{2}\\ \hline \text{Jeffrey distance} & \int (p(x) - q(x))\log \left(\frac{p(x)}{q(x)}\right) dx & (u - 1)\log u\\ \hline \text{JS divergence} & \frac{1}{2}\int p(x)\log \frac{2 p(x)}{p(x) + q(x)} + q(x)\log \frac{2 q(x)}{p(x) + q(x)} dx & -\frac{u + 1}{2}\log \frac{1 + u}{2} + \frac{u}{2} \log u\\ \hline \end{array}$$

Convex Functions

Above we listed a bunch of distribution measures along with their corresponding $f$, so a natural question is: what do these $f$ have in common?

The answer is:

1. They are all mappings from the non-negative reals to the reals ($\mathbb{R}^* \to \mathbb{R}$);
2. $f(1)=0$;
3. They are all convex functions.

The first point is standard, and the second point $f(1)=0$ guarantees that $\mathcal{D}_f(P\Vert P)=0$. So how should we understand the third point about convex functions? It's actually just a very basic application of a property of convex functions, since convex functions have an extremely important property (Jensen's inequality):

$$\begin{equation}\mathbb{E}\big[f(x)\big]\geq f\big(\mathbb{E}[x]\big)\label{eq:tuhanshu-xingzhi}\end{equation}$$

That is, "the average of the function is greater than the function of the average"—some textbooks directly take this property as the definition of convexity. If $f(u)$ is a smooth function, we generally determine convexity by checking whether the second derivative $f''(u)$ is always non-negative.

Using $\eqref{eq:tuhanshu-xingzhi}$, we have

$$\begin{equation}\begin{aligned}\int q(x) f\left(\frac{p(x)}{q(x)}\right)dx =& \mathbb{E}_{x\sim q(x)} \left[f\left(\frac{p(x)}{q(x)}\right)\right]\\ \geq& f\left(\mathbb{E}_{x\sim q(x)} \left[\frac{p(x)}{q(x)}\right]\right)\\ =& f\left(\int q(x) \frac{p(x)}{q(x)}dx\right)\\ =& f\left(\int p(x)dx\right)\\ =& f(1) = 0 \end{aligned}\end{equation}$$

In other words, these three conditions guarantee that the $f$ divergence is non-negative, and equals zero when the two distributions are exactly identical, which lets $f$... allows $\mathcal{D}_f$ to be used as a simple measure of the difference between distributions. Of course, the $f$ divergence in principle does not guarantee that $P\neq Q$ implies $\mathcal{D}_f(P\Vert Q) \gt 0$. But we typically choose a strictly convex $f$ (i.e., $f''(u)$ is always strictly positive), in which case we can guarantee that $P\neq Q$ implies $\mathcal{D}_f(P\Vert Q)\gt 0$, meaning we get $\mathcal{D}_f(P\Vert Q)=0\,\Leftrightarrow\,P=Q$. (Note: even so, in general $\mathcal{D}_f(P\Vert Q)$ still isn't a "distance" in the strict axiomatic sense, though this isn't really central to this article—just a side note.)

Convex Conjugates

Now let's discuss convex functions from a somewhat more mathematical angle. Generally, let the domain of the convex function be $\mathbb{D}$ (for this article, $\mathbb{D}=\mathbb{R}_+$). Pick any point $\xi$, and find the tangent line to $y=f(u)$ at $u=\xi$, which gives

$$\begin{equation}y = f(\xi) + f'(\xi)(u - \xi)\end{equation}$$

Consider the difference between the two:

$$\begin{equation}h(u) = f(u) - f(\xi) - f'(\xi)(u - \xi)\end{equation}$$

The intuitive meaning of convexity is that the graph of the function always lies above any of its tangent lines, so for a convex function the following always holds:

$$\begin{equation}f(u) - f(\xi) - f'(\xi)(u - \xi)\geq 0\end{equation}$$

Rearranging, we get

$$\begin{equation}f(u) \geq f(\xi) - f'(\xi) \xi + f'(\xi)u\end{equation}$$

Since this inequality always holds, and equality is achievable, we can derive

$$\begin{equation}f(u) = \max_{\xi\in\mathbb{D}}\big\{f(\xi) - f'(\xi) \xi + f'(\xi)u\big\}\end{equation}$$

Introducing new notation, let $t=f'(\xi)$, and solve for $\xi$ in terms of it (for a convex function this is always possible—readers can try to prove this themselves), then define

$$\begin{equation}g(t) = - f(\xi) + f'(\xi) \xi\end{equation}$$

which gives us

$$\begin{equation}f(u) = \max_{t\in f'(\mathbb{D})}\big\{t u - g(t)\big\}\end{equation}$$

Here $g(t)$ is called the conjugate function of $f(u)$. Notice the expression inside the curly braces: once $f$ is given, $g$ is also determined, and the whole expression is linear in $u$. So overall, what we've done is:

Give a linear approximation to a convex function, such that maximizing over the parameter inside recovers the original value.

Note that for a given $u$, we must maximize over $t$ once to get a result as close as possible to $f(u)$; otherwise, if we just plug in an arbitrary $t$, we can only guarantee a lower bound, without any control over the error size. This is why it's called the "local variational method"—because at every point (locally) we need to perform a maximization (a variation). In this way, we can understand $t$ as actually being a function of $u$, i.e.,

$$\begin{equation}f(u) = \max_{T\text{has range}f'(\mathbb{D})\text{function of}}\big\{T(u) u - g(T(u))\big\}\label{eq:max-conj}\end{equation}$$

The discussion above has actually already given us the method for computing the convex conjugate. Here we directly present the conjugate functions corresponding to the convex functions listed in the table above.

$$\begin{array}{c|c}\hline f(u) & \textbf{corresponding conjugate}g(t) & f'(\mathbb{D}) & 激活函数\\ \hline \frac{1}{2}|u - 1| & t & \left[-\frac{1}{2},\frac{1}{2}\right] & \frac{1}{2}\tanh(x)\\ \hline u \log u & e^{t-1} & \mathbb{R} & x\\ \hline - \log u & -1 - \log(-t) & \mathbb{R}_- & -e^{x}\\ \hline \frac{(1 - u)^{2}}{u} & 2 - 2\sqrt{1-t} & (-\infty, 1) & 1-e^x\\ \hline (u - 1)^{2} & \frac{1}{4}t^2+t & (-2,+\infty) & e^x-2\\ \hline (\sqrt{u} - 1)^{2} & \frac{t}{1-t} & (-\infty, 1) & 1-e^x\\ \hline (u - 1)\log u & W(e^{1-t})+\frac{1}{W(e^{1-t})}+t-2 & \mathbb{R} & x\\ \hline -\frac{u + 1}{2}\log \frac{1 + u}{2} + \frac{u}{2} \log u & -\frac{1}{2}\log(2-e^{2t}) & \left(-\infty,\frac{\log 2}{2}\right) & \frac{\log 2}{2}-\frac{1}{2}\log(1+e^{-x})\\ \hline \end{array}$$

(Note: here $W$ is the Lambert W function.)

f-GAN

From the derivation above, we can now give the estimation formula for the f-divergence, and further present the general framework of f-GAN.

f-Divergence Estimation

What's difficult about computing the $f$ divergence? By definition $\eqref{eq:f-div}$, we need to know both probability distributions $P,Q$ simultaneously in order to compute their $f$ divergence. But in practice, this is very hard to achieve in machine learning—sometimes at most we know the analytic form of only one of the two distributions, with the other available only as sampled data; and in many cases we don't know either distribution, only samples from each (that is, we want to compare the similarity between two batches of samples). So we can't directly compute the $\eqref{eq:f-div}$ divergence from $f$.

Combining $\eqref{eq:f-div}$ and $\eqref{eq:max-conj}$, we obtain

$$\begin{equation}\begin{aligned}\mathcal{D}_f(P\Vert Q) =& \max_{T}\int q(x) \left[\frac{p(x)}{q(x)}T\left(\frac{p(x)}{q(x)}\right)-g\left(T\left(\frac{p(x)}{q(x)}\right)\right)\right]dx\\ =& \max_{T}\int\left[p(x)\cdot T\left(\frac{p(x)}{q(x)}\right)-q(x)\cdot g\left(T\left(\frac{p(x)}{q(x)}\right)\right)\right]dx\end{aligned}\end{equation}$$

Writing $T\left(\frac{p(x)}{q(x)}\right)$ collectively as $T(x)$, we then have

$$\begin{equation}\mathcal{D}_f(P\Vert Q) = \max_{T}\Big(\mathbb{E}_{x\sim p(x)}[T(x)]-\mathbb{E}_{x\sim q(x)}[g(T(x))]\Big)\label{eq:f-div-e}\end{equation}$$

Equation $\eqref{eq:f-div-e}$ is the basic formula for estimating the $f$ divergence. It means: sample separately from the two distributions, compute the average values of $T(x)$ and $g(T(x))$ respectively, and optimize $T$ so as to make their difference as large as possible; the resulting value is then an approximation to the $f$ divergence. Clearly $T(x)$ can be fitted with a sufficiently complex neural network, and we just need to optimize the network's parameters.

Note that in our earlier discussion of convex functions, when maximizing the objective, there were constraints on the range of $T$. Therefore, in the last layer of $T$, we must design an appropriate activation function so that $T$ satisfies the required range. Of course, the choice of activation function isn't unique—reference activation functions have already been listed in the earlier table. Note that although in theory the choice of activation function is arbitrary, for ease of optimization we should follow a few principles:

1. The domain should be $\mathbb{R}$, and the range should be the required range (boundary points can be ignored);
2. It's best to choose a globally smooth function, rather than simply truncating—for example, if the required range is $\mathbb{R}_+$, don't just directly use $relu(x)$; instead consider $e^x$ or $\log(1+e^x)$;
3. Note that the second term in equation $\eqref{eq:f-div-e}$ contains $g(T(x))$, i.e., the composition of $g$ and $T$, so when choosing an activation function, it's best to pick one whose composition with $g$ is relatively simple.

GANs Wholesale

Alright, having talked for so long, we're almost at the end of the article, and it seems we still haven't formally gotten to GANs. In fact, GANs turn out to be merely a byproduct of this whole discussion.

A GAN aims to train a generator that maps a Gaussian distribution to the data distribution we want, which requires comparing the difference between two distributions. Given the discussion above, this is now quite simple—just pick any $f$ divergence. Then use equation $\eqref{eq:f-div-e}$ to estimate the $f$ divergence; once estimated, we have a model of the $f$ divergence. At this point, doesn't the generator want to shrink the difference between the distributions? Just minimize the $f$ divergence. So written as an expression, it's

$$\begin{equation}\min_G\max_{T}\Big(\mathbb{E}_{x\sim p(x)}[T(x)]-\mathbb{E}_{x=G(z),z\sim q(z)}[g(T(x))]\Big)\label{eq:f-div-gan}\end{equation}$$

or equivalently, the other way around:

$$\begin{equation}\min_G\max_{T}\Big(\mathbb{E}_{x=G(z),z\sim q(z)}[T(x)]-\mathbb{E}_{x\sim p(x)}[g(T(x))]\Big)\label{eq:f-div-gan-2}\end{equation}$$

And that's it, we're done!

Want a few examples? Alright, let's first try the JS divergence. Substituting everything in step by step, you'll find the final result is (omitting the constant term $\log 2$):

$$\begin{equation}\min_G\max_{D}\Big(\mathbb{E}_{x\sim p(x)}[\log D(x)] + \mathbb{E}_{x=G(z),z\sim q(z)}[\log(1-D(x))]\Big)\end{equation}$$

where $D$ is activated with $\sigma(x)=1/(1+e^{-x})$. This is the original vanilla GAN.

Let's try the Hellinger distance. The result is

$$\begin{equation}\min_G\max_{D}\Big(-\mathbb{E}_{x\sim p(x)}[e^{D(x)}] - \mathbb{E}_{x=G(z),z\sim q(z)}[e^{-D(x)}]\Big)\end{equation}$$

Here $D(x)$ uses a linear activation. This one doesn't seem to have a name yet, but the paper has already run experiments on it.

What about the KL divergence? Since the KL divergence is asymmetric, there are two results:

$$\begin{equation}\min_G\max_{D}\Big(\mathbb{E}_{x\sim p(x)}[D(x)] - \mathbb{E}_{x=G(z),z\sim q(z)}[e^{D(x)-1}]\Big)\end{equation}$$

or

$$\begin{equation}\min_G\max_{D}\Big(\mathbb{E}_{x=G(z),z\sim q(z)}[D(x)] - \mathbb{E}_{x\sim p(x)}[e^{D(x)-1}]\Big)\end{equation}$$

Here $D(x)$ also uses a linear activation.

Alright, I'll stop giving examples. In practice, these $f$ divergences are all essentially similar, and you can't really see much difference in effect. However, it's worth noting that the JS divergence and Hellinger distance are both symmetric and bounded—a very nice property that we'll make use of later.

Summary

Frankly speaking, the main purpose of this article was really just to introduce the $f$ divergence and its local variational estimation—so most of it is theoretical exposition, with GANs occupying only a small portion.

Of course, after all this effort, we really can achieve a "GAN production workshop" (depending on how many $f$ divergences you have). These newly cooked-up GANs might not look like the GANs we're used to imagining, but they are indeed optimizing the $f$ divergence. That said, the problems that standard GANs (corresponding to the JS divergence) have historically faced will still appear with the $f$ divergence. So the greater value of the f-GAN work lies in "unification"—from the perspective of generative modeling, it doesn't really represent a breakthrough.