A Soft-Label Version of Multi-label "Softmax + Cross-Entropy"

多标签“Softmax+交叉熵”的软标签版本 苏剑林 · 2022-05-07 · Machine Learning optimization loss function smoothness

(Note: the content of this post has been written up as the paper ZLPR: A Novel Loss for Multi-label Classification; if you wish to cite this work, please cite that English paper directly. Thanks.)

In Generalizing "Softmax + Cross-Entropy" to Multi-label Classification Problems, we proposed a loss function for multi-label classification:

\begin{equation}\log \left(1 + \sum\limits_{i\in\Omega_{neg}} e^{s_i}\right) + \log \left(1 + \sum\limits_{j\in\Omega_{pos}} e^{-s_j}\right)\label{eq:original}\end{equation}

This loss function inherits the nice properties of "Softmax + cross-entropy" from single-label classification—in particular, it still works well even when positive and negative classes are severely imbalanced. However, judging from its form, it only applies to "hard labels," which means techniques such as label smoothing and mixup cannot be used with it directly. This post attempts to solve that problem by proposing a soft-label version of the above loss.

An Ingenious Connection

The classical approach to multi-label classification is to reduce it to a set of binary classification problems: each category is activated with a sigmoid function $\sigma(x)=1/(1+e^{-x})$ and then trained with its own binary cross-entropy loss. When positive and negative classes are extremely imbalanced, this approach tends to perform poorly, and in comparison, the loss $\eqref{eq:original}$ is usually a better choice. more

In the comments section of the previous post, reader @wu.yan revealed an ingenious connection between multiple "sigmoid + binary cross-entropy" losses and equation $\eqref{eq:original}$: the sum of multiple "sigmoid + binary cross-entropy" losses can be appropriately rewritten as

\begin{equation}\begin{aligned} &\,-\sum_{j\in\Omega_{pos}}\log\sigma(s_j)-\sum_{i\in\Omega_{neg}}\log(1-\sigma(s_i))\\ =&\,\log\prod_{j\in\Omega_{pos}}(1+e^{-s_j})+\log\prod_{i\in\Omega_{neg}}(1+e^{s_i})\\ =&\,\log\left(1+\sum_{j\in\Omega_{pos}}e^{-s_j}+\cdots\right)+\log\left(1+\sum_{i\in\Omega_{neg}}e^{s_i}+\cdots\right) \end{aligned}\label{eq:link}\end{equation}

Comparing this with equation $\eqref{eq:original}$, we can see that equation $\eqref{eq:original}$ is exactly what you get from the sum of "sigmoid + binary cross-entropy" losses above, once you drop the higher-order term represented by $\cdots$! When positive and negative classes are imbalanced, these higher-order terms carry disproportionately large weight, which aggravates the imbalance problem and hurts performance. Conversely, dropping these higher-order terms does not change what the loss function is actually trying to achieve (namely, that the score of positive classes should exceed 0 and the score of negative classes should be below 0); moreover, since the number of terms being summed inside the parentheses scales linearly with the number of classes, the loss contributions from positive and negative classes end up being of comparable magnitude.

A Guess at the Form

This ingenious connection tells us that to find a soft-label version of equation $\eqref{eq:original}$, we can start from the soft-label version of the sum of "sigmoid + binary cross-entropy" losses, and then try dropping the higher-order term. By "soft label" we mean that the label is no longer restricted to 0 or 1, but can be any real number in between, representing the probability of belonging to that class. For binary cross-entropy, the soft-label version is simple:

\begin{equation}-t\log\sigma(s)-(1-t)\log(1-\sigma(s))\end{equation}

Here $t$ is the soft label, and $s$ is the corresponding score. Mimicking the derivation $\eqref{eq:link}$, we obtain

\begin{equation}\begin{aligned} &\,-\sum_i t_i\log\sigma(s_i)-\sum_i (1-t_i)\log(1-\sigma(s_i))\\ =&\,\log\prod_i(1+e^{-s_i})^{t_i}+\log\prod_i (1+e^{s_i})^{1-t_i}\\ =&\,\log\prod_i(1+t_i e^{-s_i} + \cdots)+\log\prod_i (1+(1-t_i)e^{s_i}+\cdots)\\ =&\,\log\left(1+\sum_i t_i e^{-s_i}+\cdots\right)+\log\left(1+\sum_i(1-t_i)e^{s_i}+\cdots\right) \end{aligned}\end{equation}

Dropping the higher-order term, we get

\begin{equation}\log\left(1+\sum_i t_i e^{-s_i}\right)+\log\left(1+\sum_i(1-t_i)e^{s_i}\right)\label{eq:soft}\end{equation}

This is our candidate form for the soft-label version of equation $\eqref{eq:original}$, and indeed it can be verified that when $t_i\in\{0,1\}$, it reduces exactly to equation $\eqref{eq:original}$.

Proving the Result

So far, equation $\eqref{eq:soft}$ is at best a "candidate" form. To promote it to something we can trust, we need to show that when $t_i$ is a floating-point number between 0 and 1, equation $\eqref{eq:soft}$ can still learn something meaningful. By "meaningful" we mean that, in principle, $s_i$ should be able to reconstruct the information in $t_i$ (where $s_i$ is the model's prediction and $t_i$ is the given label, so having $s_i$ reconstruct $t_i$ is exactly the goal of machine learning).

To this end, let us denote equation $\eqref{eq:soft}$ as $l$, and compute the partial derivatives of $s_i$:

\begin{equation}\frac{\partial l}{\partial s_i} = \frac{-t_i e^{-s_i}}{1+\sum\limits_i t_i e^{-s_i}}+\frac{(1-t_i)e^{s_i}}{1+\sum\limits_i(1-t_i)e^{s_i}}\end{equation}

We know that the minimum of $l$ occurs when all $\frac{\partial l}{\partial s_i}$ equal 0. Directly solving the system of equations $\frac{\partial l}{\partial s_i}=0$ is not easy, but the author noticed a curious "coincidence": when $t_i e^{-s_i}=(1-t_i)e^{s_i}$, each $\frac{\partial l}{\partial s_i}$ automatically equals 0! So $t_i e^{-s_i}=(1-t_i)e^{s_i}$ should indeed be the optimal solution to $l$, giving us

\begin{equation}t_i = \frac{1}{1+e^{-2s_i}}=\sigma(2s_i)\end{equation}

This is a beautiful result, and it tells us several things:

1. Equation $\eqref{eq:soft}$ is indeed a reasonable soft-label generalization of equation $\eqref{eq:original}$: it can completely reconstruct the information in $s_i$ from $t_i$, and its form happens to be related to the sigmoid function;
2. If we want to output the result as a probability value between 0 and 1, the correct thing to compute is $\sigma(2s_i)$, rather than the intuitive-seeming $\sigma(s_i)$;
3. Since the final probability formula also takes a sigmoid form, we can view this the other way around: we are still essentially learning multiple sigmoid-activated binary classification problems, except that the loss function has been replaced by equation $\eqref{eq:soft}$.

Implementation Tips

For the implementation of equation $\eqref{eq:soft}$, you can refer to the bert4keras code for multilabel_categorical_crossentropy, which contains a small detail worth discussing.

First, equation $\eqref{eq:soft}$ can be equivalently rewritten as

\begin{equation}\log\left(1+\sum_i e^{-s_i + \log t_i}\right)+\log\left(1+\sum_i e^{s_i + \log (1-t_i)}\right)\label{eq:soft-log}\end{equation}

So it looks like all we need to do is add $\log t_i$ to $-s_i$, add $\log(1-t_i)$ to $s_i$, pad with a zero, and then apply the standard $\text{logsumexp}$. But in practice, $t_i$ can potentially equal $0$ or $1$, in which case the corresponding $\log t_i$ or $\log(1-t_i)$ becomes negative infinity, and frameworks cannot handle negative infinity directly. So we usually need to clip before applying $\log$: that is, we pick a $\epsilon > 0$ and define

\begin{equation}\text{clip}(t)=\left\{\begin{aligned}&\epsilon, &t < \epsilon \\ &t, &\epsilon\leq t\leq 1-\epsilon\\ &1-\epsilon, &t > 1-\epsilon\end{aligned}\right.\end{equation}

But this clipping introduces a new problem. Since $\epsilon$ is not truly infinitesimally small—say $\epsilon=10^{-7}$—then $\log\epsilon$ is roughly around $-16$. Meanwhile, in scenarios like GlobalPointer, we mask out invalid positions $s_i$ in advance by setting the corresponding $s_i$ to a large-magnitude negative number, e.g., $-10^7$. Now look again at equation $\eqref{eq:soft-log}$: the first summation term ranges over $e^{-s_i + \log t_i}$, so $-10^7$ becomes $10^7$. If $t_i$ were not clipped, then in principle $\log t_i$ would be $\log 0 = -\infty$, which could turn $-s_i + \log t_i$ back into negative infinity. But as we've just seen, once $\log t_i$ has been clipped, it is at most $-16$, which is nowhere near large enough compared with $-s_i$'s $10^7$, so $-s_i + \log t_i$ remains a large positive number.

To fix this problem, it's not enough to just clip $t_i$—we also need to identify entries of $\epsilon$ that were originally less than $t_i$ and manually set the corresponding $-s_i$ to a large-magnitude negative number, and likewise identify entries of $1-\epsilon$ greater than $t_i$ and set the corresponding $s_i$ to a large-magnitude negative number. This effectively treats values below $\epsilon$ as exactly 0 and values above $1-\epsilon$ as exactly 1.

Summary

This post generalizes the multi-label "Softmax + cross-entropy" loss I proposed earlier to the soft-label setting. With this soft-label version in hand, we can now combine it with techniques such as label smoothing and mixup, and models like GlobalPointer gain yet another direction to explore in the training process.