Efficient GlobalPointer: Fewer Parameters, Better Results

Efficient GlobalPointer：少点参数，多点效果 苏剑林 · 2022-01-25 · Machine Learning model NLP NER

In GlobalPointer: A Unified Approach to Nested and Flat NER, we proposed a token-pair recognition module called "GlobalPointer" which, when applied to NER, can uniformly handle both nested and flat tasks, achieving faster speed than CRF in flat scenarios while matching its performance. In other words, based on current experimental results, at least for NER, we can confidently replace CRF with GlobalPointer without worrying about losing anything in terms of accuracy or speed.

In this post, we propose an improved version of GlobalPointer — Efficient GlobalPointer — which mainly addresses the low parameter efficiency of the original GlobalPointer, significantly reducing its parameter count. More interestingly, experimental results on multiple tasks show that this leaner Efficient GlobalPointer actually achieves better results.

A Large Number of Parameters

Let's briefly review GlobalPointer here; for a detailed introduction, please refer to GlobalPointer: A Unified Approach to Nested and Flat NER. In short, GlobalPointer is a token-pair recognition module based on inner products, and it can be applied to NER since for NER we only need to identify token pairs of the form "(start, end)" for each entity type. more

Suppose an input $t$ of length $n$ is encoded to obtain the vector sequence $[\boldsymbol{h}_1,\boldsymbol{h}_2,\cdots,\boldsymbol{h}_n]$. The original GlobalPointer applies transformations $\boldsymbol{q}_{i,\alpha}=\boldsymbol{W}_{q,\alpha}\boldsymbol{h}_i$ and $\boldsymbol{k}_{i,\alpha}=\boldsymbol{W}_{k,\alpha}\boldsymbol{h}_i$ to obtain vector sequences $[\boldsymbol{q}_{1,\alpha},\boldsymbol{q}_{2,\alpha},\cdots,\boldsymbol{q}_{n,\alpha}]$ and $[\boldsymbol{k}_{1,\alpha},\boldsymbol{k}_{2,\alpha},\cdots,\boldsymbol{k}_{n,\alpha}]$, and then defines

\begin{equation}s_{\alpha}(i,j) = \boldsymbol{q}_{i,\alpha}^{\top}\boldsymbol{k}_{j,\alpha}\end{equation}

as the score for the continuous span from $i$ to $j$ being an entity of type $\alpha$. Here we've temporarily omitted the bias term; feel free to add it if you think it's necessary.

This means that for however many entity types there are, there are that many $\boldsymbol{W}_{q,\alpha}$ and $\boldsymbol{W}_{k,\alpha}$. Suppose $\boldsymbol{W}_{q,\alpha},\boldsymbol{W}_{k,\alpha}\in\mathbb{R}^{d\times D}$; then every new entity type adds $2Dd$ parameters. In contrast, with CRF + BIO tagging, every new entity type only adds $2D$ parameters (the transition matrix has relatively few parameters, so we can ignore it). For BERT base, a common choice is $D=768,d=64$, so it's clear that GlobalPointer's parameter count is far larger than CRF's.

Recognition and Classification

In fact, it's not hard to imagine that for any type $\alpha$, the scoring matrix $s_{\alpha}(i,j)$ must share a lot of similarities, because for most token pairs, they represent "non-entities," and the correct scores for these non-entities are all negative. This means that we don't really need to design an independent $s_{\alpha}(i,j)$ for every entity type — they should share more in common.

How do we bring out the shared structure among $s_{\alpha}(i,j)$? Take NER as an example: we know that NER can actually be decomposed into two steps, "extraction" and "classification." "Extraction" means extracting the spans that are entities, while "classification" means determining the type of each entity. From this perspective, the "extraction" step is equivalent to NER with only a single entity type, which can be handled with a single scoring matrix, namely $(\boldsymbol{W}_q\boldsymbol{h}_i)^{\top}(\boldsymbol{W}_k\boldsymbol{h}_j)$. The "classification" step, meanwhile, can be handled via "feature concatenation + a dense layer," i.e., $\boldsymbol{w}_{\alpha}^{\top}[\boldsymbol{h}_i;\boldsymbol{h}_j]$. We can then combine these two terms into a new scoring function:

\begin{equation}s_{\alpha}(i,j) = (\boldsymbol{W}_q\boldsymbol{h}_i)^{\top}(\boldsymbol{W}_k\boldsymbol{h}_j) + \boldsymbol{w}_{\alpha}^{\top}[\boldsymbol{h}_i;\boldsymbol{h}_j]\label{eq:EGP-1}\end{equation}

In this way, the parameters for the "extraction" part are shared across all entity types, so every new entity type only requires adding the corresponding $\boldsymbol{w}_{\alpha}\in\mathbb{R}^{2D}$ — meaning the parameter increase per new entity type is just $2D$. Going further, let's write $\boldsymbol{q}_i=\boldsymbol{W}_q\boldsymbol{h}_i, \boldsymbol{k}_i=\boldsymbol{W}_k\boldsymbol{h}_i$, and then, to reduce the parameter count even further, we can replace $\boldsymbol{h}_i$ with $[\boldsymbol{q}_i;\boldsymbol{k}_i]$, giving us

\begin{equation}s_{\alpha}(i,j) = \boldsymbol{q}_i^{\top}\boldsymbol{k}_j + \boldsymbol{w}_{\alpha}^{\top}[\boldsymbol{q}_i;\boldsymbol{k}_i;\boldsymbol{q}_j;\boldsymbol{k}_j]\label{eq:EGP}\end{equation}

Now $\boldsymbol{w}_{\alpha}\in\mathbb{R}^{4d}$, so the parameter increase per new entity type is $4d$. Since typically $d \ll D$, equation $\eqref{eq:EGP}$ usually has fewer parameters than equation $\eqref{eq:EGP-1}$ — and this is exactly the scoring function ultimately used by Efficient GlobalPointer.

A Pleasant Surprise in the Experiments

Efficient GlobalPointer has already been built into bert4keras>=0.10.9, so readers only need to change a single line of code to switch to it.

# from bert4keras.layers import GlobalPointer
from bert4keras.layers import EfficientGlobalPointer as GlobalPointer

Let's now compare the results of GlobalPointer and Efficient GlobalPointer:

$$\begin{array}{c} \text{People's Daily NER experiment results} \\ {\begin{array}{c|cc} \hline & \text{val set F1} & \text{test set F1}\\ \hline \text{CRF} & 96.39\% & 95.46\% \\ \text{GlobalPointer} & \textbf{96.25%} & \textbf{95.51%} \\ \text{Efficient GlobalPointer} & 96.10\% & 95.36\%\\ \hline \end{array}$$} \\ \\

\text{CLUENER results} \\

{$$\begin{array}{c|cc} \hline & \text{val set F1} & \text{test set F1} \\ \hline \text{CRF} & 79.51\% & 78.70\% \\ \text{GlobalPointer} & 80.03\% & 79.44\%\\ \text{Efficient GlobalPointer} & \textbf{80.66%} & \textbf{80.04%} \\ \hline \end{array}$$} \\ \\

\text{CMeEE results} \\

{$$\begin{array}{c|cc} \hline & \text{val set F1} & \text{test set F1} \\ \hline \text{CRF} & 63.81\% & 64.39\% \\ \text{GlobalPointer} & 64.84\% & 65.98\%\\ \text{Efficient GlobalPointer} & \textbf{65.16%} & \textbf{66.54%} \\ \hline \end{array}$$}

\end{array}

As we can see, Efficient GlobalPointer's results are quite good overall. Aside from a slight drop on the People's Daily task, it achieves gains on the other two tasks, and the overall size of the gains outweighs the size of the drop — so Efficient GlobalPointer doesn't just save parameters, it also improves performance. In terms of speed, Efficient GlobalPointer is essentially indistinguishable from the original GlobalPointer.

Analysis and Discussion

Considering that the People's Daily NER task has only 3 entity types, while CLUENER and CMeEE have 10 and 9 entity types respectively, and that People's Daily scores higher than the other two tasks, this suggests that CLUENER and CMeEE are more difficult tasks. On the other hand, Efficient GlobalPointer achieves improvements on both CLUENER and CMeEE, so we can tentatively conclude that: the more entity types there are and the harder the task, the more effective Efficient GlobalPointer becomes.

This isn't hard to understand. The original GlobalPointer has too many parameters, so on average each parameter gets sparser updates, making it relatively more prone to overfitting. Efficient GlobalPointer, on the other hand, shares the "extraction" parameters and distinguishes between entity types only through the much smaller "classification" parameters — so the learning of the extraction step becomes more thorough, while the classification step, having fewer parameters, is also easier to learn. Conversely, the strong experimental performance of Efficient GlobalPointer also indirectly validates the soundness of the decomposition in equation $\eqref{eq:EGP}$.

Of course, we can't rule out the possibility that the original GlobalPointer would achieve better results given sufficient training data. But even so, when the number of categories is large, the original GlobalPointer may consume so much GPU memory that it becomes impractical to use. Taking the base version $D=768,d=64$ as an example again: if there were 100 categories, the original GlobalPointer's parameter count would be $2\times 768\times 64\times 100$ — close to ten million — which, it has to be said, is not very friendly at all.

Final Summary

This post pointed out the low parameter efficiency of the original GlobalPointer and proposed a corresponding improvement, Efficient GlobalPointer. Experimental results show that Efficient GlobalPointer reduces the parameter count while essentially preserving — and sometimes even improving — performance.