Ladder Side-Tuning: A "Ladder Over the Wall" for Pretrained Models

Ladder Side-Tuning：预训练模型的“过墙梯” 苏剑林 · 2022-06-20 · Machine Learning language model pretraining

If large pretrained models are the "clever schemes" ("Zhang Liang's stratagems") of natural language processing, then what would be the corresponding "ladder to get over the wall"? In my view, it's the various techniques for efficiently fine-tuning these large models on specific tasks. Besides directly fine-tuning all the parameters, there are also many parameter-efficient fine-tuning tricks like Adapter and P-Tuning, which can achieve results close to full-parameter fine-tuning while only fine-tuning a small number of parameters. However, these techniques are usually only "parameter-efficient" rather than "training-efficient," because they still require backpropagation through the entire model to obtain gradients for the small set of trainable parameters. In plain terms, the number of trainable parameters is indeed much smaller, but the training speed doesn't improve noticeably.

A recent paper, LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning, proposes a new training technique called "Ladder Side-Tuning (LST)," which claims to achieve both parameter efficiency and training efficiency at the same time. Is there really such an ideal "ladder over the wall"? Let's find out together.

Overview of the Method

The structure of this LST "ladder" can actually be explained quite clearly using Figure 2 from the original paper:

Comparison of LST with Adapter and P-tuning

Backpropagation—that is, computing the model's gradients—is calculated step by step from the output layer back to the input layer. Therefore, the depth/computational cost of backpropagation depends on the depth of the trainable parameters closest to the input layer, and isn't necessarily tied to the number of trainable parameters. For Adapter, a small-scale layer is inserted after every layer; although the rest of the parameters are frozen and only the newly inserted layers are trainable, since new layers appear at every layer, backpropagation still has to reach all the way back to the input layer. For P-tuning, essentially the only trainable parameters live in the embedding layer, but since the embedding layer is the input layer, its backpropagation also has to run through the entire model. As a result, neither of these two approaches improves training efficiency by much.

As for LST, it builds a "side branch" (a ladder) on top of the original large model, using the outputs of some of the large model's layers as inputs to this side-branch model. All the trainable parameters reside in the side-branch model, and since the large model only supplies inputs, the complexity of backpropagation depends on the size of the side-branch model — there's no need to perform backpropagation directly on the original large model. This is what allows for a clear improvement in training efficiency.

Experimental Results

The original paper conducted quite a few experiments with LST, covering both NLP and CV. Below are LST's results on the GLUE dataset:

LST's experimental results on GLUE

As we can see, LST indeed exhibits both parameter efficiency and training efficiency, achieving decent fine-tuning results with a relatively small number of trainable parameters and low training cost. The results in the last two rows in particular illustrate the feasibility of fine-tuning large models under constrained training resources using LST.

I also did a simple experiment on Chinese CLUE tasks; the reference code is here:

GitHub: https://github.com/bojone/LST-CLUE

Note that in the original paper, the "ladder" is built using MLP layers similar to those in Adapter, whereas in my implementation above, I directly used an "Attention + FFN" combination just like a Transformer. The number of trainable parameters is kept at around 1 million, which is about 1.2% of the base version or 0.4% of the large version. The ladder is initialized with plain random initialization. The final results on the validation set are as follows:

$$\small{\begin{array}{c|ccccccccccc} \hline & \text{iflytek} & \text{tnews} & \text{afqmc} & \text{cmnli} & \text{ocnli} & \text{wsc} & \text{csl} & \text{cmrc2018} & \text{c3} & \text{chid} & \text{cluener}\\ \hline \text{BERT base} & 60.06 & 56.80 & 72.41 & 79.56 & 73.93 & 78.62 & 83.93 & 56.17 & 60.54 & 85.69 & 79.45 \\ \text{RoBERTa base} & 60.64 & 58.06 & 74.05 & 81.24 & 76.00 & 87.50 & 84.50 & 56.54 & 67.66 & 86.71 & 79.47\\ \hline \text{RoBERTa base + LST} & 59.29 & 56.82 & 70.37 & 76.27 & 71.02 & 68.09 & 82.63 & 42.50 & 56.97 & 69.35 & 78.30\\ \text{RoBERTa large + LST} & 60.41 & 57.12 & 72.36 & 75.80 & 72.07 & 75.00 & 84.23 & 39.98 & 60.19 & 72.55 & 77.80\\ \hline \end{array}}$$

As we can see, the experimental results aren't as optimistic as those in the original paper's English experiments (of course, this could just be due to my own implementation not being good enough), but the training efficiency does show a clear improvement (roughly a twofold speedup on average). Based on the overall experiment, my impression is that for fairly standard, moderate-difficulty classification tasks, LST can achieve comparable results, but for more difficult tasks, such as reading comprehension, LST shows a quite noticeable drop in performance.

Of course, this is probably not a problem unique to LST — most methods claiming to be parameter-efficient fine-tuning approaches likely have this same issue, since most of their experimental tasks are just GLUE, and GLUE consists almost entirely of relatively simple classification tasks...

Further Thoughts

With the benefit of hindsight, LST isn't really that clever an idea — at its core, it just freezes the pretrained model and trains a new small model using the outputs of its final layer and some intermediate layers as auxiliary inputs. Once you understand this, many readers have probably already started brewing up their own similar schemes in their heads. That said, the real significance of LST lies in showing us that this approach is workable, providing a feasible reference design, and experimentally demonstrating that it is indeed an effective way to make use of a large pretrained model.

Readers with experience in similar research will notice that the initialization of the newly added "ladder" branch in LST is an issue: if it's fully randomly initialized, training difficulties may arise, leading to suboptimal results. The original paper mentions this too, and provides a scheme where the small model's matrix weights are initialized by extracting weights from the large model's matrices, which improves LST's final performance. The details can be found in the paper; as for my own implementation, since it was purely meant to verify LST's effectiveness, I was lazy and skipped this step.

Taking this a step further: since the newly added "ladder" branch in LST has this initialization difficulty, and LST is indeed an effective scheme for fine-tuning large models, could we not, when training new large models in the future, reserve this "ladder" ahead of time? In other words, we could directly make this "ladder" part of the pretrained model and pretrain it at scale, so that during later fine-tuning we'd only need to fine-tune the "ladder." This would let us fine-tune large models efficiently without having to worry about the initialization problem.

Structurally speaking, I feel that LST bears quite a resemblance to BERT-of-Theseus, introduced in BERT-of-Theseus: A Model Compression Method Based on Module Replacement, except that one's goal is to distill a small model — which still requires backpropagating through the large model — while LST's goal is to improve training efficiency, requiring no backpropagation through the large model, though the large model is still needed for the forward pass at inference time. In a sense, the two are somewhat complementary.

Summary

This article mainly introduced Ladder Side-Tuning, a fine-tuning method for large models that achieves both parameter efficiency and training efficiency at the same time.