Building a DialoGPT from Scratch: An LM-Based Generative Multi-Turn Dialogue Model

动手做个DialoGPT：基于LM的生成式多轮对话模型 苏剑林 · 2020-09-07 · Machine Learning language model text generation attention

A while back, while browsing Arxiv, I noticed that Tsinghua University had open-sourced a large-scale Chinese chit-chat corpus called LCCC (paper link, project page). Judging from the released files, this is likely the largest and highest-quality open-source chit-chat corpus available at the moment, and it even includes some multi-turn conversations — overall, quite a lot of fun to play with. I got drawn in myself and tried using it to train a chit-chat dialogue model. The results looked pretty good, so let me share my experience here.

Introduction to the Corpus

Let me briefly introduce this dataset, LCCC (Large-scale Cleaned Chinese Conversation); for details, you can check out the GitHub repo, whose link is given above. LCCC comes in two versions, base and large. The base version mainly comes from Weibo conversations, while large builds on base by incorporating other open-source dialogue corpora. According to the authors, LCCC went through a rigorous cleaning process, so overall the quality looks quite solid.

$$\begin{array}{c|c|c} \hline \text{LCCC-base} & \text{single-turn dialogue} & \text{multi-turn dialogue} \\ \hline \text{total dialogue turns} & \text{3,354,382} & \text{3,466,607}\\ \hline \text{total dialogue utterances} & \text{6,708,554} & \text{13,365,268}\\ \hline \end{array}$$$$\begin{array}{c|c|c} \hline \text{LCCC-large} & \text{single-turn dialogue} & \text{multi-turn dialogue} \\ \hline \text{total dialogue turns} & \text{7,273,804} & \text{4,733,955}\\ \hline \text{total dialogue utterances} & \text{14,547,608} & \text{18,341,167}\\ \hline \end{array}$$

To simplify the task, all samples were processed into two-party dialogues. Here are a few examples:

A: Let's buy some rabbit heads when we go back for New Year and have a proper hotpot
B: I haven't seen any good rabbit heads in Taiyuan
A: I'll bring you some back from Hongqiao, I spotted an authentic place the other day
B: Love you the most
A: Of course
A: Mhm, I'll wait a bit more! Are you in Shanghai right now? The wind in Shanghai seems even stronger than in Nanjing, better not go out much
B: Yeah, I'm home, nothing going on. Be careful out there!
A: I went back for a trip last year too, ran into my old PE teacher, even took a photo together
B: Haha I went to look for my first-year English teacher too but couldn't find her, she must have had something on and wasn't at school
A: You're really digging up the memories, huh
B: Haha, haven't been back since graduating, wanted to go take a look

Model Design

Now that we know what the data looks like, the next step is to design the model. Clearly, what we need to do is train a model that predicts what to reply next. Since the corpus contains multi-turn conversations, we also need the model to support multi-turn dialogue. The simplest way to account for conversation history is to concatenate all prior turns up to the current one into a single piece of text, which then serves as the model's input.

Given some input, predicting an output — on the surface this looks like a job for a Seq2Seq model. Using Seq2Seq directly isn't really a problem per se, but standard Seq2Seq models are generally used for inputs and outputs with fairly fixed shapes — for instance, the input text length is expected to stay within some limited range and shouldn't vary too much. But when we consider multi-turn dialogue, in principle we have no idea how many turns came before, so the input length is, in principle, unbounded. There's also an efficiency problem with using Seq2Seq: in each turn we can only train on one reply at a time, so if a multi-turn dialogue has $n$ replies, it needs to be split into $n$ separate training samples.

So what we need is a model whose length can vary quite freely, and which can predict an entire multi-turn conversation at once. A reasonably good choice for this is a unidirectional language model (LM, GPT-style), as illustrated below:

Diagram of using a unidirectional language model for multi-turn dialogue

As shown in the figure, we choose the currently mainstream Transformer architecture and, following BERT's usual input format, concatenate each utterance with [SEP], then train a left-to-right unidirectional language model on top of that. To distinguish between different speaker roles, we use different Segment IDs for different speakers. Furthermore, considering that both BERT and GPT use absolute positional encoding — which imposes an upper bound on the text length that can be handled, whereas the number of dialogue turns is in principle unbounded — we instead adopt NEZHA, which uses relative position embeddings, as the base architecture, and initialize the model with NEZHA's pretrained weights.

In plain terms, this just means adding a lower-triangular attention mask to NEZHA, turning it into a language model. For more on this, see From Language Models to Seq2Seq: Transformer as Theater, All Thanks to Masking.

Training Details

First, here's my reference implementation along with the trained model:

https://github.com/bojone/nezha_gpt_dialog

The code is implemented with bert4keras, and the base architecture is NEZHA Base. To make training more "budget-friendly," I only used a single GPU (a 22G RTX) together with gradient accumulation; readers with less GPU memory can reduce the batch size accordingly. The released model was trained with the Adam optimizer at a learning rate of $2\times 10^{-5}$ and a weight decay rate of $0.01$, for 10 million steps (a bit over 20 days), with a per-step batch size of 16 and 16 accumulation steps — equivalent to training with an effective batch size of 256 for 625,000 steps. The final loss came out to about 2.83 (average cross-entropy per token).

Let's test out our training results.

print(chatbot.response([u'我最近天天吃火锅']))
火锅好吃啊
print(chatbot.response([u'我最近天天吃火锅']))
我也是
print(chatbot.response([u'我最近天天吃火锅']))
哈哈哈哈哈哈我最近一周吃了两顿火锅
print(chatbot.response([u'我最近天天吃火锅']))
我也是，每次都吃的撑
print(chatbot.response([u'我最近天天吃火锅']))
吃火锅不长胖
print(chatbot.response([u'我最近天天吃火锅']))
火锅好啊
print(chatbot.response([u'我最近天天吃火锅']))
你们那边有这种火锅？
print(chatbot.response([u'我最近天天吃火锅']))
火锅好吃吗？我也想吃
print(chatbot.response([u'我最近天天吃火锅']))
吃吧，吃了就开心了
print(chatbot.response([u'我最近天天吃火锅']))
我也想天天吃火锅！
print(chatbot.response([u'我最近天天吃火锅']))
火锅吃不了多久的。
print(chatbot.response([u'你那天气好吗']))
还好，不热，有点冷
print(chatbot.response([u'你那天气好吗', u'还好，不热，有点冷', u'那你得穿多厚的衣服']))
我穿了两件

(For reference, the exchanges above translate roughly to: "I've been eating hotpot every day lately" → various replies like "Hotpot is delicious," "Me too," "Haha I had hotpot twice this week," "Eating hotpot doesn't make you fat," and so on; and "Is the weather good where you are?" → "Not bad, not hot, a bit cold" → "Then you must be wearing thick clothes" → "I'm wearing two layers.")

Comparative Analysis

CDial-GPT has also open-sourced its own pretrained model, which I've likewise converted into a format loadable by bert4keras (CDial-GPT-tf); readers are welcome to test and compare it themselves. In terms of training setup, CDial-GPT's model is implemented in PyTorch, with a base architecture of GPT Base. It used 4 2080Ti GPUs, a total batch size of 32, and 64 accumulation steps. The paper states it was trained for 30 epochs, roughly 21 million total steps (about twice as many as mine), which works out to roughly equivalent to training with a batch size of 2048 for 330,000 steps.

The input design also differs in CDial-GPT, as shown below:

Diagram of the CDial-GPT model

As shown in the figure, the main difference between CDial-GPT and the design described above lies in how multi-turn utterances are concatenated. We connect them directly with [SEP], whereas CDial-GPT connects them using role markers like [speaker1] and [speaker2] (abbreviated S1, S2 in the figure), with a single [SEP] at the end marking the end of the reply. As a result, since the format of the segment being predicted differs from the format of the history, only one reply can be trained on at a time, meaning multi-turn dialogues must be split into multiple training samples — in principle this adds training complexity, since multiple steps are needed to fully train on a single multi-turn sample.

As for actual performance, my personal impression from testing is that there's no obvious difference between the two. Interested readers are welcome to compare and test for themselves.

Summary

This post shared an exercise in building a dialogue model: based on the open-source LCCC chit-chat corpus from CDial-GPT, I used a language model (GPT) to generatively model multi-turn dialogue, obtaining a fairly general-purpose chit-chat dialogue model. Finally, I compared the approach described here with CDial-GPT's own open-sourced model.