A Couplet Robot Based on CNN and Sequence Labeling

基于CNN和序列标注的对联机器人 苏剑林 · 2019-01-14 · Machine Learning NLP text generation

Origin

A few days ago, I saw an article on the Qbitai WeChat account, "This whimsical couplet AI has everyone playing around with it", and I found it quite interesting. What's rare is that the author also organized and released the dataset, so I decided to give it a try myself.

Getting Started

"Composing couplets" can be viewed as a sentence generation task, which can be tackled with seq2seq — much like what I wrote before in "Playing with Keras' seq2seq for Automatic Title Generation", just with a slight tweak to the input. The method used in the article mentioned above is also seq2seq, so this seems to be the standard approach. more

Analysis

However, on further reflection, we notice that compared with general sentence generation tasks, "composing couplets" is much more regular: 1) the first line and the second line have the same number of characters; 2) almost every character in the first line has a corresponding character in the second line. Given this, composing couplets can actually be treated directly as a sequence labeling task, in the same way as word segmentation or named entity recognition. This is the starting point of this post.

Having said that, this post really doesn't involve much technical sophistication — sequence labeling is about as ordinary a task as they come, and it's much simpler than general seq2seq. By sequence labeling, I mean: given an input vector sequence, we output another vector sequence of usually the same length, and then classify each "frame" of that sequence. Readers can learn more about the related concepts in "A Concise Introduction to Conditional Random Fields (with a Pure Keras Implementation)".

Model

This post will introduce the model directly alongside the code. Readers who want to dig further into the underlying background knowledge can also refer to "Chinese Word Segmentation Series 4: seq2seq Character Tagging Based on Bidirectional LSTM", "Chinese Word Segmentation Series 6: Chinese Word Segmentation Based on Fully Convolutional Networks", and "A Poetry-Writing Robot Based on CNN and VAE: Random Poem Generation".

The model code we use is as follows:

x_in = Input(shape=(None,))
x = x_in
x = Embedding(len(chars)+1, char_size)(x)
x = Dropout(0.25)(x)

x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)

x = Dense(len(chars)+1, activation='softmax')(x)

model = Model(x_in, x)
model.compile(loss='sparse_categorical_crossentropy',
              optimizer='adam')

Here gated_resnet is the gated convolution module I defined (this module was also introduced in "A Reading-Comprehension-Style QA Model Based on CNN: DGCNN"):

def gated_resnet(x, ksize=3):
    # 门卷积 + 残差
    x_dim = K.int_shape(x)[-1]
    xo = Conv1D(x_dim*2, ksize, padding='same')(x)
    return Lambda(lambda x: x[0] * K.sigmoid(x[1][..., :x_dim]) \
                            + x[1][..., x_dim:] * K.sigmoid(-x[1][..., :x_dim]))([x, xo])

That's all there is to it~

That's really the whole thing — everything else is just data preprocessing. Of course, readers can also try replacing gated_resnet with an ordinary bidirectional LSTM, but in my experiments I found that the bidirectional LSTM didn't perform as well as gated_resnet, and LSTM is also relatively slower, so LSTM was abandoned here.

Results

The training dataset comes from: https://github.com/wb14123/couplet-dataset, with thanks to the author for compiling it.

Full code:

https://github.com/bojone/seq2seq/blob/master/couplet_by_seq_tagging.py

Training process:

Couplet robot training process

Sample results:

First line: 晚风摇树树还挺 (Evening wind sways the trees, and the trees stand firm), Second line: 夜雨敲花花更香 (Night rain taps the flowers, and the flowers grow more fragrant)
First line: 今天天气不错 (The weather is nice today), Second line: 昨日人情无明 (Yesterday's human feelings were unclear)
First line: 鱼跃此时海 (The fish leaps in this sea), Second line: 鸟鸣何日人 (The bird sings — on what day, a person)
First line: 只有香如故 (Only the fragrance remains as before), Second line: 不无月若新 (Not without a moon that seems new)
First line: 科学空间 (Scientific Spaces), Second line: 文明大中 (Civilization, grand and central)

It still has a certain flavor to it, I'd say. Note that "晚风摇树树还挺" is actually a first line from the training set, whose standard paired second line is "晨露润花花更红" (Morning dew moistens the flowers, and the flowers grow redder), whereas the model came up with "夜雨敲花花更香" (Night rain taps the flowers, and the flowers grow more fragrant) — which shows that the model isn't simply memorizing the training set, but does have some degree of understanding. In fact, I feel the model's second line is even more vivid.

Overall, the model seems capable of getting the basic character-to-character correspondences right, but it lacks a sense of the whole. The overall effect isn't as good as the following two systems, but as a little toy project, it should be satisfying enough.

Wang Bin's AI Couplet: https://ai.binwang.me/couplet/
Microsoft Couplet: https://duilian.msra.cn/default.htm

Conclusion

Finally, there isn't really much to sum up. I just felt that composing couplets ought to count as a sequence labeling task, so I thought I'd try using a sequence labeling model to see how it goes, and it turned out to work reasonably well~ Of course, to do better, some adjustments to the model would be needed — one could also consider introducing attention, and so on — and during decoding, more prior knowledge would need to be incorporated to ensure the results conform to what we expect from a proper couplet. I'll leave these to readers who are interested in pursuing this further.