bert4keras in Hand, Baseline in Sight: Baidu LIC2020

bert4keras在手，baseline我有：百度LIC2020 苏剑林 · 2020-04-02 · Machine Learning model Keras attention

Baidu's "2020 Language and Intelligence Technology Competition" has kicked off. This year there are five tracks: machine reading comprehension, recommendation-oriented dialogue, semantic parsing, relation extraction, and event extraction. For each track, the organizers provided a PaddlePaddle-based baseline model. Here I'm sharing my own personal baselines for three of these tracks, built on top of bert4keras, from which you can see just how convenient and concise it is to build baseline models with bert4keras.

Address: https://github.com/bojone/lic2020_baselines

A Brief Analysis of the Approach

Let's take a quick look at the characteristics of the tasks in these three tracks and the corresponding baseline designs. more

Reading Comprehension

Sample:

{
    "context": "这位朋友你好,女性出现妊娠反应一般是从6-12周左右,也就是女性怀孕1个多月就会开始出现反应,第3个月的时候,妊辰反应基本结束。 而大部分女性怀孕初期都会出现恶心、呕吐的感觉,这些症状都是因人而异的,除非恶心、呕吐的非常厉害,才需要就医,否则这些都是刚怀孕的的正常症状。1-3个月的时候可以观察一下自己的皮肤,一般女性怀孕初期可能会产生皮肤色素沉淀或是腹壁产生妊娠纹,特别是在怀孕的后期更加明显。 还有很多女性怀孕初期会出现疲倦、嗜睡的情况。怀孕三个月的时候,膀胱会受到日益胀大的子宫的压迫,容量会变小,所以怀孕期间也会有尿频的现象出现。月经停止也是刚怀孕最容易出现的症状,只要是平时月>      经正常的女性,在性行为后超过正常经期两周,就有可能是怀孕了。 如果你想判断自己是否怀孕,可以看看自己有没有这些反应。当然这也只是多数人的怀孕表现,也有部分女性怀孕表现并不完全是这样,如果你无法确定自己是否怀孕,最好去医院检查一下。",
    "qas": [
        {
            "question": "怀孕多久会有反应",
            "id": "f2843cffb845aad0100062841222023e",
            "answers": [
                {
                    "text": "6-12周左右",
                    "answer_start": -1
                },
                {
                    "text": "6-12周",
                    "answer_start": -1
                },
                {
                    "text": "1个多月",
                    "answer_start": -1
                }
            ]
        }
    ]
}

There's not much to say about this baseline — after passing through BERT, we attach two fully-connected layers + softmax to predict the start and end of the answer respectively. Some training samples are annotated with multiple answers, but at prediction time only one answer needs to be predicted, so during training we randomly pick just one answer each time to train on.

Relation Extraction

Sample:

{
    "text": "谢霆锋扮演的花无缺2004年由大导演王晶执导的40集电视连续剧《绝代双骄》的改版《小鱼儿与花无缺》上映，小编不是针对导演和演员，这部电视连续剧确实可说是天雷滚滚的魔改版，得亏全部主演凭借颜值和演技把观众拴住了，故事情节改的一塌糊涂",
    "spo_list": [
        {
            "predicate": "导演",
            "object": {
                "@value": "王晶"
            },
            "subject": "小鱼儿与花无缺"
        },
        {
            "predicate": "主演",
            "object": {
                "@value": "谢霆锋"
            },
            "subject": "小鱼儿与花无缺"
        },
        {
            "predicate": "主角",
            "object": {
                "@value": "花无缺"
            },
            "subject": "绝代双骄"
        },
        {
            "predicate": "饰演",
            "object": {
                "inWork": "小鱼儿与花无缺",
                "@value": "花无缺"
            },
            "subject": "谢霆锋"
        }
    ]
}

Relation extraction is essentially last year's triple extraction task, just with a bit of an upgrade this time. The upgrade lies in accounting for the polysemy of a single predicate — for example, "饰演" (played the role of) could refer to which TV drama someone appeared in, or which character within that drama they played. If a single sentence contains multiple distinct objects being "played," all of them need to be extracted correctly to count. Although it's called an "upgrade," there's actually no fundamental change: we simply need to concatenate the predicate with the corresponding object-type prefix to form distinct predicates, and the task reduces back to the standard triple extraction problem. For instance, "饰演_@value" and "饰演_inWork" are treated as two separate predicates to be extracted independently. My baseline model is still built on last year's "half-pointer, half-tagging" design — see A Lightweight Information Extraction Model Based on DGCNN and Probabilistic Graphs for details.

Event Extraction

Sample:

{
    "text": "雀巢裁员4000人：时代抛弃你时，连招呼都不会打！",
    "id": "409389c96efe78d6af1c86e0450fd2d7",
    "event_list": [
        {
            "event_type": "组织关系-裁员",
            "trigger": "裁员",
            "trigger_start_index": 2,
            "arguments": [
                {
                    "argument_start_index": 0,
                    "role": "裁员方",
                    "argument": "雀巢",
                    "alias": [
                        
                    ]
                },
                {
                    "argument_start_index": 4,
                    "role": "裁员人数",
                    "argument": "4000人",
                    "alias": [
                        
                    ]
                }
            ],
            "class": "组织关系"
        }
    ]
}

Event extraction is a relatively new task, requiring extraction of the event type along with certain elements describing that event. A single sentence may contain multiple events, and a single entity may simultaneously describe multiple events (for example, "on such-and-such date" might be the time of occurrence for several events at once). Event extraction is inherently a fairly complex task, but for this competition the organizers only evaluate triples of the form (event_type, role, argument) — that is, each such triple that matches correctly earns 1 point. Since event_type and role are both discrete categories, and argument is simply an entity from the original text, this evaluation metric effectively reduces the task to a plain entity tagging problem. It can therefore be solved with a conventional sequence labeling model — both my baseline and the official baseline formulate it this way, as a sequence labeling task.

Aligning Back to the Original Sequence

All three of the competitions above are, at their core, extraction problems — meaning the output entities are all spans of the original text. However, after the original text passes through BERT's tokenizer, it doesn't necessarily align perfectly with the original text anymore; there's the possibility of small-scale "insertions," "deletions," or "substitutions" (e.g., lowercasing, changes in the number of spaces, certain characters being transliterated). Such minor changes are usually inconsequential for engineering purposes, but they matter a great deal for competitions or academic evaluation like this, because even characters that look identical but aren't actually the same will cause a mismatch. As an example, readers can copy the following code into Python and run it:

u'à' == u'à'

To map the tokenized result back onto the original sequence, I spent some time adding a rematch method to bert4keras's Tokenizer: given the original text and the tokenized result, it returns the mapping from tokens back to the original text. With this mapping in hand, you can directly slice the original text. For the specifics, just take a look at the baseline code.

Summary

Wrote three baselines, and churned out another blog post along the way~