Learning with Noise: Improving Distantly-Supervised Fine-grained Entity Typing via Automatic Relabeling

1 Introduction

Fine-grained entity typing (FET) is a task which aims to find a proper fine-grained semantic type given an entity mention and its corresponding context text. Knowledge acquired through FET is informative and can benefit a wide range of natural language processing (NLP) applications, such as relation extraction [Liu et al., 2014], knowledge expansion [Dong et al., 2014], factoid question answering [Dong et al., 2015], and entity linking [Onoe and Durrett, 2019a].

Due to the lack of manually annotated fine-grained labels, distant supervision [Mintz et al., 2009] method is popularly adopted by recent FET systems. This method links the entity mention to an entity in the knowledge base and annotates all associated types as distantly-supervised labels. Although distant supervision is adequate to label data automatically, it suffers from the noisy labeling problem severely. As illustrated in Figure 1, the entity mention "Amazon" in two different sentences will be labeled with same entity type set, in which some types are inappropriate given the context. Obviously, direct distant supervision produces noisy training data, which will hurt the performance of the FET systems [Ren et al., 2016a].

∗ Indicates equal contribution.

† Corresponding author.

To address the issue of noisy labeling, most of previous studies try to model the samples with only one label (treated as "clean" samples) and samples with multiple labels (treated as "noisy" samples) separately to improve the prediction performance [Ren et al., 2016a; Abhishek et al., 2017; Xu and Barbosa, 2018; Chen et al., 2019]. In fact, we find that samples with only one distant label are not absolutely correct. [Wu et al., 2019] proposed to detect and weight out noise based on the assumption that the distantly-supervised label set must contain the correct type for each training sample, which is overly strong. [Onoe and Durrett, 2019b] proposed to refine the distantly-labeled data with a learned model trained on human-annotated data. Consequently, previous studies suffer from two limitations: 1) Still rely on some prerequisites (e.g. human-curated clean dataset), making them inefficient in real applications; 2) Incapable of eliminating the impact of samples with only one type label but are falsepositive (Table 4 shows more details).

In this paper, to handle the above two limitations simultaneously, we propose a probabilistic automatic relabeling method. As the ground-truth label distribution is not available, our method aims at estimating the pseudo-truth label distribution during the training process. In detail, each sample is assigned a continuous label distribution ˜ p over all candidate labels, and ˜ p will be jointly updated as trainable parameters through the back-propagation algorithm. The learning purpose is minimizing the Kullback-Leibler (KL) divergence between the predicted distribution and the pseudo-truth label distribution. Finally, we take the label with the highest value in ˜ p as the only one pseudo-truth label. In order to ensure the rationality of the final estimated pseudo-truth distribution, we integrate the golden-noisy information during the training process with two specific designed constraints. In this way, our method can effectively relabel the noisy distant samples during training, thus improving the predictive performance.

To show the effectiveness and robustness of our approach, we conduct experiments on three benchmarks. Experimental results show that our approach achieves state-of-the-art results, significantly outperforms previous methods. Furthermore, we design additional subsidiary experiments to demonstrate that the above-mentioned problems are alleviated on FET tasks trained with noisy data.

2 Our Approach

The overall architecture of our proposed model is illustrated in Figure 2. Concretely, our model consists of a feature encoder (which has the same structure and objective function with NFETC model proposed by [Xu and Barbosa, 2018]) and a Probabilistic Automatic Relabeling module, along with a three-phase training strategy.

2.1 ProblemDefinition

Given a training corpus D labeled with entity type hierarchy of knowledge base by distant supervision, we define G = {t1, t2, · · · , t|T|} as all candidate type labels, where |T| is the total number of types. Each type label ti is a path from root node to the terminal node (e.g. artist represents /person/artist), and a terminal node could be either a leaf node or a non-leaf node. In this paper, we use the terminal types as the predicted targets following settings in previous studies [Xu and Barbosa, 2018; Chen et al., 2019]. In most part of this paper, we use type to refer to terminal type for simplicity.

Training corpus D constructed by distant supervision consists of triplets with form (mi, ci, yi), i = 1, 2, · · · , N, where yi ∈ R|T| is the label vector (also denoted as noisy label as it may contain types which are not appropriate). For each training sample, we denote the context sentence as a word sequence ci = {w1,w2, · · · ,wn}, and entity mention mi = {wm1 ,wm2 , · · · ,wml } as a continuous sub-sequence from the context sentence. Given the mention-context input pair, the FET task aims at predicting the most appropriate type y∗ i from the pre-defined candidate set G.

2.2 Feature Encoder

For fair comparison, we adopt the feature encoder used in [Xu and Barbosa, 2018]. For each sample triple (mi, ci, yi) ∈ D, each word wj in ci is first mapped into word embedding ewj ∈ Rdw with a word embedding matrix W ∈ Rdw×|V | , where |V | is the vocabulary size and dw is the embedding size. Analogously, word position embedding [Zeng et al., 2014] epj ∈ Rdp is incorporated to reflect relative distances between each word and the target mention. Thus, the final embedding of the j-th word can be represented as concatenation of the two parts ej = [ewj , epj ].

Context Representation. The Bidirectional LSTM (BiLSTM) [Hochreiter and Schmidhuber, 1997; Graves and Schmidhuber, 2005] is then applied to model contextualized representations of context ci. The sequence of embedding {e1, e2, ..., ej , ..., en} will be fed into the Bi-LSTM network. By combining the last hidden state of forward and backward pass, we get the contextualized representation of word wj as hj = [-→hj ⊕ ←- hj], where ⊕ represents the element-wise sum operation. The hidden state size of Bi-LSTM notes as ds. Following [Zhou et al., 2016], a word-level attention module is adopted to attend to the most influenced words, and generate the final context representation rci .

Mention Representation. The mention encoder contains two parts. Average encoder: given {em1 , ...emk , ..., eml } be the sequence word embeddings of mi, the average encoder simply averages value of each word embedding vector: ra = 1l ∑lk=1 emk . LSTM encoder: by applying an LSTM over the extended mention embeddings, the context-aware representation {hm1-1, ...hmk , ..., hml+1} is achieved. The last hidden state rl = hml+1 ∈ Rds is concatenated with ra to from the final mention representation rmi = [ra, rl].

2.3 Cross-Entropy Objective Function

As illustrated by left dash box of Figure 2, the feature vector is represented using ri = [rmi , rci ] and fed into a MLP layer. Then we use a softmax classifier to predict ¯ yi over the candidate set G, where W ∈ Rdr×|T| and b ∈ R|T| are trainable parameters. The cross-entropy loss function is denoted as Eq. 2, where θ denotes for trainable parameters of entire model.

Clearly, optimizing the FET model with standard crossentropy loss function has limitations for learning with noisy labels. The one-hot labels y for training samples with multiple entity types are not reasonable distribution, which will lead the model over-fitting to noisy labels and thus degrade the model performance. In the next section, we propose a probabilistic automatic relabeling method which helps to estimate the pseudo-truth label of each training sample and then address the noisy labeling problem.

2.4 Automatic RelabelingModule

In this section, we propose the probabilistic Automatic Relabeling (AR) module, an approach that strikes a balance between sufficient learning and robustness to noisy labels.

Our method aims to investigate the underlying truth label for each sample. The basic assumption of our idea is: for each mention-context pair (m, c), there exists only one most appropriate type given the candidate set G. Hence, the de-noising process over training set D implies that we seek

̂

to find the pseudo-truth label distribution ˜ p over G of each sample. From the probabilistic perspective, how can we estimate this distribution? We realize it by treating ˜ p as part of trainable parameters and ˜ p will be jointly updated during the learning process. Therefore, when training with crossentropy loss without automatic relabeling, the optimization problem of FET task can be defined as:

In our automatic relabeling module, the optimization objective is formulated as:

As shown in Figure 2, the proposed automatic relabeling module contains a pseudo-truth label distribution estimation term accompanied by two specifically designed constraints.

Pseudo-truth Label Distribution Estimation

The ground truth label distribution is not available, which must be estimated under proper assumptions. Although the training set is constructed through distant supervision method, the label noises are correlated and context-aware. This basic observation makes it possible to estimate the pseudo-truth label distribution ˜ p using a self-supervised learning approach, which has also been verified by previous work [Wu et al., 2019]. Concretely, we target on maximizing the information-theoretic dependency between the context and the pseudo-truth label associated. Although the original predicted distribution p is not completely correct, it is still a high-quality prior knowledge to guide the distribution estimation process of ˜ p. Inspired by [Hu et al., 2017; Wang and Wu, 2019], we could directly estimate the pseudotruth label distribution ˜ p via minimizing the KL-divergence between ˜ p and p(m, c; θ).

Formally, we assign a continuous label distribution ˜ pi to each training sample, where ˜ pi = {˜ pij: ˜ pij ∈ [0, 1], ∑j ˜ pij = 1}. The KL-divergence loss function can be represented as:

where θ denotes for all trainable parameters in the basic model. Additionally, ˜ p ∈ RN×|T| will be updated as other trainable parameters.

However, using the KL-divergence loss alone is not enough to get a well-estimated pseudo-truth label distribution. As we take the prediction p as objective, the Lkl can't deal with the wrong predictions produced by the basic typing model. These errors will continue to accumulate during the training process and then cause confirmation bias. Moreover, we empirically observe that the final estimated distribution ˜ p is very smooth when employing only Lkl, which is unhelpful to optimize it towards the ground-truth one-hot label distribution. Thus, we add two additional constraints to guide the pseudo-truth label distribution estimation process.

Distant label Constraint

As illustrated in Table 2, the majority of training data (64.46% of Wiki, 73.13% of OntoNotes and 75.92% of BBN) constructed by distant supervision contains only one type label, which is treated as clean sample in previous studies [Xu and Barbosa, 2018; Chen et al., 2019]. The distantly-supervised labels still contain valuable information about the ground truth label distribution, although not all training samples with only one label are absolutely correct. By using this information, we can constrain the distribution estimation process and reduce the learning difficulty.

Considering all above, we make use of the distant noisy labels y with three operations: 1) At the beginning of the automatic relabeling process, we initialize the trainable distribution ˜ p with the normalized version of original noisy labels y, i.e., ˜ p = softmax(y). 2) We also maintain the cross-entropy loss Lce of the basic model as a part of the final optimization objective to ensure the rationality of the predicted distribution p(m, c; θ). 3) To avoid the estimated pseudo-truth label distribution ˜ p from being totally different from the original noisy label distribution y, we add a cross-entropy loss between ˜ p and y, which can be denoted as:

Distribution Sharpen Constraint

As mentioned above, the FET task assumes that each sample has only one most appropriate true type label [Wu et al., 2019]; thus we want to keep the distribution ˜ p obeying the same property. We try to control the pseudo distribution by encouraging the predicted label distribution to be sharpen for each sample. Under this purpose, another regularization term is introduced: the distribution sharpen constraint, which pushes the predicted probability of each type to nearly 0 or 1, as shown in Eq. 7.

2.5 Training Strategy

To this end, we can train the AR module in an end-to-end manner with the standard back-propagation algorithm. The overall loss function can be formulated as:

where β, γ, ω and δ are hyper-parameters.

We divide the entire optimization process into three phases: 1) Preliminary training. Since the proposed approach does not rely on any extra supervision, at the beginning of the entire training process, we first train a basic entity typing model with parameters θ only using the original noisy labels (can be treated as warm-up training). 2) Probabilistic Automatic Relabeling. After a preliminary network is trained, we can step into the second phase and jointly estimate the pseudo label distribution ˜ p. 3) Fine tuning. Finally, we normalize the estimated distribution ˜ p to one-hot labels ˆ y by choosing one label with maximum value for each sample, and then the pseudo labels ˆ y are used for continuous fine-tuning the basic typing model. More details about the three-phase optimization process are described in Algorithm 1. The training epoch number for each phase e1, e2, e3 are hyper-parameters and chosen by grid search.

3 Experiments

We thoroughly evaluate the performance of our method on three benchmarks using several comparable approaches as baselines. Furthermore, we conduct several auxiliary experiments to analyze the effectiveness of our proposed method.

3.1 Settings

Datasets. We conduct the experiments on three standard and publicly available datasets. Wiki [Ling and Weld, 2012],

Algorithm 1: Training Procedure

Input:

- Xb ← Batch(D)

- Yb ← Batch(D)

Parameters:

- Model parameters: θ, Pseudo distribution: ˜ p

Randomly initialize model parameters θ

for epoch ←1 to e1 do

Initialize ˜ p ← sof tmax(y)

for epoch ←1 to e2 do

ˆ y ← argmax(˜ p)

ˆ Yb ← Batch(ˆ y)

for epoch ←1 to e3 do

Update θ w.r.t. Lce (Xb, ˆ Yb; θ)

OntoNotes [Weischedel et al., 2013] and BBN [Weischedel and Brunstein, 2005]. We follow the same hierarchy refinement pre-processing of datasets used in [Abhishek et al., 2017; Xu and Barbosa, 2018]. The detailed statistics of the datasets are shown in Table 2.

Implementation Details. Table 3 shows the hyperparameters used in our method. The model is trained using mini-batched back-propagation, and Adam optimizer [Kingma and Ba, 2014] is used for optimization.

Evaluation Metrics. We adopt three commonly used evaluation metrics: Strict Accuracy (Strict Acc), Micro-averaged F1 (Micro-F1) score and Macro-averaged F1 (MacroF1) [Ling and Weld, 2012]. On all datasets, we use the same training/development/test sets settings with previous studies, and the development sets are used to select the model with the best performance among all epochs.

Baselines. We compare our method with several state-ofthe-art FET systems, including AFET [Ren et al., 2016a], AAA [Abhishek et al., 2017], Attentive [Shimaoka et al., 2016], NDP [Wu et al., 2019], NFETC [Xu and Barbosa, 2018] and NFETC-CLSC [Chen et al., 2019]. [Xu and Barbosa, 2018] propose to model the type hierarchy with the hierarchical loss, which has been proven effective for FET task. Thus, for all NFETC based methods, we report results produced by NFETC and NFETChier respectively.

3.2 Overall Results

Table 1 presents the final results of our method with three metrics. The scores of each metric are calculated by running the model for five times and computing the mean and standard deviation values. We highlight the best performances of each metric in bold. We find that our approach achieves the best performance among all the baselines.

Our model improves the performances by a large margin compared to the basic model (NFETC) on all three datasets. Specifically, the auto-relabeling enhanced model improves the strict accuracy on the three datasets from 68.9 to 70.1,

60.2 to 64.0 and 73.9 to 76.7, respectively. This observation illustrates the necessity of noise reduction in the distantlysupervised FET task and the effectiveness of our proposed auto-relabeling method.

Comparing our NFETC-AR method with previous methods which process "clean" and "noise" data separately(e.g. NFETC-CLSC), NFETC-AR performs much better. The reason might be that the separate processing method can reduce the impact of noisy samples to some degree, but it also neglects the valuable information contained in noisy samples. These results further verify the intuition for relabeling the noisy samples and converting them into helpful training data. Also, there is an obvious margin between NFETC-AR and NDP. This result demonstrates that our model is more effective in finding correct type from the distantly-supervised type set, and the ability to handle false-positive samples is quite effective in improving the predictive performance.

3.3 Discussion and Analysis

Learning Curve of NFETC-AR. To better analyze the advantages of our model in tackling the noisy labeling problem, we compute the strict accuracy of our model on development set and test set after each training epoch, as shown in Figure 3. Note that the horizontal dotted lines in Figure 3 represent test set performances of NFETC and NFETC-CLSC respectively. We can see that: 1) In phase 1, our model is actually the basic NFETC; thus, the performance of our model cannot exceed the NFETC dotted lines by a meaningful margin. 2) In phase 2, as the loss function of the automatic relabeling module is adopted, the NFETC-AR model is able to reduce the negative effect of noisy data. As a result, the strict accuracy on the test set is further improved and then exceeds the NFETC model significantly. 3) In the third phase, the estimated pseudo-label distribution (soft-labels) in phase 2 is transformed into onehot labels (hard-labels). Then we use these hard-labels (after relabeling) to continue fine-tuning the basic model. As a result, the performance of our model keeps improving and then surpasses the previous state-of-the-art model NFETC-CLSC.

Ablation Study of AR. We further evaluate the influence of each component in the automatic relabeling (AR) module by conducting an ablation study. From Table 5, we observe that each component statistically improves the model performance. Specifically, the KL-divergence loss is the key component of the AR process. Without Lkl, the model produces similar performance with the basic model (shown in w/o AR

line) on Marco-F1 and Micro-F1. Moreover, among the three operations in the distant label constraint, we find that noisy label initialization has the greatest impact (switching to random initialization will degrade the accuracy to 55.0). This result again proves that the original noisy information is valuable to some extent, and fully making use of this information is very important to the automatic relabeling process.

Statistical Analysis of AR. In order to examine the relabeling results of our proposed model, we statistically analyze the differences between the noisy labels y and the pseudo labels ˆ y after automatic relabeling. We find that there are three types of modification. Multi-to-one-in: samples with multiple distantly-supervised labels are modified into one of them; In contrast, Multi-to-one-out represents samples with multiple labels are finally modified into one label out of the original ones. Besides, we find that a part of the samples with only one label has also been modified, and it is noted as One-toone-out. Taking OntoNotes dataset as an example, 27.52% of the training samples are modified. Among the modified data, the percentages of the three different types are 97.49%, 0.12% and 2.39% respectively. Similar results are obtained on both Wiki and BBN datasets. After careful inspection of the corrected labels ˆ y, we find: 1) not all samples with only one type label are correctly labeled; 2) the assumption that the ground truth label must exist in the distantly-supervised labels is also biased. In Table 4, we select some samples of each correction type for better understanding. The result of statistical analysis and case studies illustrates the ability of our model in automatic relabeling. Moreover, our proposed method does not rely on any pre-set assumptions that are actually problematic.

Robustness to Noisy Data. To study the robustness of our model in handling noisy data, we compare our method with previous start-of-the-art systems after removing "clean" samples on the training set. For each dataset, we split the training set into "clean" and "noisy" samples, and "clean" samples are removed by different proportions (75% - 95%). Here, we treat samples with one type label as clean data following [Chen et al., 2019], as most of them contain correct information. We compute the strict accuracy given remaining clean samples and all noisy samples to evaluate the model's robustness to more noisy data. According to Figure 4, our model surpasses all baselines consistently on the OntoNotes dataset. On the BBN dataset, our model achieves comparable performances with NFETC-CLSC in most cases except when removing 95% clean data. The reason might be that NFETCCLSC applies an unsupervised cluster-based noise reduction method and thus is more robust with extremely limited clean data. Overall, the experiment proves that our proposed approach makes the basic model much more robust.

4 RelatedWork

Fine-grained entity typing (FET) is a common task in NLP. In general, a large amount of labeled data is required to train a FET model, which is quite expensive. To address this issue, [Ling and Weld, 2012] first proposed building the training data via distant supervision [Mintz et al., 2009]. Embedding-based models were applied to FET since then [Yogatama et al., 2015; Dong et al., 2015]. Recently, deep neural methods have been widely applied to FET. [Shimaoka et al., 2016] proposed to use LSTMs to encode the context information and then use attention mechanism to select informative information. [Ma et al., 2016; Lin and Ji, 2019] exploited to improve the model performance by leveraging the entity type information. [Xin et al., 2018] proposed a knowledge attention model by jointly considering the information from knowledge bases (KBs).

Despite the success of distance supervision, it still suffers from the noisy labeling problem. To alleviate this issue, [Gillick et al., 2014] refined the training data by applying a set of heuristics to prune types, [Ren et al., 2016a; Ren et al., 2016b] proposed the AFET model which separates the loss function for clean and noisy entity separately, and incorporates the imperfect annotation by partial-label loss. Additionally, [Abhishek et al., 2017; Xu and Barbosa, 2018] proposed two variants of partial-label loss. Nevertheless, these methods still struggle with confirmation bias. Based on the assumption that the distantly-supervised label set must contain the ground-truth label, [Wu et al., 2019] proposed to model the structured, noisy labels directly and then weight out noisy labels during training with the help of random walking process. [Chen et al., 2019] leveraged the noisy samples as regularization via compact latent space clustering method. [Onoe and Durrett, 2019b] tried to filter and relabel the noisy samples with the help of additional human-labeled data. Unlike these methods, we propose a unified framework that treats the clean and noisy samples equally and solves the noise labeling problem without extra supervision.

5 Conclusion and FutureWork

In this paper, we propose a probabilistic automatic relabeling method for fine-grained entity typing. The proposed approach is able to infer the pseudo label of each sample and use the relabelled samples to fine-tune the backbone network. The proposed approach does not rely on any prerequisite or extra supervision, making it robust and effective on real applications. Extensive experimental results show that the proposed method performs better than previous approaches, and can certainly alleviate the noisy labeling problem in finegrained entity typing task. Our proposed method is independent of the backbone network; thus, in future work we plan to investigate the performance of our method under different backbone networks. In addition, the automatic relabeling approach can also be applied to other NLP tasks established on distantly-supervised datasets, such as relation extraction, lexicon-based named entity recognition. More research is underway to explore these two directions.

Acknowledgments

We thank all the anonymous reviewers for their valuable suggestions. This research is partially supported by the National key R&D program of China (No. 2017YFB1001802) and the National Nature Science Foundation of China (No. 61532007).