{"type":"ScholarlyArticle","title":"Learning with Noise: Improving Distantly-Supervised Fine-grained Entity Typing via Automatic Relabeling","authors":[{"name":"Haoyu Zhang","slug":"haoyu-zhang","given":"Haoyu","family":"Zhang","affiliation":"State Key Laboratory of High Performance Computing , National University of Defense Technology; Alibaba Group"},{"name":"Dingkun Long","slug":"dingkun-long","given":"Dingkun","family":"Long","affiliation":"Alibaba Group"},{"name":"Guangwei Xu","slug":"guangwei-xu","given":"Guangwei","family":"Xu","affiliation":"Alibaba Group"},{"name":"Muhua Zhu","slug":"muhua-zhu","given":"Muhua","family":"Zhu","affiliation":"Alibaba Group"},{"name":"Pengjun Xie","slug":"pengjun-xie","given":"Pengjun","family":"Xie","affiliation":"Alibaba Group"},{"name":"Fei Huang","slug":"fei-huang","given":"Fei","family":"Huang","affiliation":"Alibaba Group"},{"name":"Ji Wang","slug":"ji-wang","given":"Ji","family":"Wang","affiliation":"State Key Laboratory of High Performance Computing , National University of Defense Technology"}],"abstract":"Fine-grained entity typing (FET) is a fundamental task for various entity-leveraging applications. Although great success has been made, existing systems still have challenges in handling noisy samples in training data introduced by distant supervision methods. To address these noise, previous studies either focus on processing the clean samples (i,e., have only one label) and noisy samples (i,e., have multiple labels) with different strategies or filtering the noisy labels based on the assumption that the distantly-supervised label set certainly contains the correct type label. In this paper, we propose a probabilistic automatic relabeling method which treats all training samples uniformly. Our method aims to estimate the pseudo-truth label distribution of each sample, and the pseudo-truth distribution will be treated as part of trainable parameters which are jointly updated during the training process. The proposed approach does not rely on any prerequisite or extra supervision, making it effective on real applications. Experiments on several benchmarks show that our method outperforms previous approaches and alleviates the noisy labeling problem.","doi":"10.24963/ijcai.2020/527","url":"https://ijcai.org/proceedings/2020/527","pdf":"https://ijcai.org/proceedings/2020/0527.pdf","markdown":"https://ijcai.org/proceedings/2020/527.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/527","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":527,"pages":{"first":3808,"last":3815},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Natural Language Processing","Natural Language Processing: Named Entities","Natural Language Processing: Information Extraction"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25089/file_25089.m3u8","length_sec":300,"provider":"hls"}],"references":[{"position":1,"raw":"[Abhishek et al., 2017] Abhishek, Ashish Anand, and Amit Awekar. Fine-grained entity type classification by jointly learning representations and label embeddings. In EACL, pages 797-807, 2017.","title":"Fine-grained entity type classification by jointly learning representations and label embeddings","authors":"Ashish Abhishek, Amit Anand, Awekar","year":"2017","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019] Bo Chen, Xiaotao Gu, Yufeng Hu, Sil- iang Tang, Guoping Hu, Yueting Zhuang, and Xiang Ren. Improving distantly-supervised entity typing with compact latent space clustering. In NAACL-HLT, pages 2862-2872, 2019.","title":"Improving distantly-supervised entity typing with compact latent space clustering","authors":"Bo Chen, Xiaotao Gu, Yufeng Hu, Siliang Tang, Guoping Hu, Yueting Zhuang, Xiang Ren","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dong et al., 2014] Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In KDD, pages 601-610, 2014.","title":"Knowledge vault: a web-scale approach to probabilistic knowledge fusion","authors":"Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, Wei Zhang","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dong et al., 2015] Li Dong, Furu Wei, Hong Sun, Ming Zhou, and Ke Xu. A hybrid neural model for type clas- sification of entity mentions. In IJCAI, pages 1243-1249, 2015.","title":"A hybrid neural model for type classification of entity mentions","authors":"Li Dong, Furu Wei, Hong Sun, Ming Zhou, Ke Xu","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/179"},{"position":5,"raw":"[Gillick et al., 2014] Dan Gillick, Nevena Lazic, Kuzman Ganchev, Jesse Kirchner, and David Huynh. Context- dependent fine-grained entity type tagging. arXiv preprint arXiv:1412.1820, 2014.","title":"Contextdependent fine-grained entity type tagging","authors":"Dan Gillick, Nevena Lazic, Kuzman Ganchev, Jesse Kirchner, David Huynh","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Graves and Schmidhuber, 2005] Alex Graves and Jürgen Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5-6):602-610, 2005.","title":"Framewise phoneme classification with bidirectional lstm and other neural network architectures","authors":"Schmidhuber Graves, Alex Graves, Jürgen Schmidhuber","year":"2005","venue":"Neural networks","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2017] Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, and Masashi Sugiyama. Learning discrete representations via information maximizing self- augmented training. In ICML, pages 1558-1567, 2017.","title":"Learning discrete representations via information maximizing selfaugmented training","authors":"Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, Masashi Sugiyama","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin and Ji, 2019] Ying Lin and Heng Ji. An attentive fine- grained entity typing model with latent type representa- tion. In EMNLP-IJCNLP, pages 6196-6201, 2019.","title":"An attentive finegrained entity typing model with latent type representation","authors":"Ying Lin, Heng Ji","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ling and Weld, 2012] Xiao Ling and Daniel S. Weld. Fine- grained entity recognition. In AAAI, 2012.","title":"Finegrained entity recognition","authors":"Xiao Ling, Daniel S Weld","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2014] Yang Liu, Kang Liu, Liheng Xu, and Jun Zhao. Exploring fine-grained entity type constraints for distantly supervised relation extraction. In COLING, pages 2107-2116, 2014.","title":"Exploring fine-grained entity type constraints for distantly supervised relation extraction","authors":"Yang Liu, Kang Liu, Liheng Xu, Jun Zhao","year":"2014","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ma et al., 2016] Yukun Ma, Erik Cambria, and Sa Gao. La- bel embedding for zero-shot fine-grained named entity typing. In COLING, pages 171-180, 2016.","title":"Label embedding for zero-shot fine-grained named entity typing","authors":"Yukun Ma, Erik Cambria, Sa Gao","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mintz et al., 2009] Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. Distant supervision for relation ex- traction without labeled data. In ACL-IJCNLP, pages 1003-1011, 2009.","title":"Distant supervision for relation extraction without labeled data","authors":"Mike Mintz, Steven Bills, Rion Snow, Daniel Jurafsky","year":"2009","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Onoe and Durrett, 2019a] Yasumasa Onoe and Greg Dur- rett. Fine-grained entity typing for domain independent entity linking. arXiv preprint arXiv:1909.05780, 2019.","title":"Fine-grained entity typing for domain independent entity linking","authors":"Durrett Onoe","year":"2019","venue":"Yasumasa Onoe and Greg Durrett","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Onoe and Durrett, 2019b] Yasumasa Onoe and Greg Dur- rett. Learning to denoise distantly-labeled data for entity typing. In NAACL-HLT, pages 2407-2417, 2019.","title":"Learning to denoise distantly-labeled data for entity typing","authors":"Durrett Onoe","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ren et al., 2016a] Xiang Ren, Wenqi He, Meng Qu, Lifu Huang, Heng Ji, and Jiawei Han. AFET: automatic fine- grained entity typing by hierarchical partial-label embed- ding. In EMNLP, pages 1369-1378, 2016.","title":"AFET: automatic finegrained entity typing by hierarchical partial-label embedding","authors":"Ren","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ren et al., 2016b] Xiang Ren, Wenqi He, Meng Qu, Clare R. Voss, Heng Ji, and Jiawei Han. Label noise re- duction in entity typing by heterogeneous partial-label em- bedding. In KDD, pages 1825-1834, 2016.","title":"Label noise reduction in entity typing by heterogeneous partial-label embedding","authors":"Ren","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shimaoka et al., 2016] Sonse Shimaoka, Pontus Stenetorp, Kentaro Inui, and Sebastian Riedel. An attentive neural architecture for fine-grained entity type classification. In AKBC@NAACL-HLT, pages 69-74, 2016.","title":"An attentive neural architecture for fine-grained entity type classification","authors":"Shimaoka","year":"2016","venue":"AKBC@NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang and Wu, 2019] Guo-Hua Wang and Jianxin Wu. Repetitive reprediction deep decipher for semi-supervised learning. arXiv preprint arXiv:1908.04345, 2019.","title":"Repetitive reprediction deep decipher for semi-supervised learning","authors":"Guo-Hua Wang, Jianxin Wu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Weischedel and Brunstein, 2005] Ralph Weischedel and Ada Brunstein. Bbn pronoun coreference and entity type corpus. Linguistic Data Consortium, Philadelphia, 112, 2005.","title":"Bbn pronoun coreference and entity type corpus","authors":"Brunstein Weischedel, Ralph Weischedel, Ada Brunstein","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Weischedel et al., 2013] Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. Ontonotes release 5.0 ldc2013t19. Linguistic Data Consortium, Philadelphia, PA, 23, 2013.","title":"Ontonotes release 5.0 ldc2013t19. Linguistic Data Consortium","authors":"Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2019] Junshuang Wu, Richong Zhang, Yongyi Mao, Hongyu Guo, and Jinpeng Huai. Modeling noisy hierarchical types in fine-grained entity typing: A content- based weighting approach. In IJCAI, pages 5264-5270, 2019.","title":"Modeling noisy hierarchical types in fine-grained entity typing: A contentbased weighting approach","authors":"Wu","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xin et al., 2018] Ji Xin, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Improving neural fine-grained entity typ- ing with knowledge attention. In AAAI, 2018.","title":"Improving neural fine-grained entity typing with knowledge attention","authors":"Ji Xin, Yankai Lin, Zhiyuan Liu, Maosong Sun","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu and Barbosa, 2018] Peng Xu and Denilson Barbosa. Neural fine-grained entity type classification with hierarchy-aware loss. In NAACL-HLT, pages 16-25, 2018.","title":"Neural fine-grained entity type classification with hierarchy-aware loss","authors":"Peng Xu, Denilson Barbosa","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yogatama et al., 2015] Dani Yogatama, Daniel Gillick, and Nevena Lazic. Embedding methods for fine grained entity type classification. In ACL, pages 291-296, 2015.","title":"Embedding methods for fine grained entity type classification","authors":"Dani Yogatama, Daniel Gillick, Nevena Lazic","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zeng et al., 2014] Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. Relation classification via convolutional deep neural network. In COLING, pages 2335-2344, 2014.","title":"Relation classification via convolutional deep neural network","authors":"Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao","year":"2014","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2016] Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. Attention- based bidirectional long short-term memory networks for relation classification. In ACL, 2016.","title":"Attentionbased bidirectional long short-term memory networks for relation classification","authors":"Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, Bo Xu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}