{"type":"ScholarlyArticle","title":"None Class Ranking Loss for Document-Level Relation Extraction","authors":[{"name":"Yang Zhou","slug":"yang-zhou","given":"Yang","family":"Zhou","affiliation":"School of Computing , National University of Singapore"},{"name":"Wee Sun Lee","slug":"wee-sun-lee","given":"Wee Sun","family":"Lee","affiliation":"School of Computing , National University of Singapore"}],"abstract":"Document-level relation extraction (RE) aims at extracting relations among entities expressed across multiple sentences, which can be viewed as a multi-label classification problem. In a typical document, most entity pairs do not express any pre-defined relation and are labeled as \"none\" or \"no relation\". For good document-level RE performance, it is crucial to distinguish such none class instances (entity pairs) from those of pre-defined classes (relations). However, most existing methods only estimate the probability of pre-defined relations independently without considering the probability of \"no relation\". This ignores the context of entity pairs and the label correlations between the none class and pre-defined classes, leading to sub-optimal predictions. To address this problem, we propose a new multi-label loss that encourages large margins of label confidence scores between each pre-defined class and the none class, which enables captured label correlations and context-dependent thresholding for label prediction. To gain further robustness against positive-negative imbalance and mislabeled data that could appear in real-world RE datasets, we propose a margin regularization and a margin shifting technique. Experimental results demonstrate that our method significantly outperforms existing multi-label losses for document-level RE and works well in other multi-label tasks such as emotion classification when none class instances are available for training.","doi":"10.24963/ijcai.2022/630","url":"https://ijcai.org/proceedings/2022/630","pdf":"https://ijcai.org/proceedings/2022/0630.pdf","markdown":"https://ijcai.org/proceedings/2022/630.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/630","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":630,"pages":{"first":4538,"last":4544},"keywords":["Natural Language Processing: Information Extraction","Machine Learning: Multi-label"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986647","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984790","length_sec":343,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ben-Baruch et al., 2021] Emanuel Ben-Baruch, Tal Ridnik, Na- dav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. Asymmetric loss for multi-label classification. In Proceedings of ICCV, 2021.","title":"Asymmetric loss for multi-label classification","authors":"Emanuel Ben-Baruch, Tal Ridnik, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, Lihi Zelnik-Manor","year":"2021","venue":"Proceedings of ICCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Christopoulou et al., 2019] Fenia Christopoulou, Makoto Miwa, and Sophia Ananiadou. Connecting the dots: Document-level neural relation extraction with edge-oriented graphs. In Proceed- ings of EMNLP-IJCNLP, pages 4927-4938, 2019.","title":"Connecting the dots: Document-level neural relation extraction with edge-oriented graphs","authors":"Fenia Christopoulou, Makoto Miwa, Sophia Ananiadou","year":"2019","venue":"Proceedings of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dai et al., 2021] Damai Dai, Jing Ren, Shuang Zeng, Baobao Chang, and Zhifang Sui. Coarse-to-fine entity representations for document-level relation extraction. In Proceedings of AAAI, 2021.","title":"Coarse-to-fine entity representations for document-level relation extraction","authors":"Damai Dai, Jing Ren, Shuang Zeng, Baobao Chang, Zhifang Sui","year":"2021","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Demszky et al., 2020] Dorottya Demszky, Dana Movshovitz- Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. GoEmotions: A dataset of fine-grained emotions. In Pro- ceedings of ACL, pages 4040-4054, 2020.","title":"GoEmotions: A dataset of fine-grained emotions","authors":"Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, Sujith Ravi","year":"2020","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Durand et al., 2019] Thibaut Durand, Nazanin Mehrasa, and Greg Mori. Learning a deep convnet for multi-label classification with partial labels. In Proceedings of CVPR, pages 647-657, 2019.","title":"Learning a deep convnet for multi-label classification with partial labels","authors":"Thibaut Durand, Nazanin Mehrasa, Greg Mori","year":"2019","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fürnkranz et al., 2008] Johannes Fürnkranz, Eyke Hüllermeier, Eneldo Loza Mencía, and Klaus Brinker. Multilabel classifica- tion via calibrated label ranking. Machine Learning, 73(2):133- 153, 2008.","title":"Multilabel classification via calibrated label ranking","authors":"Johannes Fürnkranz, Eyke Hüllermeier, Eneldo Loza Mencía, Klaus Brinker","year":"2008","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goyal et al., 2017] Priya Goyal, P. Dollár, Ross B. Girshick, P. No- ordhuis, L. Wesolowski, Aapo Kyrola, Andrew Tulloch, Y. Jia, and Kaiming He. Accurate, large minibatch SGD: Training ima- genet in 1 hour. ArXiv, abs/1706.02677, 2017.","title":"Accurate, large minibatch SGD: Training imagenet in 1 hour","authors":"Priya Goyal, P Dollár, Ross B Girshick, P Noordhuis, L Wesolowski, Aapo Kyrola, Andrew Tulloch, Y Jia, Kaiming He","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2021] Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. DeBERTa: Decoding-enhanced bert with disen- tangled attention. In Proceedings of ICLR, 2021.","title":"DeBERTa: Decoding-enhanced bert with disentangled attention","authors":"Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen","year":"2021","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2017] Yuncheng Li, Yale Song, and Jiebo Luo. Improv- ing pairwise ranking for multi-label image classification. In Pro- ceedings of CVPR, pages 3617-3625, 2017.","title":"Improving pairwise ranking for multi-label image classification","authors":"Yuncheng Li, Yale Song, Jiebo Luo","year":"2017","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaim- ing He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of ICCV, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"Proceedings of ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2019] Y. Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized bert pre- training approach. ArXiv, abs/1907.11692, 2019.","title":"RoBERTa: A robustly optimized bert pretraining approach","authors":"Y Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M Lewis, Luke Zettlemoyer, Veselin Stoyanov","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Loshchilov and Hutter, 2019] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Proceedings of ICLR, 2019.","title":"Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization","authors":"Hutter Loshchilov","year":"2019","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Miwa and Bansal, 2016] Makoto Miwa and Mohit Bansal. End-to- end relation extraction using lstms on sequences and tree struc- tures. In Proceedings of ACL, 2016.","title":"End-toend relation extraction using lstms on sequences and tree structures","authors":"Bansal Miwa, Makoto Miwa, Mohit Bansal","year":"2016","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nan et al., 2020] Guoshun Nan, Zhijiang Guo, Ivan Sekulic, and Wei Lu. Reasoning with latent structure refinement for document-level relation extraction. In Proceedings of ACL, pages 1546-1557, 2020.","title":"Reasoning with latent structure refinement for document-level relation extraction","authors":"Nan","year":"2020","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tang et al., 2020] Hengzhu Tang, Yanan Cao, Zhenyu Zhang, Jiangxia Cao, Fang Fang, Shigang Wang, and Pengfei Yin. Hin: Hierarchical inference network for document-level relation ex- traction. In Proceedings of PAKDD, 2020.","title":"Hin: Hierarchical inference network for document-level relation extraction","authors":"Tang","year":"2020","venue":"Proceedings of PAKDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2020] Difeng Wang, Wei Hu, Ermei Cao, and Weijian Sun. Global-to-local neural networks for document-level relation extraction. In Proceedings of EMNLP, pages 3711-3721, 2020.","title":"Global-to-local neural networks for document-level relation extraction","authors":"Wang","year":"2020","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wolf et al., 2020] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cis- tac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of- the-art natural language processing. In Proceedings of EMNLP, pages 38-45, 2020.","title":"Transformers: State-ofthe-art natural language processing","authors":"Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Clara Patrick Von Platen, Yacine Ma, Julien Jernite, Canwen Plu, Teven Xu, Sylvain Le Scao, Mariama Gugger, Quentin Drame, Alexander Lhoest, Rush","year":"2020","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wu et al., 2019] Jiawei Wu, Wenhan Xiong, and William Yang Wang. Learning to learn and predict: A meta-learning ap- proach for multi-label classification. In Proceedings of EMNLP- IJCNLP, pages 4354-4364, 2019.","title":"Learning to learn and predict: A meta-learning approach for multi-label classification","authors":"Jiawei Wu, Wenhan Xiong, William Yang, Wang","year":"2019","venue":"Proceedings of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wu et al., 2020] Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, and Dahua Lin. Distribution-balanced loss for multi-label classi- fication in long-tailed datasets. In Proceedings of ECCV, pages 162-178, 2020.","title":"Distribution-balanced loss for multi-label classification in long-tailed datasets","authors":"Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, Dahua Lin","year":"2020","venue":"Proceedings of ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2019] Miao Xu, Yu-Feng Li, and Zhi-Hua Zhou. Robust multi-label learning with pro loss. IEEE Transactions on Knowl- edge and Data Engineering, 32(8):1610-1624, 2019.","title":"Robust multi-label learning with pro loss","authors":"Miao Xu, Yu-Feng Li, Zhi-Hua Zhou","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2021a] Benfeng Xu, Quan Wang, Yajuan Lyu, Yong Zhu, and Zhendong Mao. Entity structure within and through- out: Modeling mention dependencies for document-level relation extraction. In Proceedings of AAAI, 2021.","title":"Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction","authors":"Xu","year":"2021","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2021b] Wang Xu, Kehai Chen, and Tiejun Zhao. Document-level relation extraction with reconstruction. In Pro- ceedings of AAAI, 2021.","title":"Document-level relation extraction with reconstruction","authors":"Xu","year":"2021","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xue et al., 2021] Fuzhao Xue, Aixin Sun, Hao Zhang, and Eng Siong Chng. GDPNet: Refining latent multi-view graph for relation extraction. In Proceedings of AAAI, 2021.","title":"GDPNet: Refining latent multi-view graph for relation extraction","authors":"Xue","year":"2021","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yao et al., 2019] Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, and Maosong Sun. DocRED: A large-scale document-level relation extraction dataset. In Proceedings of ACL, 2019.","title":"DocRED: A large-scale document-level relation extraction dataset","authors":"Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun","year":"2019","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ye et al., 2020] Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Maosong Sun, and Zhiyuan Liu. Coreferential reasoning learning for language representation. In Proceedings of EMNLP, 2020.","title":"Coreferential reasoning learning for language representation","authors":"Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Maosong Sun, Zhiyuan Liu","year":"2020","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yeh et al., 2017] Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, and Yu-Chiang Frank Wang. Learning deep latent space for multi- label classification. In Proceedings of AAAI, 2017.","title":"Learning deep latent space for multilabel classification","authors":"Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, Yu-Chiang Frank, Wang","year":"2017","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2020] Dian Yu, Kai Sun, Claire Cardie, and Dong Yu. Dialogue-based relation extraction. In Proceedings of ACL, pages 4927-4940, 2020.","title":"Dialogue-based relation extraction","authors":"Dian Yu, Kai Sun, Claire Cardie, Dong Yu","year":"2020","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zeng et al., 2020] Shuang Zeng, Runxin Xu, Baobao Chang, and Lei Li. Double graph based reasoning for document-level relation extraction. In Proceedings of EMNLP, pages 1630-1640, 2020.","title":"Double graph based reasoning for document-level relation extraction","authors":"Shuang Zeng, Runxin Xu, Baobao Chang, Lei Li","year":"2020","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2018] Yuhao Zhang, Peng Qi, and Christopher D. Manning. Graph convolution over pruned dependency trees im- proves relation extraction. In Proceedings of EMNLP, 2018.","title":"Graph convolution over pruned dependency trees improves relation extraction","authors":"Yuhao Zhang, Peng Qi, Christopher D Manning","year":"2018","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhou et al., 2021] Wenxuan Zhou, Kevin Huang, Tengyu Ma, and Jing Huang. Document-level relation extraction with adaptive thresholding and localized context pooling. In Proceedings of AAAI, 2021.","title":"Document-level relation extraction with adaptive thresholding and localized context pooling","authors":"Wenxuan Zhou, Kevin Huang, Tengyu Ma, Jing Huang","year":"2021","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}