{"type":"ScholarlyArticle","title":"Propose-and-Refine: A Two-Stage Set Prediction Network for Nested Named Entity Recognition","authors":[{"name":"Shuhui Wu","slug":"shuhui-wu","given":"Shuhui","family":"Wu","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Yongliang Shen","slug":"yongliang-shen","given":"Yongliang","family":"Shen","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Zeqi Tan","slug":"zeqi-tan","given":"Zeqi","family":"Tan","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Weiming Lu","slug":"weiming-lu","given":"Weiming","family":"Lu","affiliation":"College of Computer Science and Technology , Zhejiang University"}],"abstract":"Nested named entity recognition (nested NER) is a fundamental task in natural language processing. Various span-based methods have been proposed to detect nested entities with span representations. However, span-based methods do not consider the relationship between a span and other entities or phrases, which is helpful in the NER task. Besides, span-based methods have trouble predicting long entities due to limited span enumeration length. To mitigate these issues, we present the Propose-and-Refine Network (PnRNet), a two-stage set prediction network for nested NER. In the propose stage, we use a span-based predictor to generate some coarse entity predictions as entity proposals. In the refine stage, proposals interact with each other, and richer contextual information is incorporated into the proposal representations. The refined proposal representations are used to re-predict entity boundaries and classes. In this way, errors in coarse proposals can be eliminated, and the boundary prediction is no longer constrained by the span enumeration length limitation. Additionally, we build multi-scale sentence representations, which better model the hierarchical structure of sentences and provide richer contextual information than token-level representations. Experiments show that PnRNet achieves state-of-the-art performance on four nested NER datasets and one flat NER dataset.","doi":"10.24963/ijcai.2022/613","url":"https://ijcai.org/proceedings/2022/613","pdf":"https://ijcai.org/proceedings/2022/0613.pdf","markdown":"https://ijcai.org/proceedings/2022/613.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/613","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":613,"pages":{"first":4418,"last":4424},"keywords":["Natural Language Processing: Information Extraction","Natural Language Processing: Named Entities"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986835","length_sec":79,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984978","length_sec":334,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chiu et al., 2016] Billy Chiu, Gamal Crichton, Anna Ko- rhonen, and Sampo Pyysalo. How to Train good Word Embeddings for Biomedical NLP. In Proceedings of the 15th Workshop on Biomedical Natural Language Process- ing, 2016.","title":"How to Train good Word Embeddings for Biomedical NLP","authors":"Chiu","year":"2016","venue":"Proceedings of the 15th Workshop on Biomedical Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Under- standing. In Proc. of ACL, 2019.","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Doddington et al., 2004] George R. Doddington, Alexis Mitchell, Mark A. Przybocki, Lance A. Ramshaw, Stephanie M. Strassel, and Ralph M. Weischedel. The Au- tomatic Content Extraction (ace) Program -Tasks, Data, and Evaluation. In Proc. of Lrec, 2004.","title":"The Automatic Content Extraction (ace) Program -Tasks, Data, and Evaluation","authors":"George R Doddington, Alexis Mitchell, Mark A Przybocki, Lance A Ramshaw, Stephanie M Strassel, Ralph M Weischedel","year":"2004","venue":"Proc. of Lrec","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Comput., 1997.","title":"Long Short-Term Memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Comput","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Huang et al., 2015] Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional LSTM-CRF Models for Sequence Tagging. ArXiv150801991 Cs, August 2015.","title":"Bidirectional LSTM-CRF Models for Sequence Tagging","authors":"Zhiheng Huang, Wei Xu, Kai Yu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ji et al., 2017] Heng Ji, Xiaoman Pan, Boliang Zhang, Joel Nothman, James Mayfield, Paul McNamee, and Cash Costello. Overview of TAC-KBP2017 13 Languages En- tity Discovery and Linking. In Proc. of Tac, 2017.","title":"Overview of TAC-KBP2017 13 Languages Entity Discovery and Linking","authors":"Ji Heng, Xiaoman Pan, Boliang Zhang, Joel Nothman, James Mayfield, Paul Mcnamee, Cash Costello","year":"2017","venue":"Proc. of Tac","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ju et al., 2018] Meizhi Ju, Makoto Miwa, and Sophia Ana- niadou. A Neural Layered Model for Nested Named Entity Recognition. In Proc. of ACL, 2018.","title":"A Neural Layered Model for Nested Named Entity Recognition","authors":"Ju","year":"2018","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Katiyar and Cardie, 2018] Arzoo Katiyar and Claire Cardie. Nested Named Entity Recognition Revisited. In Proc. of ACL, 2018.","title":"Nested Named Entity Recognition Revisited","authors":"Cardie Katiyar, Arzoo Katiyar, Claire Cardie","year":"2018","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kuhn, 1955] H. W. Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics, 1955.","title":"The Hungarian method for the assignment problem","authors":"H W Kuhn","year":"1955","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lample et al., 2016] Guillaume Lample, Miguel Balles- teros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural Architectures for Named Entity Recognition. In Proc. of ACL, 2016.","title":"Neural Architectures for Named Entity Recognition","authors":"Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer","year":"2016","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lee et al., 2020] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. BioBERT: A pre-trained biomedical lan- guage representation model for biomedical text mining. Bioinformatics, 2020.","title":"BioBERT: A pre-trained biomedical language representation model for biomedical text mining","authors":"Lee","year":"2020","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2020] Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. A Unified MRC Framework for Named Entity Recognition. In Proc. of ACL, 2020.","title":"A Unified MRC Framework for Named Entity Recognition","authors":"Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, Jiwei Li","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2019] Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun. Sequence-to-Nuggets: Nested Entity Mention De- tection via Anchor-Region Networks. In Proc. of ACL, 2019.","title":"Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks","authors":"Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lu and Roth, 2015] Wei Lu and Dan Roth. Joint Mention Extraction and Classification with Mention Hypergraphs. In Proc. of EMNLP, 2015.","title":"Joint Mention Extraction and Classification with Mention Hypergraphs","authors":"Roth ; ] Wei Lu, Dan Lu, Roth","year":"2015","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ohta et al., 2002] Tomoko Ohta, Yuka Tateisi, and Jin- Dong Kim. The Genia Corpus: An Annotated Research Abstract Corpus in Molecular Biology Domain. In Pro- ceedings of the Second International Conference on Hu- man Language Technology Research, 2002.","title":"The Genia Corpus: An Annotated Research Abstract Corpus in Molecular Biology Domain","authors":"Tomoko Ohta, Yuka Tateisi, Jin-Dong Kim","year":"2002","venue":"Proceedings of the Second International Conference on Human Language Technology Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global Vectors for Word Representation. In Proc. of EMNLP, 2014.","title":"Glove: Global Vectors for Word Representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2021] Yongliang Shen, Xinyin Ma, Zeqi Tan, Shuai Zhang, Wen Wang, and Weiming Lu. Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition. In Proc. of ACL, 2021.","title":"Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition","authors":"Yongliang Shen, Xinyin Ma, Zeqi Tan, Shuai Zhang, Wen Wang, Weiming Lu","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sohrab and Miwa, 2018] Mohammad Golam Sohrab and Makoto Miwa. Deep Exhaustive Model for Nested Named Entity Recognition. In Proc. of EMNLP, 2018.","title":"Deep Exhaustive Model for Nested Named Entity Recognition","authors":"Mohammad Golam, Sohrab, Makoto Miwa","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Straková et al., 2019] Jana Straková, Milan Straka, and Jan Hajic. Neural Architectures for Nested NER through Lin- earization. In Proc. of ACL, 2019.","title":"Neural Architectures for Nested NER through Linearization","authors":"Jana Straková, Milan Straka, Jan Hajic","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tan et al., 2020] Chuanqi Tan, Wei Qiu, Mosha Chen, Rui Wang, and Fei Huang. Boundary Enhanced Neural Span Classification for Nested Named Entity Recognition. AAAI, 2020.","title":"Boundary Enhanced Neural Span Classification for Nested Named Entity Recognition","authors":"Chuanqi Tan, Wei Qiu, Mosha Chen, Rui Wang, Fei Huang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tan et al., 2021] Zeqi Tan, Yongliang Shen, Shuai Zhang, Weiming Lu, and Yueting Zhuang. A Sequence-to-Set Network for Nested Named Entity Recognition. In Proc. of IJCAI, 2021.","title":"A Sequence-to-Set Network for Nested Named Entity Recognition","authors":"Zeqi Tan, Yongliang Shen, Shuai Zhang, Weiming Lu, Yueting Zhuang","year":"2021","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/542"},{"position":22,"raw":"[Tjong Kim Sang and De Meulder, 2003] Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. In Proc. of HLT-NAACL, 2003.","title":"Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition","authors":"Tjong Kim, Sang, De Meulder, ; ] Erik, F Tjong, Kim Sang, Fien De, Meulder","year":"2003","venue":"Proc. of HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Proc. of NeurIPS, 2017.","title":"Attention is All you Need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Walker et al., 2006] Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. ACE 2005 Multilingual Training Corpus, February 2006.","title":"ACE 2005 Multilingual Training Corpus","authors":"Christopher Walker, Stephanie Strassel, Julie Medero, Kazuaki Maeda","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018] Bailin Wang, Wei Lu, Yu Wang, and Hongxia Jin. A Neural Transition-based Model for Nested Mention Recognition. In Proc. of EMNLP, 2018.","title":"A Neural Transition-based Model for Nested Mention Recognition","authors":"Bailin Wang, Wei Lu, Yu Wang, Hongxia Jin","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2020] Jue Wang, Lidan Shou, Ke Chen, and Gang Chen. Pyramid: A Layered Model for Nested Named Entity Recognition. In Proc. of ACL, 2020.","title":"Pyramid: A Layered Model for Nested Named Entity Recognition","authors":"Jue Wang, Lidan Shou, Ke Chen, Gang Chen","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yan et al., 2021] Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, and Xipeng Qiu. A Unified Gener- ative Framework for Various NER Subtasks. In Proc. of ACL, 2021.","title":"A Unified Generative Framework for Various NER Subtasks","authors":"Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, Xipeng Qiu","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yu et al., 2020] Juntao Yu, Bernd Bohnet, and Massimo Poesio. Named Entity Recognition as Dependency Pars- ing. In Proc. of ACL, 2020.","title":"Named Entity Recognition as Dependency Parsing","authors":"Juntao Yu, Bernd Bohnet, Massimo Poesio","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}