{"type":"ScholarlyArticle","title":"Attention-based Multi-level Feature Fusion for Named Entity Recognition","authors":[{"name":"Zhiwei Yang","slug":"zhiwei-yang","given":"Zhiwei","family":"Yang","affiliation":"College of Computer Science and Technology , Jilin University , Changchun , China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education , China"},{"name":"Hechang Chen","slug":"hechang-chen","given":"Hechang","family":"Chen","affiliation":"School of Artificial Intelligence , Jilin University , Changchun , China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education , China"},{"name":"Jiawei Zhang","slug":"jiawei-zhang","given":"Jiawei","family":"Zhang","affiliation":"IFM Lab , Department of Computer Science , Florida State University , Tallahassee FL , USA"},{"name":"Jing Ma","slug":"jing-ma","given":"Jing","family":"Ma","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong , China; Washington University ,; named after George Washington . Word_Local Char_Global Word_Global Char_Local"},{"name":"Yi Chang","slug":"yi-chang","given":"Yi","family":"Chang","affiliation":"School of Artificial Intelligence , Jilin University , Changchun , China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education , China; International Center of Future Science , Jilin University , Changchun , China; which is located in Missouri , is"}],"abstract":"Named entity recognition (NER) is a fundamental task in the natural language processing (NLP) area. Recently, representation learning methods (e.g., character embedding and word embedding) have achieved promising recognition results. However, existing models only consider partial features derived from words or characters while failing to integrate semantic and syntactic information (e.g., capitalization, inter-word relations, keywords, lexical phrases, etc.) from multi-level perspectives. Intuitively, multi-level features can be helpful when recognizing named entities from complex sentences. In this study, we propose a novel framework called attention-based multi-level feature fusion (AMFF), which is used to capture the multi-level features from different perspectives to improve NER. Our model consists of four components to respectively capture the local character-level, global character-level, local word-level, and global word-level features, which are then fed into a BiLSTM-CRF network for the final sequence labeling. Extensive experimental results on four benchmark datasets show that our proposed model outperforms a set of state-of-the-art baselines.","doi":"10.24963/ijcai.2020/497","url":"https://ijcai.org/proceedings/2020/497","pdf":"https://ijcai.org/proceedings/2020/0497.pdf","markdown":"https://ijcai.org/proceedings/2020/497.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/497","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":497,"pages":{"first":3594,"last":3600},"keywords":["Natural Language Processing: Tagging, chunking, and parsing","Natural Language Processing: Information Extraction","Natural Language Processing: Natural Language Processing","Natural Language Processing: Named Entities"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/23989/file_23989.m3u8","length_sec":271,"provider":"hls"}],"references":[{"position":1,"raw":"[Akbik et al., 2018] Alan Akbik, Duncan Blythe, and Roland Vollgraf. Contextual string embeddings for se- quence labeling. In COLING, pages 1638-1649, 2018.","title":"Contextual string embeddings for sequence labeling","authors":"Alan Akbik, Duncan Blythe, Roland Vollgraf","year":"2018","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine transla- tion by jointly learning to align and translate. arXiv preprint:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beltagy et al., 2019] Iz Beltagy, Kyle Lo, and Arman Co- han. Scibert: A pretrained language model for scientific text. In EMNLP, pages 3606-3611, 2019.","title":"Scibert: A pretrained language model for scientific text","authors":"Beltagy","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chiu and Nichols, 2016] Jason PC Chiu and Eric Nichols. Named entity recognition with bidirectional lstm-cnns. Transactions of the Association for Computational Lin- guistics, 4:357-370, 2016.","title":"Named entity recognition with bidirectional lstm-cnns","authors":"Jason Pc Chiu, Eric Nichols","year":"2016","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. Journal of machine learning research, 12(1):2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa","year":"2011","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dogan et al., 2014] Rezarta Islamaj Dogan, Robert Lea- man, and Zhiyong Lu. Ncbi disease corpus: a resource for disease name recognition and concept normalization. Journal of biomedical informatics, 47:1-10, 2014.","title":"Rezarta Islamaj Dogan, Robert Leaman, and Zhiyong Lu. Ncbi disease corpus: a resource for disease name recognition and concept normalization","authors":"Dogan","year":"2014","venue":"Journal of biomedical informatics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dong et al., 2016] Chuanhai Dong, Jiajun Zhang, Chengqing Zong, Masanori Hattori, and Hui Di. Character-based lstm-crf with radical-level features for chinese named entity recognition. In Natural Lan- guage Understanding and Intelligent Applications, pages 239-250. Springer, 2016.","title":"Character-based lstm-crf with radical-level features for chinese named entity recognition","authors":"Dong","year":"2016","venue":"Natural Language Understanding and Intelligent Applications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2015] Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional lstm-crf models for sequence tagging. arXiv preprint:1508.01991, 2015.","title":"Bidirectional lstm-crf models for sequence tagging","authors":"Zhiheng Huang, Wei Xu, Kai Yu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2004] Jin-Dong Kim, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Nigel Collier. Introduction to the bio-entity recognition task at jnlpba. In JNLPBA, pages 70-75. Citeseer, 2004.","title":"Introduction to the bio-entity recognition task at jnlpba","authors":"Jin-Dong Kim, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, Nigel Collier","year":"2004","venue":"JNLPBA","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kim et al., 2016] Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. Character-aware neural language models. In AAAI, pages 2741-2749, 2016.","title":"Character-aware neural language models","authors":"Yoon Kim, Yacine Jernite, David Sontag, Alexander M Rush","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kuru et al., 2016] Onur Kuru, Ozan Arkan Can, and Deniz Yuret. Charner: Character-level named entity recognition. In COLING, pages 911-921, 2016.","title":"Charner: Character-level named entity recognition","authors":"Onur Kuru, Ozan Arkan Can, Deniz Yuret","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lample et al., 2016] Guillaume Lample, Miguel Balles- teros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural architectures for named entity recog- nition. arXiv preprint:1603.01360, 2016.","title":"Neural architectures for named entity recognition","authors":"Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2018] Liyuan Liu, Jingbo Shang, Xiang Ren, Frank Fangzheng Xu, Huan Gui, Jian Peng, and Jiawei Han. Empower sequence labeling with task-aware neural language model. In AAAI, pages 5253-5260, 2018.","title":"Empower sequence labeling with task-aware neural language model","authors":"Liyuan Liu, Jingbo Shang, Xiang Ren, Frank Fangzheng Xu, Huan Gui, Jian Peng, Jiawei Han","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Luan et al., 2018] Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. Multi-task identification of en- tities, relations, and coreference for scientific knowledge graph construction. arXiv preprint:1808.09602, 2018.","title":"Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction","authors":"Luan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma and Hovy, 2016] Xuezhe Ma and Eduard Hovy. End- to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint:1603.01354, 2016.","title":"Xuezhe Ma and Eduard Hovy. Endto-end sequence labeling via bi-directional lstm-cnns-crf","authors":"Hovy Ma","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Quimbaya et al., 2016] Alexandra Pomares Quimbaya, Alejandro Sierra Múnera, Rafael Andrés González Rivera, Julián Camilo Daza Rodríguez, Oscar Mauricio Muñoz Velandia, Angel Alberto Garcia Peña, and Cyril Labbé. Named entity recognition over electronic health records through a combined dictionary-based approach. Procedia Computer Science, 100:55-61, 2016.","title":"Named entity recognition over electronic health records through a combined dictionary-based approach","authors":"Alexandra Pomares Quimbaya, Alejandro Sierra Múnera, Rafael Andrés, González Rivera, Julián Camilo Daza, Oscar Rodríguez, Mauricio Muñoz, Angel Velandia, Alberto Garcia Peña, Cyril Labbé","year":"2016","venue":"Procedia Computer Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rahem and Omar, 2014] Khmael Rakm Rahem and Nazlia Omar. Drug-related crime information extraction and anal- ysis. In ICIMU, pages 250-254. IEEE, 2014.","title":"Khmael Rakm Rahem and Nazlia Omar. Drug-related crime information extraction and analysis","authors":"Omar Rahem","year":"2014","venue":"ICIMU","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sang and De Meulder, 2003] Erik F Sang and Fien De Meulder. Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050, 2003.","title":"Introduction to the conll-2003 shared task: Language-independent named entity recognition","authors":"Sang, De Meulder, ; Erik, F Sang, Fien De, Meulder","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xin et al., 2018] Yingwei Xin, Ethan Hart, Vibhuti Maha- jan, and Jean-David Ruvini. Learning better internal struc- ture of words for sequence labeling. In EMNLP, pages 2584-2593, 2018.","title":"Learning better internal structure of words for sequence labeling","authors":"Xin","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yadav et al., 2018] Vikas Yadav, Rebecca Sharp, and Steven Bethard. Deep affix features improve neural named entity recognizers. In *SEM, pages 167-172, 2018.","title":"Deep affix features improve neural named entity recognizers","authors":"Vikas Yadav, Rebecca Sharp, Steven Bethard","year":"2018","venue":"SEM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yijin Liu and Zhou, 2019] Jinchao Zhang Jinan Xu Yufeng Chen Yijin Liu, Fandong Meng and Jie Zhou. Gcdt: A global context enhanced deep transition archi- tecture for sequence labeling. In ACL, pages 2431-2441, 2019.","title":"Gcdt: A global context enhanced deep transition architecture for sequence labeling","authors":"Yijin Liu, ; ] Jinchao Zhou, Jinan Zhang, Yufeng Xu, Fandong Chen Yijin Liu, Jie Meng, Zhou","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yoon et al., 2019] Wonjin Yoon, Chan Ho So, Jinhyuk Lee, and Jaewoo Kang. Collabonet: collaboration of deep neural networks for biomedical named entity recognition. BMC bioinformatics, 20(10):249, 2019.","title":"Collabonet: collaboration of deep neural networks for biomedical named entity recognition","authors":"Yoon","year":"2019","venue":"BMC bioinformatics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zheng et al., 2019] Changmeng Zheng, Yi Cai, Jingyun Xu, Ho-fung Leung, and Guandong Xu. A boundary-aware neural model for nested named entity recognition. In EMNLP, pages 357-366, 2019.","title":"A boundary-aware neural model for nested named entity recognition","authors":"Changmeng Zheng, Yi Cai, Jingyun Xu, Ho-Fung Leung, Guandong Xu","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}