{"type":"ScholarlyArticle","title":"Neural Collective Entity Linking Based on Recurrent Random Walk Network Learning","authors":[{"name":"Mengge Xue","slug":"mengge-xue","given":"Mengge","family":"Xue","affiliation":"Xiamen University; Institute of Information Engineering , Chinese Academy of Sciences; School of Cyber Security , University of Chinese Academy of Sciences"},{"name":"Weiming Cai","slug":"weiming-cai","given":"Weiming","family":"Cai","affiliation":"Xiamen University"},{"name":"Jinsong Su","slug":"jinsong-su","given":"Jinsong","family":"Su","affiliation":"Xiamen University"},{"name":"Linfeng Song","slug":"linfeng-song","given":"Linfeng","family":"Song","affiliation":"Rochester University"},{"name":"Yubin Ge","slug":"yubin-ge","given":"Yubin","family":"Ge","affiliation":"Rochester University"},{"name":"Yubao Liu","slug":"yubao-liu","given":"Yubao","family":"Liu","affiliation":"Rochester University"},{"name":"Bin Wang","slug":"bin-wang","given":"Bin","family":"Wang","affiliation":"Xiaomi AI Lab , Xiaomi Inc. , Beijing , China"}],"abstract":"Benefiting from the excellent ability of neural networks on learning semantic representations, existing studies for entity linking (EL) have resorted to neural networks to exploit both the local mention-to-entity compatibility and the global interdependence between different EL decisions for target entity disambiguation. However, most neural collective EL methods depend entirely upon neural networks to automatically model the semantic dependencies between different EL decisions, which lack of the guidance from external knowledge. In this paper, we propose a novel end-to-end neural network with recurrent random-walk layers for collective EL, which introduces external knowledge to model the semantic interdependence between different EL decisions. Specifically, we first establish a model based on local context features, and then stack random-walk layers to reinforce the evidence for related EL decisions into high-probability decisions, where the semantic interdependence between candidate entities is mainly induced from an external knowledge base. Finally, a semantic regularizer that preserves the collective EL decisions consistency is incorporated into the conventional objective function, so that the external knowledge base can be fully exploited in collective EL decisions. Experimental results and in-depth analysis on various datasets show that our model achieves better performance than other state-of-the-art models. Our code and data are released at https://github.com/DeepLearnXMU/RRWEL.","doi":"10.24963/ijcai.2019/740","url":"https://ijcai.org/proceedings/2019/740","pdf":"https://ijcai.org/proceedings/2019/0740.pdf","markdown":"https://ijcai.org/proceedings/2019/740.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/740","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":740,"pages":{"first":5327,"last":5333},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Named Entities","Natural Language Processing: Information Extraction"],"videos":[],"references":[{"position":1,"raw":"[Blanco et al., 2015] Roi Blanco, Giuseppe Ottaviano, and Edgar Meij. Fast and space-efficient entity linking for queries. In WSDM, pages 179-188, 2015.","title":"Fast and space-efficient entity linking for queries","authors":"Roi Blanco, Giuseppe Ottaviano, Edgar Meij","year":"2015","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2018] Yixin Cao, Lei Hou, Juanzi Li, and Zhiyuan Liu. Neural collective entity linking. In COL- ING, pages 675-686, 2018.","title":"Neural collective entity linking","authors":"Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu","year":"2018","venue":"COL-ING","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng and Roth, 2013] X. Cheng and D. Roth. Relational inference for wikification. In EMNLP, pages 1787-1796, 2013.","title":"Relational inference for wikification","authors":"Roth ; Cheng, X Cheng, D Roth","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chisholm and Hachey, 2015] Andrew Chisholm and Ben Hachey. Entity disambiguation with web links. TACL, pages 145-156, 2015.","title":"Entity disambiguation with web links","authors":"Hachey Chisholm, Andrew Chisholm, Ben Hachey","year":"2015","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Duchi et al., 2011] John C. Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learn- ing and stochastic optimization. JMLR, pages 2121-2159, 2011.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"John C Duchi, Elad Hazan, Yoram Singer","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Francis-Landau et al., 2016] Matthew Francis-Landau, Greg Durrett, and Klein Dan. Capturing semantic similar- ity for entity linking with convolutional neural networks. In NAACL, pages 1256-1261, 2016.","title":"Capturing semantic similarity for entity linking with convolutional neural networks","authors":"Matthew Francis-Landau, Greg Durrett, Klein Dan","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gabrilovich et al., 2013] Evgeniy Gabrilovich, Michael Ringgaard, and Amarnag Subramanya. Facc1: Freebase annotation of clueweb corpora, version 1 (release date 2013-06-26, format version 1, correction level 0). 2013.","title":"Facc1: Freebase annotation of clueweb corpora","authors":"Evgeniy Gabrilovich, Michael Ringgaard, Amarnag Subramanya","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ganea and Hofmann, 2017] Octavian Eugen Ganea and Thomas Hofmann. Deep joint entity disambiguation with local neural attention. In EMNLP, pages 2619-2629, 2017.","title":"Deep joint entity disambiguation with local neural attention","authors":"Hofmann ; Ganea, Eugen Octavian, Thomas Ganea, Hofmann","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ganea et al., 2015] Octavian Eugen Ganea, Marina Ganea, Aurelien Lucchi, Carsten Eickhoff, and Thomas Hofmann. Probabilistic bag-of-hyperlinks model for entity linking. In WWW, pages 927-938, 2015.","title":"Probabilistic bag-of-hyperlinks model for entity linking","authors":"Ganea","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gilks et al., 1995] Walter R Gilks, Sylvia Richardson, and David Spiegelhalter. Markov Chain Monte Carlo in Prac- tice. CRC press, 1995.","title":"Markov Chain Monte Carlo in Practice","authors":"Sylvia Walter R Gilks, David Richardson, Spiegelhalter","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Globerson et al., 2016] Amir Globerson, Nevena Lazic, Soumen Chakrabarti, Amarnag Subramanya, Michael Ringaard, and Fernando Pereira. Collective entity reso- lution with multi-focal attention. In ACL, pages 621-631, 2016.","title":"Collective entity resolution with multi-focal attention","authors":"Nevena Amir Globerson, Soumen Lazic, Amarnag Chakrabarti, Michael Subramanya, Fernando Ringaard, Pereira","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo and Barbosa, 2018] Zhaochen Guo and Denilson Bar- bosa. Robust named entity disambiguation with random walks. Semantic Web, pages 459-479, 2018.","title":"Zhaochen Guo and Denilson Barbosa. Robust named entity disambiguation with random walks","authors":"Barbosa Guo","year":"2018","venue":"Semantic Web","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Han et al., 2011] Xianpei Han, Le Sun, and Jun Zhao. Col- lective entity linking in web text:a graph-based method. In SIGIR, pages 765-774, 2011.","title":"Collective entity linking in web text:a graph-based method","authors":"Han","year":"2011","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2013] Z. He, S. Liu, Y. Song, M. Li, M. Zhou, and H. Wang. Efficient collective entity linking with stacking. In EMNLP, pages 30-34, 2013.","title":"Efficient collective entity linking with stacking","authors":"Z He, S Liu, Y Song, M Li, M Zhou, H Wang","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hoffart et al., 2011] Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen , Furstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum. Robust disambiguation of named entities in text. In EMNLP, pages 782-792, 2011.","title":"Robust disambiguation of named entities in text","authors":"Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen, Manfred Furstenau, Marc Pinkal, Bilyana Spaniol, Stefan Taneva, Gerhard Thater, Weikum","year":"2011","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huu et al., 2016] Thien Huu, Nicolas Fauceglia, Mari- ano Rodriguez Muro, Oktie Hassanzadeh, Alfio Massim- iliano Gliozzo, and Mohammad Sadoghi. Joint learning of local and global features for entity linking via neural networks. In COLING, pages 2310-2320, 2016.","title":"Joint learning of local and global features for entity linking via neural networks","authors":"Huu","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ji and Grishman, 2011] Heng Ji and Ralph Grishman. Knowledge base population: Successful approaches and challenges. In ACL, pages 1148-1158, 2011.","title":"Knowledge base population: Successful approaches and challenges","authors":"Ji, Grishman ; Heng, Ji, Ralph Grishman","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ji and Nothman, 2016] Heng Ji and Joel Nothman. Overview of TAC-KBP2016 tri-lingual EDL and its impact on end-to-end KBP. In TAC, 2016.","title":"Overview of TAC-KBP2016 tri-lingual EDL and its impact on end-to-end KBP","authors":"Heng Ji, Joel Nothman","year":"2016","venue":"TAC","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kolitsas et al., 2018] Nikolaos Kolitsas, Octavianeugen Ganea, and Thomas Hofmann. End-to-end neural entity linking. In CoNLL, pages 519-529, 2018.","title":"End-to-end neural entity linking","authors":"Nikolaos Kolitsas, Octavianeugen Ganea, Thomas Hofmann","year":"2018","venue":"CoNLL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Le and Titov, 2018] Phong Le and Ivan Titov. Improving entity linking by modeling latent relations between men- tions. In ACL, pages 1595-1604, 2018.","title":"Phong Le and Ivan Titov. Improving entity linking by modeling latent relations between mentions","authors":"Titov Le","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mendes et al., 2011] Pablo N. Mendes, Max Jakob, and Christian Bizer. Dbpedia spotlight:shedding light on the web of documents. In I-Semantics, pages 1-8, 2011.","title":"Dbpedia spotlight:shedding light on the web of documents","authors":"Pablo N Mendes, Max Jakob, Christian Bizer","year":"2011","venue":"I-Semantics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed rep- resentations of words and phrases and their composition- ality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shen et al., 2015] Wei Shen, Jianyong Wang, and Jiawei Han. Entity linking with a knowledge base: Issues, tech- niques, and solutions. IEEE Trans Knowl Data Eng, pages 443-460, 2015.","title":"Entity linking with a knowledge base: Issues, techniques, and solutions","authors":"Wei Shen, Jianyong Wang, Jiawei Han","year":"2015","venue":"IEEE Trans Knowl Data Eng","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Song et al., 2019] Linfeng Song, Daniel Gildea, Yue Zhang, Zhiguo Wang, and Jinsong Su. Semantic neural machine translation using amr. arXiv preprint arXiv:1902.07282, 2019.","title":"Semantic neural machine translation using amr","authors":"Song","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Su et al., 2015] Jinsong Su, Deyi Xiong, Shujian Huang, Xianpei Han, and Junfeng Yao. Graph-based collective lexical selection for statistical machine translation. In EMNLP, pages 1238-1247, 2015.","title":"Graph-based collective lexical selection for statistical machine translation","authors":"Jinsong Su, Deyi Xiong, Shujian Huang, Xianpei Han, Junfeng Yao","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yamada et al., 2016] Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. Joint learning of the embedding of words and entities for named entity disambiguation. In CoNLL, pages 250-259, 2016.","title":"Joint learning of the embedding of words and entities for named entity disambiguation","authors":"Yamada","year":"2016","venue":"CoNLL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2018] Yue Zhang, Qi Liu, and Linfeng Song. Sentence-state LSTM for text representation. In ACL, pages 317-327, 2018.","title":"Sentence-state LSTM for text representation","authors":"Yue Zhang, Qi Liu, Linfeng Song","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhao et al., 2017] Zhou Zhao, Hanqing Lu, Deng Cai, Xi- aofei He, and Yueting Zhuang. Microblog sentiment clas- sication via recurrent random walk network learning. In IJCAI, pages 3532-3538, 2017.","title":"Microblog sentiment classication via recurrent random walk network learning","authors":"Zhou Zhao, Hanqing Lu, Deng Cai, Xiaofei He, Yueting Zhuang","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}