{"type":"ScholarlyArticle","title":"Knowledge Graphs Enhanced Neural Machine Translation","authors":[{"name":"Yang Zhao","slug":"yang-zhao","given":"Yang","family":"Zhao","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; School of Artificial Intelligence , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Jiajun Zhang","slug":"jiajun-zhang","given":"Jiajun","family":"Zhang","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; School of Artificial Intelligence , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Yu Zhou","slug":"yu-zhou","given":"Yu","family":"Zhou","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; Beijing Fanyu Technology Co., Ltd , Beijing , China"},{"name":"Chengqing Zong","slug":"chengqing-zong","given":"Chengqing","family":"Zong","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; School of Artificial Intelligence , University of Chinese Academy of Sciences , Beijing , China; CAS Center for Excellence in Brain Science and Intelligence Technology , Beijing , China"}],"abstract":"Knowledge graphs (KGs) store much structured information on various entities, many of which are not covered by the parallel sentence pairs of neural machine translation (NMT). To improve the translation quality of these entities, in this paper\r\nwe propose a novel KGs enhanced NMT method. Specifically, we first induce the new translation results of these entities by transforming the source and target KGs into a unified semantic space. We then generate adequate pseudo parallel sentence pairs that contain these induced entity pairs. Finally, NMT model is jointly trained by the original and pseudo sentence pairs. The extensive experiments on Chinese-to-English and Englishto-Japanese translation tasks demonstrate that our method significantly outperforms the strong baseline models in translation quality, especially in handling the induced entities.","doi":"10.24963/ijcai.2020/559","url":"https://ijcai.org/proceedings/2020/559","pdf":"https://ijcai.org/proceedings/2020/0559.pdf","markdown":"https://ijcai.org/proceedings/2020/559.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/559","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":559,"pages":{"first":4039,"last":4045},"keywords":["Natural Language Processing: Natural Language Generation","Natural Language Processing: Machine Translation","Knowledge Representation and Reasoning: Semantic Web"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25593/file_25593.m3u8","length_sec":159,"provider":"hls"}],"references":[{"position":1,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi- relational data. In Proceedings of NeurIPS 2013, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multirelational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"Proceedings of NeurIPS 2013","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2019] Yixin Cao, Zhiyuan Liu, Chengjiang Li, Zhiyuan Liu, Juanzi Li, and Tat-Seng Chua. Multi-channel graph neural network for entity alignment. In Proceedings of ACL 2019, pages 1452-1461, 2019.","title":"Multi-channel graph neural network for entity alignment","authors":"Yixin Cao, Zhiyuan Liu, Chengjiang Li, Zhiyuan Liu, Juanzi Li, Tat-Seng Chua","year":"2019","venue":"Proceedings of ACL 2019","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017] Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In Proceedings of IJCAI 2017, pages 1511-1517, 2017.","title":"Multilingual knowledge graph embeddings for cross-lingual knowledge alignment","authors":"Chen","year":"2017","venue":"Proceedings of IJCAI 2017","doi":null,"url":null,"internal_url":"/proceedings/2017/209"},{"position":4,"raw":"[Dinu et al., 2019] Georgiana Dinu, Prashant Mathur, Mar- cello Federico, and Yaser Al-Onaizan. Training neural machine translation to apply terminology constraints. In Proceedings of ACL 2019, pages 3063-3068, 2019.","title":"Training neural machine translation to apply terminology constraints","authors":"Georgiana Dinu, Prashant Mathur, Marcello Federico, Yaser Al-Onaizan","year":"2019","venue":"Proceedings of ACL 2019","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hao et al., 2016] Yanchao Hao, Yuanzhe Zhang, Shizhu He, Kang Liu, and Jun Zhao. A joint embedding method for entity alignment of knowledge bases. In Proceedings of CCKS 2016, pages 3-14, 2016.","title":"A joint embedding method for entity alignment of knowledge bases","authors":"Hao","year":"2016","venue":"Proceedings of CCKS 2016","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hasler et al., 2018] Eva Hasler, Adrià de Gispert, Gonzalo Iglesias, and Bill Byrne. Neural machine translation de- coding with terminology constraints. In Proceedings of NAACL-NLT 2018, pages 506-512, 2018.","title":"Neural machine translation decoding with terminology constraints","authors":"Eva Hasler, Adrià De Gispert, Gonzalo Iglesias, Bill Byrne","year":"2018","venue":"Proceedings of NAACL-NLT 2018","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huck et al., 2019] Matthias Huck, Viktor Hangya, and Alexander Fraser. Better OOV translation with bilingual terminology mining. In Proceedings of ACL 2019, pages 5809-5815, 2019.","title":"Better OOV translation with bilingual terminology mining","authors":"Huck","year":"2019","venue":"Proceedings of ACL 2019","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lin et al., 2015] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation em- beddings for knowledge graph completion. In Proceedings of AAAI 2015, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu","year":"2015","venue":"Proceedings of AAAI 2015","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lu et al., 2018] Yu Lu, Jiajun Zhang, and Chengqing Zong. Exploiting knowledge graph in neural machine translation. In Proceedings of CWMT 2018, pages 27-38, 2018.","title":"Exploiting knowledge graph in neural machine translation","authors":"Yu Lu, Jiajun Zhang, Chengqing Zong","year":"2018","venue":"Proceedings of CWMT 2018","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Luong et al., 2015] Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention- based neural machine translation. In Proceedings of EMNLP 2015, pages 1412-1421, 2015.","title":"Effective approaches to attentionbased neural machine translation","authors":"Minh-Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":"Proceedings of EMNLP 2015","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Moussallem et al., 2018] Diego Moussallem, Matthias Wauer, and Axel-Cyrille Ngonga Ngomo. Machine translation using semantic web technologies: A survey. Journal of Web Semantics, 51:1-19, 2018.","title":"Machine translation using semantic web technologies: A survey","authors":"Moussallem","year":"2018","venue":"Journal of Web Semantics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Moussallem et al., 2019] Diego Moussallem, Axel-Cyrille Ngonga Ngomo, Paul Buitelaar, and Mihael Arcan. Utiliz- ing knowledge graphs for neural machine translation aug- mentation. In Proceedings of K-CAP 2019, pages 139- 146, 2019.","title":"Utilizing knowledge graphs for neural machine translation augmentation","authors":"Moussallem","year":"2019","venue":"Proceedings of K-CAP 2019","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sennrich et al., 2016] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of ACL 2016, pages 1715-1725, 2016.","title":"Neural machine translation of rare words with subword units","authors":"Sennrich","year":"2016","venue":"Proceedings of ACL 2016","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Shi et al., 2016] Chen Shi, Shujie Liu, Shuo Ren, Shi Feng, Mu Li, Ming Zhou, Xu Sun, and Houfeng Wang. Knowledge-based semantic embedding for machine trans- lation. In Proceedings of ACL 2016, pages 2245-2254, 2016.","title":"Knowledge-based semantic embedding for machine translation","authors":"Chen Shi, Shujie Liu, Shuo Ren, Shi Feng, Mu Li, Ming Zhou, Xu Sun, Houfeng Wang","year":"2016","venue":"Proceedings of ACL 2016","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Suchanek et al., 2007] Fabian M Suchanek, Gjergji Kas- neci, and Gerhard Weikum. Yago: a core of semantic knowledge. In Proceedings of WWW 2007, pages 697- 706, 2007.","title":"Yago: a core of semantic knowledge","authors":"Gjergji Fabian M Suchanek, Gerhard Kasneci, Weikum","year":"2007","venue":"Proceedings of WWW 2007","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sun et al., 2017] Zequn Sun, Wei Hu, and Chengkai Li. Cross-lingual entity alignment via joint attribute- preserving embedding. In Proceedings of ISWC 2017, pages 628-644, 2017.","title":"Cross-lingual entity alignment via joint attributepreserving embedding","authors":"Sun","year":"2017","venue":"Proceedings of ISWC 2017","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ugawa et al., 2018] Arata Ugawa, Akihiro Tamura, Takashi Ninomiya, Hiroya Takamura, and Manabu Okumura. Neu- ral machine translation incorporating named entity. In Pro- ceedings of COLING 2018, pages 3240-3250, 2018.","title":"Neural machine translation incorporating named entity","authors":"Arata Ugawa, Akihiro Tamura, Takashi Ninomiya, Hiroya Takamura, Manabu Okumura","year":"2018","venue":"Proceedings of COLING 2018","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of NeurIPS 2017, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proceedings of NeurIPS 2017","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of AAAI 2014, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen","year":"2014","venue":"Proceedings of AAAI 2014","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2018] Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. Cross-lingual knowledge graph align- ment via graph convolutional networks. In Proceedings of EMNLP 2018, pages 349-357, 2018.","title":"Cross-lingual knowledge graph alignment via graph convolutional networks","authors":"Zhichun Wang, Qingsong Lv, Xiaohan Lan, Yu Zhang","year":"2018","venue":"Proceedings of EMNLP 2018","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2019] Tao Wang, Shaohui Kuang, Deyi Xiong, and António Branco. Merging external bilingual pairs into neural machine translation. arXiv preprint arXiv:1912.00567, 2019.","title":"Merging external bilingual pairs into neural machine translation","authors":"Tao Wang, Shaohui Kuang, Deyi Xiong, António Branco","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2019] Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, and Dongyan Zhao. Jointly learning entity and relation representations for entity alignment. In Pro- ceedings of EMNLP-IJCNLP 2019, pages 240-249, 2019.","title":"Jointly learning entity and relation representations for entity alignment","authors":"Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, Dongyan Zhao","year":"2019","venue":"Proceedings of EMNLP-IJCNLP 2019","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and Zong, 2016] Jiajun Zhang and Chengqing Zong. Bridging neural machine translation and bilingual dictio- naries. arXiv preprint arXiv:1610.07272, 2016.","title":"Bridging neural machine translation and bilingual dictionaries","authors":"Jiajun Zhang, Chengqing Zong","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2018a] Yang Zhao, Yining Wang, Jiajun Zhang, and Chengqing Zong. Phrase table as recommendation memory for neural machine translation. In Proceedings of IJCAI 2018, pages 4609-4615, 2018.","title":"Phrase table as recommendation memory for neural machine translation","authors":"Zhao","year":"2018","venue":"Proceedings of IJCAI 2018","doi":null,"url":null,"internal_url":"/proceedings/2018/641"},{"position":25,"raw":"[Zhao et al., 2018b] Yang Zhao, Jiajun Zhang, Zhongjun He, Chengqing Zong, and Hua Wu. Addressing troublesome words in neural machine translation. In Proceedings of EMNLP 2018, pages 391-400, 2018.","title":"Addressing troublesome words in neural machine translation","authors":"Zhao","year":"2018","venue":"Proceedings of EMNLP 2018","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2017] Hao Zhu, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. Iterative entity alignment via joint knowl- edge embeddings. In Proceedings of IJCAI 2017, pages 4258-4264, 2017.","title":"Iterative entity alignment via joint knowledge embeddings","authors":"Ruobing Hao Zhu, Zhiyuan Xie, Maosong Liu, Sun","year":"2017","venue":"Proceedings of IJCAI 2017","doi":null,"url":null,"internal_url":"/proceedings/2017/595"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}