{"type":"ScholarlyArticle","title":"Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment","authors":[{"name":"Muhao Chen","slug":"muhao-chen","given":"Muhao","family":"Chen","affiliation":"Department of Computer Science , Stony Brook University"},{"name":"Yingtao Tian","slug":"yingtao-tian","given":"Yingtao","family":"Tian","affiliation":"Department of Computer Science , UCLA"},{"name":"Mohan Yang","slug":"mohan-yang","given":"Mohan","family":"Yang","affiliation":"Department of Computer Science , Stony Brook University"},{"name":"Carlo Zaniolo","slug":"carlo-zaniolo","given":"Carlo","family":"Zaniolo","affiliation":"Department of Computer Science , Stony Brook University"}],"abstract":"Many recent works have demonstrated the benefits of knowledge graph embeddings in completing monolingual knowledge graphs. Inasmuch as related knowledge bases are built in several different languages, achieving cross-lingual knowledge alignment will help people in constructing a coherent knowledge base, and assist machines in dealing with different expressions of entity relationships across diverse human languages. Unfortunately, achieving this highly desirable cross-lingual alignment by human labor is very costly and error-prone. Thus, we propose MTransE, a translation-based model for multilingual knowledge graph embeddings, to provide a simple and automated solution. By encoding entities and relations of each language in a separated embedding space, MTransE provides transitions for each embedding vector to its cross-lingual counterparts in other spaces, while preserving the functionalities of monolingual embeddings. We deploy three different techniques to represent cross-lingual transitions, namely axis calibration, translation vectors, and linear transformations, and derive five variants for MTransE using different loss functions. Our models can be trained on partially aligned graphs, where just a small portion of triples are aligned with their cross-lingual counterparts. The experiments on cross-lingual entity matching and triple-wise alignment verification show promising results, with some variants consistently outperforming others on different tasks. We also explore how MTransE preserves the key properties of its monolingual counterpart.","doi":"10.24963/ijcai.2017/209","url":"https://ijcai.org/proceedings/2017/209","pdf":"https://ijcai.org/proceedings/2017/0209.pdf","markdown":"https://ijcai.org/proceedings/2017/209.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/209","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":209,"pages":{"first":1511,"last":1517},"keywords":["Natural Language Processing: Natural Language Processing","Machine Learning: Relational Learning","Natural Language Processing: Natural Language Semantics"],"videos":[],"references":[{"position":1,"raw":"[Bond and Foster, 2013] Francis Bond and Ryan Foster. Linking and extending an open multilingual Wordnet. In ACL, pages 1352-1362, 2013.","title":"Francis Bond and Ryan Foster. Linking and extending an open multilingual Wordnet","authors":"Foster Bond","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes et al., 2011] Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. Learning structured em- beddings of knowledge bases. In AAAI, 2011.","title":"Learning structured embeddings of knowledge bases","authors":"Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bordes et al., 2012] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. Joint learning of words and meaning representations for open-text semantic parsing. In AISTATS, pages 127-135, 2012.","title":"Joint learning of words and meaning representations for open-text semantic parsing","authors":"Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio","year":"2012","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bordes et al., 2014] Antoine Bordes, Jason Weston, and Nicolas Usunier. Open question answering with weakly supervised embedding models. In ECML-PKDD, pages 165-180, 2014.","title":"Open question answering with weakly supervised embedding models","authors":"Antoine Bordes, Jason Weston, Nicolas Usunier","year":"2014","venue":"ECML-PKDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2017] Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. Multi-lingual knowledge graph embeddings for cross-lingual knowledge alignment. arXiv preprint arXiv:1611.03954, 2017.","title":"Multi-lingual knowledge graph embeddings for cross-lingual knowledge alignment","authors":"Chen","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Collobert and Weston, 2008] Ronan Collobert and Jason Weston. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In ICML, pages 160-167, 2008.","title":"A unified architecture for natural language processing: Deep neural networks with multitask learning","authors":"Weston ; Collobert, Ronan Collobert, Jason Weston","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Culotta and Sorensen, 2004] Aron Culotta and Jeffrey Sorensen. Dependency tree kernels for relation extraction. In ACL, page 423, 2004.","title":"Dependency tree kernels for relation extraction","authors":"Sorensen Culotta, Aron Culotta, Jeffrey Sorensen","year":"2004","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Faruqui and Dyer, 2014] Manaal Faruqui and Chris Dyer. Improving vector space word representations using mul- tilingual correlation. EACL, 2014.","title":"Improving vector space word representations using multilingual correlation","authors":"Dyer Faruqui, Manaal Faruqui, Chris Dyer","year":"2014","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jenatton et al., 2012] Rodolphe Jenatton, Nicolas L Roux, Antoine Bordes, and Guillaume R Obozinski. A laten- t factor model for highly multi-relational data. In NIPS, 2012.","title":"A latent factor model for highly multi-relational data","authors":"Rodolphe Jenatton, Nicolas L Roux, Antoine Bordes, Guillaume R Obozinski","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ji et al., 2015] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dy- namic mapping matrix. In ACL, pages 687-696, 2015.","title":"Knowledge graph embedding via dynamic mapping matrix","authors":"Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, Jun Zhao","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jia et al., 2016] Yantao Jia, Yuanzhuo Wang, Hailun Lin, X- iaolong Jin, and Xueqi Cheng. Locally adaptive translation for knowledge graph embedding. In AAAI, 2016.","title":"Locally adaptive translation for knowledge graph embedding","authors":"Yantao Jia, Yuanzhuo Wang, Hailun Lin, Xiaolong Jin, Xueqi Cheng","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lehmann et al., 2015] Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, et al. Dbpedia-a large-scale, multi- lingual knowledge base extracted from Wikipedia. Seman- tic Web, 6(2):167-195, 2015.","title":"Dbpedia-a large-scale, multilingual knowledge base extracted from Wikipedia","authors":"Lehmann","year":"2015","venue":"Semantic Web","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2015] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In AAAI, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mahdisoltani et al., 2015] Farzaneh Mahdisoltani, Joanna Biega, Fabian Suchanek, et al. Yago3: A knowledge base from multilingual Wikipedias. In CIDR, 2015.","title":"Yago3: A knowledge base from multilingual Wikipedias","authors":"Farzaneh Mahdisoltani, Joanna Biega, Fabian Suchanek","year":"2015","venue":"CIDR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Quoc V Le, and Ilya Sutskever. Exploiting similarities among languages for machine translation. arXiv, 2013.","title":"and Ilya Sutskever. Exploiting similarities among languages for machine translation","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nguyen et al., 2011] Thanh Nguyen, Viviane Moreira, Huong Nguyen, Hoa Nguyen, and Juliana Freire. Multilin- gual schema matching for Wikipedia infoboxes. PVLDB, 5(2):133-144, 2011.","title":"Multilingual schema matching for Wikipedia infoboxes","authors":"Thanh Nguyen, Viviane Moreira, Huong Nguyen, Hoa Nguyen, Juliana Freire","year":"2011","venue":"PVLDB","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nguyen et al., 2016] Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. Stranse: a novel embedding model of entities and relationships in knowledge bases. In NAA- CL HLT, pages 460-466, 2016.","title":"Stranse: a novel embedding model of entities and relationships in knowledge bases","authors":"Nguyen","year":"2016","venue":"NAA-CL HLT","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nickel et al., 2016] Maximilian Nickel, Lorenzo Rosasco, Tomaso Poggio, et al. Holographic embeddings of knowl- edge graphs. In AAAI, 2016.","title":"Holographic embeddings of knowledge graphs","authors":"Maximilian Nickel, Lorenzo Rosasco, Tomaso Poggio","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rinser et al., 2013] Daniel Rinser, Dustin Lange, Felix Naumann, and Gerhard Weikum. Cross-lingual entity matching and infobox alignment in Wikipedia. Informa- tion Systems, 38(6):887-907, 2013.","title":"Cross-lingual entity matching and infobox alignment in Wikipedia","authors":"Daniel Rinser, Dustin Lange, Felix Naumann, Gerhard Weikum","year":"2013","venue":"Information Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Saxe et al., 2014] Andrew M Saxe, James L McClelland, and Surya Ganguli. Exact solutions to the nonlinear dy- namics of learning in deep linear neural networks. ICLR, 2014.","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","authors":"James L Andrew M Saxe, Surya Mcclelland, Ganguli","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Socher et al., 2013] Richard Socher, Danqi Chen, Christo- pher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. In NIPS, pages 926-934, 2013.","title":"Reasoning with neural tensor networks for knowledge base completion","authors":"Richard Socher, Danqi Chen, Christopher D Manning, Andrew Ng","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Speer and Havasi, 2013] Robert Speer and Catherine Havasi. Conceptnet 5: A large semantic network for relational knowledge. The People's Web Meets NLP, pages 161-176, 2013.","title":"Conceptnet 5: A large semantic network for relational knowledge","authors":"Havasi Speer, Robert Speer, Catherine Havasi","year":"2013","venue":"The People's Web Meets NLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Suchanek et al., 2011] Fabian M Suchanek, Serge Abite- boul, Pierre Senellart, and Tom Mitchell. Paris: Prob- abilistic alignment of relations, instances, and schema. PVLDB, 5(3):157-168, 2011.","title":"Paris: Probabilistic alignment of relations, instances, and schema","authors":"Serge Fabian M Suchanek, Pierre Abiteboul, Tom Senellart, Mitchell","year":"2011","venue":"PVLDB","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun et al., 2011] Ang Sun, Ralph Grishman, and Satoshi Sekine. Semi-supervised relation extraction with large- scale word clustering. In ACL, pages 521-529, 2011.","title":"Semi-supervised relation extraction with largescale word clustering","authors":"Ang Sun, Ralph Grishman, Satoshi Sekine","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Denny Vrandečić. Wikidata: A new plat- form for collaborative data collection. In WWW, 2012.","title":"Wikidata: A new platform for collaborative data collection","authors":"Denny Vrandečić","year":"2012","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In AAAI, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Weston et al., 2013] Jason Weston, Antoine Bordes, Ok- sana Yakhnenko, and Nicolas Usunier. Connecting lan- guage and knowledge bases with embedding models for relation extraction. In EMNLP, pages 1366-1371, 2013. [Wikipedia, 2017] Wikipedia, 2017. https://www. wikipedia.org/.","title":"Connecting language and knowledge bases with embedding models for relation extraction","authors":"Jason Weston, Antoine Bordes, Oksana Yakhnenko, Nicolas Usunier","year":"2013","venue":"EMNLP","doi":null,"url":"https://www.wikipedia.org/","internal_url":null},{"position":29,"raw":"[Wilson and Martinez, 2003] D Randall Wilson and Tony R Martinez. The general inefficiency of batch training for gradient descent learning. Neural Networks, 16(10), 2003.","title":"The general inefficiency of batch training for gradient descent learning","authors":"Randall Wilson, Tony R Martinez","year":"2003","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xing et al., 2015] Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. Normalized word embedding and orthogonal transform for bilingual word translation. In NAACL HLT, pages 1006-1011, 2015.","title":"Normalized word embedding and orthogonal transform for bilingual word translation","authors":"Chao Xing, Dong Wang, Chao Liu, Yiye Lin","year":"2015","venue":"NAACL HLT","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2015a] Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Chang. Network representa- tion learning with rich text information. In IJCAI, 2015.","title":"Network representation learning with rich text information","authors":"Yang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/299"},{"position":32,"raw":"[Yang et al., 2015b] Yang Yang, Yizhou Sun, Jie Tang, Bo Ma, and Juanzi Li. Entity matching across heteroge- neous sources. In KDD, pages 1395-1404, 2015.","title":"Entity matching across heterogeneous sources","authors":"Yang","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhong et al., 2015] Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, and Zheng Chen. Aligning knowledge and text embeddings by entity descriptions. In EMNLP, pages 267-272, 2015.","title":"Aligning knowledge and text embeddings by entity descriptions","authors":"Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, Zheng Chen","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhou et al., 2005] Guodong Zhou, Jian Su, Jie Zhang, and Min Zhang. Exploring various knowledge in relation ex- traction. In ACL, pages 427-434, 2005.","title":"Exploring various knowledge in relation extraction","authors":"Guodong Zhou, Jian Su, Jie Zhang, Min Zhang","year":"2005","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}