{"type":"ScholarlyArticle","title":"Improving Cross-lingual Entity Alignment via Optimal Transport","authors":[{"name":"Shichao Pei","slug":"shichao-pei","given":"Shichao","family":"Pei","affiliation":"The Computer , Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology (KAUST) , Thuwal , 23955 , SA"},{"name":"Lu Yu","slug":"lu-yu","given":"Lu","family":"Yu","affiliation":"The Computer , Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology (KAUST) , Thuwal , 23955 , SA"},{"name":"Xiangliang Zhang","slug":"xiangliang-zhang","given":"Xiangliang","family":"Zhang","affiliation":"The Computer , Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology (KAUST) , Thuwal , 23955 , SA"}],"abstract":"Cross-lingual entity alignment identifies entity pairs that share the same meanings but locate in different language knowledge graphs (KGs). The study in this paper is to address two limitations that widely exist in current solutions:  1) the alignment loss functions defined at the entity level serve well the purpose of aligning labeled entities but fail to match the whole picture of labeled and unlabeled entities in different KGs;  2) the translation from one domain to the other has been considered (e.g., X to Y by M1 or Y to X by M2). However, the important duality of alignment between different KGs  (X to Y by M1 and Y to X by M2) is ignored. We propose a novel entity alignment framework (OTEA), which dually optimizes the entity-level loss and group-level loss via optimal transport theory. We also impose a regularizer on the dual translation matrices to mitigate the effect of noise during transformation. Extensive experimental results show that our model consistently outperforms the state-of-the-arts with significant improvements on alignment accuracy.","doi":"10.24963/ijcai.2019/448","url":"https://ijcai.org/proceedings/2019/448","pdf":"https://ijcai.org/proceedings/2019/0448.pdf","markdown":"https://ijcai.org/proceedings/2019/448.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/448","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":448,"pages":{"first":3231,"last":3237},"keywords":["Machine Learning: Semi-Supervised Learning","Machine Learning: Data Mining","Machine Learning: Knowledge-based Learning","Machine Learning: Relational Learning"],"videos":[],"references":[{"position":1,"raw":"WK31-15K dataset Language En-Fr Fr-En En-De De-En Metric Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR MTransE 16.77 21.64 0.198 19.85 31.27 0.261 6.17 8.48 0.078 4.69 6.61 0.059 ITransE 18.21 24.34 0.214 18.61 33.64 0.248 15.98 28.63 0.218 13.42 25.63 0.205 JAPE 15.68 23.45 0.208 16.22 28.93 0.219 16.85 27.32 0.226 13.92 22.15 0.189 GCN 17.24 27.29 0.220 17.58 30.82 0.237 18.25 31.30 0.248 15.70 27.53 0.217 OTEA w/o reg 34.06 53.74 0.432 36.96 57.09 0.457 32.56 53.68 0.421 31.86 47.61 0.394 OTEA w/o dual 32.49 52.43 0.411 34.84 55.21 0.443 31.21 52.78 0.409 31.54 45.86 0.378 BootEA 29.72 52.92 0.395 30.77 55.44 0.428 33.13 54.13 0.435 30.47 45.33 0.381 OTEA 37.53 57.74 0.472 40.47 60.90 0.502 37.41 57.19 0.470 36.80 56.79 0.465 Improv. % 26.28 9.11 19.50 31.52 9.85 17.30 12.92 5.65 8.04 20.77 25.28 22.05 WK31-60K dataset Language En-Fr Fr-En En-De De-En Metric Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR MTransE 12.05 14.94 0.141 13.95 20.25 0.177 0.86 2.15 0.017 3.37 10.07 0.072 ITransE 17.27 25.31 0.192 18.72 32.96 0.263 15.53 24.61 0.204 16.74 24.96 0.215 JAPE 15.32 27.63 0.226 16.85 35.41 0.271 12.52 22.74 0.171 14.71 23.86 0.192 GCN 18.35 32.35 0.250 21.47 37.81 0.293 14.48 23.54 0.189 13.8 24.55 0.190 OTEA w/o reg 31.41 48.23 0.412 34.18 49.48 0.428 23.59 35.78 0.286 22.46 38.94 0.306 OTEA w/o dual 31.35 47.47 0.398 32.98 48.51 0.413 22.90 35.27 0.282 21.07 36.99 0.287 BootEA 30.82 49.42 0.406 33.31 51.14 0.425 24.45 37.63 0.308 23.28 39.29 0.316 OTEA 34.47 51.51 0.428 36.07 54.08 0.447 27.05 42.12 0.345 26.97 43.97 0.352 Improv. % 11.84 4.23 5.42 8.28 5.75 5.17 10.63 11.93 12.01 15.85 11.91 11.39 WK31-120K dataset Language En-Fr Fr-En En-De De-En Metric Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR MTransE 21.01 22.24 0.217 21.11 23.63 0.227 5.38 6.53 0.062 4.97 7.39 0.066 ITransE 11.54 20.41 0.176 13.35 21.20 0.197 7.62 15.54 0.112 6.41 12.82 0.085 JAPE 6.98 16.10 0.127 8.64 17.85 0.134 4.37 12.91 0.076 5.23 10.46 0.071 GCN 9.32 18.62 0.146 10.81 18.22 0.153 6.32 15.14 0.109 5.91 13.85 0.092 OTEA w/o reg 24.71 33.84 0.302 25.13 33.32 0.291 14.81 26.60 0.217 14.06 26.43 0.214 OTEA w/o dual 23.06 33.17 0.287 23.58 33.50 0.289 14.40 26.15 0.208 13.14 25.42 0.395 BootEA 17.56 27.41 0.235 18.46 28.65 0.241 11.57 22.08 0.179 10.32 22.11 0.169 OTEA 27.92 37.33 0.328 28.07 37.41 0.332 17.98 30.41 0.244 17.00 29.46 0.235 Improv. % 59.00 36.20 39.57 52.06 30.58 37.76 55.40 37.73 36.31 64.73 33.24 39.05","title":"K dataset Language En-Fr Fr-En En-De De-En Metric Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5 MRR Hits@1 Hits@5","authors":null,"year":null,"venue":"MRR MTransE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In ICML, 2017.","title":"Wasserstein generative adversarial networks","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2016] Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. arXiv preprint arXiv:1611.03954, 2016.","title":"Multilingual knowledge graph embeddings for cross-lingual knowledge alignment","authors":"Chen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/209"},{"position":5,"raw":"[Chen et al., 2018a] Liqun Chen, Shuyang Dai, Chenyang Tao, Haichao Zhang, Zhe Gan, Dinghan Shen, Yizhe Zhang, Guoyin Wang, Ruiyi Zhang, and Lawrence Carin. Adversarial text generation via feature-mover's distance. In NIPS, 2018.","title":"Adversarial text generation via feature-mover's distance","authors":"Chen","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2018b] Muhao Chen, Yingtao Tian, Kai-Wei Chang, Steven Skiena, and Carlo Zaniolo. Co-training em- beddings of knowledge graphs and entity descriptions for cross-lingual entity alignment. IJCAI, 2018.","title":"Co-training embeddings of knowledge graphs and entity descriptions for cross-lingual entity alignment","authors":"Chen","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/556"},{"position":7,"raw":"[Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Light- speed computation of optimal transport. In NIPS, 2013.","title":"Sinkhorn distances: Lightspeed computation of optimal transport","authors":"Marco Cuturi","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NIPS, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hinton et al., 2012] Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Neural networks for machine learning lec- ture 6a overview of mini-batch gradient descent. Cited on, 14, 2012.","title":"Neural networks for machine learning lecture 6a overview of mini-batch gradient descent","authors":"Geoffrey Hinton, Nitish Srivastava, Kevin Swersky","year":"2012","venue":"Cited on","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2011] Wei Hu, Jianfeng Chen, and Yuzhong Qu. A self-training approach for resolving object coreference on the semantic web. In WWW, 2011.","title":"A self-training approach for resolving object coreference on the semantic web","authors":"Wei Hu, Jianfeng Chen, Yuzhong Qu","year":"2011","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ji et al., 2015] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dy- namic mapping matrix. In ACL, 2015.","title":"Knowledge graph embedding via dynamic mapping matrix","authors":"Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, Jun Zhao","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2015] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In AAAI, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Lin","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luise et al., 2018] Giulia Luise, Alessandro Rudi, Massim- iliano Pontil, and Carlo Ciliberto. Differential properties of sinkhorn approximation for learning with wasserstein distance. In NIPS, 2018.","title":"Differential properties of sinkhorn approximation for learning with wasserstein distance","authors":"Giulia Luise, Alessandro Rudi, Massimiliano Pontil, Carlo Ciliberto","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mahdisoltani et al., 2013] Farzaneh Mahdisoltani, Joanna Biega, and Fabian M Suchanek. Yago3: A knowledge base from multilingual wikipedias. In CIDR, 2013.","title":"Yago3: A knowledge base from multilingual wikipedias","authors":"Farzaneh Mahdisoltani, Joanna Biega, Fabian M Suchanek","year":"2013","venue":"CIDR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nentwig et al., 2017] Markus Nentwig, Michael Hartung, Axel-Cyrille Ngonga Ngomo, and Erhard Rahm. A sur- vey of current link discovery frameworks. Semantic Web, 8(3):419-436, 2017.","title":"A survey of current link discovery frameworks","authors":"Nentwig","year":"2017","venue":"Semantic Web","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ng and Cardie, 2002] Vincent Ng and Claire Cardie. Im- proving machine learning approaches to coreference reso- lution. In ACL, 2002.","title":"Improving machine learning approaches to coreference resolution","authors":"Cardie Ng, Vincent Ng, Claire Cardie","year":"2002","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nguyen et al., 2011] Thanh Nguyen, Viviane Moreira, Huong Nguyen, Hoa Nguyen, and Juliana Freire. Mul- tilingual schema matching for wikipedia infoboxes. Pro- ceedings of the VLDB Endowment, 5(2):133-144, 2011.","title":"Multilingual schema matching for wikipedia infoboxes","authors":"Thanh Nguyen, Viviane Moreira, Huong Nguyen, Hoa Nguyen, Juliana Freire","year":"2011","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. Efficient and robust feature selection via joint l2, 1-norms minimization. In NIPS, 2010.","title":"Efficient and robust feature selection via joint l2, 1-norms minimization","authors":"Feiping Nie, Heng Huang, Xiao Cai, Chris H Ding","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Peyré and Cuturi, 2018] Gabriel Peyré and Marco Cu- turi. Computational optimal transport. arXiv preprint arXiv:1803.00567, 2018.","title":"Computational optimal transport","authors":"Gabriel Peyré, Marco Cuturi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Peyré et al., 2012] Gabriel Peyré, Jalal Fadili, and Julien Rabin. Wasserstein active contours. In ICIP, 2012.","title":"Wasserstein active contours","authors":"Gabriel Peyré, Jalal Fadili, Julien Rabin","year":"2012","venue":"ICIP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Smith et al., 2017] Samuel L Smith, David HP Turban, Steven Hamblin, and Nils Y Hammerla. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. ICLR, 2017.","title":"Offline bilingual word vectors, orthogonal transformations and the inverted softmax","authors":"L Samuel, David Smith, Steven Hp Turban, Nils Y Hamblin, Hammerla","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sun et al., 2017] Zequn Sun, Wei Hu, and Chengkai Li. Cross-lingual entity alignment via joint attribute- preserving embedding. In ISWC, 2017.","title":"Cross-lingual entity alignment via joint attributepreserving embedding","authors":"Sun","year":"2017","venue":"ISWC","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sun et al., 2018] Zequn Sun, Wei Hu, Qingheng Zhang, and Yuzhong Qu. Bootstrapping entity alignment with knowl- edge graph embedding. In IJCAI, 2018.","title":"Bootstrapping entity alignment with knowledge graph embedding","authors":"Zequn Sun, Wei Hu, Qingheng Zhang, Yuzhong Qu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/611"},{"position":25,"raw":"[Villani, 2008] Cédric Villani. Optimal transport: old and new. Springer Science & Business Media, 2008.","title":"Optimal transport: old and new","authors":"Cédric Villani","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vrandečić and Krötzsch, 2014] Denny Vrandečić and Markus Krötzsch. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 2014.","title":"Wikidata: a free collaborative knowledgebase","authors":"Krötzsch Vrandečić, Denny Vrandečić, Markus Krötzsch","year":"2014","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In AAAI, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2018] Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. Cross-lingual knowledge graph align- ment via graph convolutional networks. In EMNLP, 2018.","title":"Cross-lingual knowledge graph alignment via graph convolutional networks","authors":"Zhichun Wang, Qingsong Lv, Xiaohan Lan, Yu Zhang","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xu et al., 2018] Hongteng Xu, Wenlin Wang, Wei Liu, and Lawrence Carin. Distilled wasserstein learning for word embedding and topic modeling. In NIPS, 2018.","title":"Distilled wasserstein learning for word embedding and topic modeling","authors":"Xu","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yang et al., 2014] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575, 2014.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, Li Deng","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2015] Yang Yang, Yizhou Sun, Jie Tang, Bo Ma, and Juanzi Li. Entity matching across heteroge- neous sources. In KDD, 2015.","title":"Entity matching across heterogeneous sources","authors":"Yang Yang, Yizhou Sun, Jie Tang, Bo Ma, Juanzi Li","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yin et al., 2015] Penghang Yin, Yifei Lou, Qi He, and Jack Xin. Minimization of 1-2 for compressed sensing. SIAM Journal on Scientific Computing, 2015.","title":"Minimization of 1-2 for compressed sensing","authors":"Yin","year":"2015","venue":"SIAM Journal on Scientific Computing","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhu et al., 2017] Hao Zhu, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. Iterative entity alignment via joint knowl- edge embeddings. In IJCAI, 2017.","title":"Iterative entity alignment via joint knowledge embeddings","authors":"Ruobing Hao Zhu, Zhiyuan Xie, Maosong Liu, Sun","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/595"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}