{"type":"ScholarlyArticle","title":"Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment","authors":[{"name":"Muhao Chen","slug":"muhao-chen","given":"Muhao","family":"Chen","affiliation":"Department of Computer Science , University of California , Los Angeles"},{"name":"Yingtao Tian","slug":"yingtao-tian","given":"Yingtao","family":"Tian","affiliation":"Department of Computer Science , Stony Brook University"},{"name":"Kai-Wei Chang","slug":"kai-wei-chang","given":"Kai-Wei","family":"Chang","affiliation":"Department of Computer Science , University of California , Los Angeles"},{"name":"Steven Skiena","slug":"steven-skiena","given":"Steven","family":"Skiena","affiliation":"Department of Computer Science , Stony Brook University"},{"name":"Carlo Zaniolo","slug":"carlo-zaniolo","given":"Carlo","family":"Zaniolo","affiliation":"Department of Computer Science , University of California , Los Angeles"}],"abstract":"Multilingual knowledge graph (KG) embeddings provide latent semantic representations of entities and structured knowledge with cross-lingual inferences, which benefit various knowledge-driven cross-lingual NLP tasks. However, precisely learning such cross-lingual inferences is usually hindered by the low coverage of entity alignment in many KGs. Since many multilingual KGs also provide literal descriptions of entities, in this paper, we introduce an embedding-based approach which leverages a weakly aligned multilingual KG for semi-supervised cross-lingual learning using entity descriptions. Our approach performs co-training of two embedding models, i.e. a multilingual KG embedding model and a multilingual literal description embedding model. The models are trained on a large Wikipedia-based trilingual dataset where most entity alignment is unknown to training. Experimental results show that the performance of the proposed approach on the entity alignment task improves at each iteration of co-training, and eventually reaches a stage at which it significantly surpasses previous approaches. We also show that our approach has promising abilities for zero-shot entity alignment, and cross-lingual KG completion.","doi":"10.24963/ijcai.2018/556","url":"https://ijcai.org/proceedings/2018/556","pdf":"https://ijcai.org/proceedings/2018/0556.pdf","markdown":"https://ijcai.org/proceedings/2018/556.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/556","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":556,"pages":{"first":3998,"last":4004},"keywords":["Machine Learning: Relational Learning","Machine Learning: Knowledge-based Learning","Machine Learning: Semi-Supervised Learning","Natural Language Processing: Embeddings"],"videos":[],"references":[{"position":1,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, et al. Translating embeddings for modeling multi-relational data. In NIPS, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Bordes","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes et al., 2014] Antoine Bordes, Jason Weston, et al. Open question answering with weakly supervised embed- ding models. In ECML-PKDD, 2014.","title":"Open question answering with weakly supervised embedding models","authors":"Antoine Bordes, Jason Weston","year":"2014","venue":"ECML-PKDD","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Guestrin, 2016] Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In KDD, 2016.","title":"Xgboost: A scalable tree boosting system","authors":"Tianqi Chen, Carlos Guestrin","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017a] Muhao Chen, Yingtao Tian, et al. Mul- tilingual knowledge graph embeddings for cross-lingual knowledge alignment. In IJCAI, 2017.","title":"Multilingual knowledge graph embeddings for cross-lingual knowledge alignment","authors":"Chen","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/209"},{"position":5,"raw":"[Chen et al., 2017b] Muhao Chen, Tao Zhou, et al. Multi- graph affinity embeddings for multilingual knowledge graphs. In AKBC, 2017.","title":"Multigraph affinity embeddings for multilingual knowledge graphs","authors":"Chen","year":"2017","venue":"AKBC","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2017c] Ting Chen, Yizhou Sun, et al. On sam- pling strategies for neural network-based collaborative fil- tering. In KDD, 2017.","title":"On sampling strategies for neural network-based collaborative filtering","authors":"Chen","year":"2017","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2018] Muhao Chen, Yingtao Tian, et al. On2vec: Embedding-based relation prediction for ontol- ogy population. In SDM, 2018.","title":"On2vec: Embedding-based relation prediction for ontology population","authors":"Chen","year":"2018","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dettmers et al., 2018] Tim Dettmers, Pasquale Minervini, et al. Convolutional 2d knowledge graph embeddings. In AAAI, 2018.","title":"Convolutional 2d knowledge graph embeddings","authors":"Tim Dettmers, Pasquale Minervini","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2017] Yuan Fang, Kingsley Kuan, et al. Object detection meets knowledge graphs. In IJCAI, 2017.","title":"Object detection meets knowledge graphs","authors":"Yuan Fang, Kingsley Kuan","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/230"},{"position":10,"raw":"[Faruqui and Dyer, 2014] Manaal Faruqui and Chris Dyer. Improving vector space word representations using mul- tilingual correlation. EACL, 2014.","title":"Improving vector space word representations using multilingual correlation","authors":"Dyer Faruqui, Manaal Faruqui, Chris Dyer","year":"2014","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gouws et al., 2015] Stephan Gouws, Yoshua Bengio, et al. Bilbowa: Fast bilingual distributed representations with- out word alignments. In ICML, 2015.","title":"Bilbowa: Fast bilingual distributed representations without word alignments","authors":"Stephan Gouws, Yoshua Bengio","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2017] He He, Anusha Balakrishnan, et al. Learn- ing symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. In ACL, 2017.","title":"Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings","authors":"He","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ji et al., 2015] Guoliang Ji, Shizhu He, et al. Knowledge graph embedding via dynamic matrix. In ACL, 2015.","title":"Knowledge graph embedding via dynamic matrix","authors":"Guoliang Ji, Shizhu He","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jia et al., 2016] Yantao Jia, Yuanzhuo Wang, et al. Locally adaptive translation for knowledge graph embedding. In AAAI, 2016.","title":"Locally adaptive translation for knowledge graph embedding","authors":"Jia","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jozefowicz et al., 2015] Rafal Jozefowicz, Wojciech Zaremba, et al. An empirical exploration of recurrent network architectures. In ICML, 2015.","title":"An empirical exploration of recurrent network architectures","authors":"Rafal Jozefowicz, Wojciech Zaremba","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kim et al., 2017] Yoon Kim, Carl Denton, et al. Structured attention networks. In ICLR, 2017.","title":"Structured attention networks","authors":"Yoon Kim, Carl Denton","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Koehn, 2005] Philipp Koehn. Europarl: A parallel corpus for statistical machine translation. In MT summit, 2005.","title":"Europarl: A parallel corpus for statistical machine translation","authors":"Philipp Koehn","year":"2005","venue":"MT summit","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lehmann et al., 2015] Jens Lehmann, Robert Isele, et al. Dbpedia-a large-scale, multilingual knowledge base ex- tracted from Wikipedia. Semantic Web, 2015.","title":"Dbpedia-a large-scale, multilingual knowledge base extracted from Wikipedia","authors":"Lehmann","year":"2015","venue":"Semantic Web","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2015] Yankai Lin, Zhiyuan Liu, et al. Learning entity and relation embeddings for knowledge graph com- pletion. In AAAI, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Lin","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mahdisoltani et al., 2015] Farzaneh Mahdisoltani, Joanna Biega, et al. Yago3: A knowledge base from multilingual Wikipedias. In CIDR, 2015.","title":"Yago3: A knowledge base from multilingual Wikipedias","authors":"Farzaneh Mahdisoltani, Joanna Biega","year":"2015","venue":"CIDR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Quoc V Le, et al. Exploiting similarities among languages for machine translation. CoRR, 2013.","title":"Exploiting similarities among languages for machine translation","authors":"Mikolov","year":"2013","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, et al. Distributed representations of words and phrases and their compositionality. In NIPS, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nickel et al., 2016] Maximilian Nickel, Lorenzo Rosasco, et al. Holographic embeddings of knowledge graphs. In AAAI, 2016.","title":"Holographic embeddings of knowledge graphs","authors":"Maximilian Nickel, Lorenzo Rosasco","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Speer et al., 2017] Robert Speer, Joshua Chin, et al. Con- ceptnet 5.5: An open multilingual graph of general knowl- edge. In AAAI, 2017.","title":"Conceptnet 5.5: An open multilingual graph of general knowledge","authors":"Robert Speer, Joshua Chin","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun et al., 2017] Zequn Sun, Wei Hu, et al. Cross-lingual entity alignment via joint attribute-preserving embedding. In ISWC, 2017.","title":"Cross-lingual entity alignment via joint attribute-preserving embedding","authors":"Sun","year":"2017","venue":"ISWC","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Thi et al., 2016] Do Thi, Ngoc Quynh, et al. Facing the most difficult case of semantic role labeling: A collabo- ration of word embeddings and co-training. In ACL, 2016.","title":"Facing the most difficult case of semantic role labeling: A collaboration of word embeddings and co-training","authors":"Thi","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wan, 2009] Xiaojun Wan. Co-training for cross-lingual sen- timent classification. In ACL-IJCNLP, 2009.","title":"Co-training for cross-lingual sentiment classification","authors":"Xiaojun Wan","year":"2009","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, et al. Knowledge graph embedding by translating on hyper- planes. In AAAI, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Zhen Wang, Jianwen Zhang","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2016] Ruobing Xie, Zhiyuan Liu, et al. Rep- resentation learning of knowledge graphs with entity de- scriptions. In AAAI, 2016.","title":"Representation learning of knowledge graphs with entity descriptions","authors":"Xie","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xing et al., 2015] Chao Xing, Dong Wang, et al. Normal- ized word embedding and orthogonal transform for bilin- gual word translation. In NAACL, 2015.","title":"Normalized word embedding and orthogonal transform for bilingual word translation","authors":"Chao Xing, Dong Wang","year":"2015","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2015] Bishan Yang, Wen-tau Yih, et al. Em- bedding entities and relations for learning and inference in knowledge bases. In ICLR, 2015.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen-Tau Yih","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2014] Mi Zhang, Jie Tang, et al. Addressing cold start in recommender systems: A semi-supervised co- training algorithm. In SIGIR, 2014.","title":"Addressing cold start in recommender systems: A semi-supervised cotraining algorithm","authors":"Mi Zhang, Jie Tang","year":"2014","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhu et al., 2017] Hao Zhu, Ruobing Xie, et al. Iterative en- tity alignment via knowledge embeddings. In IJCAI, 2017.","title":"Iterative entity alignment via knowledge embeddings","authors":"Ruobing Hao Zhu, Xie","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}