{"type":"ScholarlyArticle","title":"Translating Embeddings for Knowledge Graph Completion with Relation Attention Mechanism","authors":[{"name":"Wei Qian","slug":"wei-qian","given":"Wei","family":"Qian","affiliation":"† State Key Lab of CAD&CG , College of Computer Science , Zhejiang University , China"},{"name":"Cong Fu","slug":"cong-fu","given":"Cong","family":"Fu","affiliation":"† State Key Lab of CAD&CG , College of Computer Science , Zhejiang University , China"},{"name":"Yu Zhu","slug":"yu-zhu","given":"Yu","family":"Zhu","affiliation":"† State Key Lab of CAD&CG , College of Computer Science , Zhejiang University , China; Alibaba-Zhejiang University Joint Institute of Frontier Technologies Fabu Inc. , Hangzhou , China"},{"name":"Deng Cai","slug":"deng-cai","given":"Deng","family":"Cai","affiliation":"† State Key Lab of CAD&CG , College of Computer Science , Zhejiang University , China"},{"name":"Xiaofei He","slug":"xiaofei-he","given":"Xiaofei","family":"He","affiliation":"† State Key Lab of CAD&CG , College of Computer Science , Zhejiang University , China"}],"abstract":"Knowledge graph embedding is an essential problem in knowledge extraction. Recently, translation based embedding models (e.g., TransE) have received increasingly attentions. These methods try to interpret the relations among entities as translations from head entity to tail entity and achieve promising performance on knowledge graph completion. Previous researchers attempt to transform the entity embedding concerning the given relation for distinguishability. Also, they naturally think the relation-related transforming should reflect attention mechanism, which means it should focus on only a part of the attributes. However, we found previous methods are failed with creating attention mechanism, and the reason is that they ignore the hierarchical routine of human cognition. When predicting whether a relation holds between two entities, people first check the category of entities, then they focus on fined-grained relation-related attributes to make the decision. In other words, the attention should take effect on entities filtered by the right category. In this paper, we propose a novel knowledge graph embedding method named TransAt to learn the translation based embedding, relation-related categories of entities and relation-related attention simultaneously. Extensive experiments show that our approach outperforms state-of-the-art methods significantly on public datasets, and our method can learn the true attention varying among relations.","doi":"10.24963/ijcai.2018/596","url":"https://ijcai.org/proceedings/2018/596","pdf":"https://ijcai.org/proceedings/2018/0596.pdf","markdown":"https://ijcai.org/proceedings/2018/596.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/596","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":596,"pages":{"first":4286,"last":4292},"keywords":["Natural Language Processing: Knowledge Extraction","Natural Language Processing: Embeddings"],"videos":[],"references":[{"position":1,"raw":"[Bakker, 1987] René Ronald Bakker. Knowledge Graph- s: representation and structuring of scientific knowledge. 1987.","title":"Knowledge Graphs: representation and structuring of scientific knowledge","authors":"; Bakker, Ronald René, Bakker","year":"1987","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bollacker et al., 2008] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD, pages 1247-1250. AcM, 2008.","title":"Freebase: a collaboratively created graph database for structuring human knowledge","authors":"Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, Jamie Taylor","year":"2008","venue":"SIGMOD","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bordes et al., 2011] Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio, et al. Learning structured em- beddings of knowledge bases. In AAAI, volume 6, page 6, 2011.","title":"Learning structured embeddings of knowledge bases","authors":"Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bordes et al., 2012] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. Joint learning of words and meaning representations for open-text semantic parsing. In AISTATS, pages 127-135, 2012.","title":"Joint learning of words and meaning representations for open-text semantic parsing","authors":"Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio","year":"2012","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bordes et al., 2014] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. A semantic matching energy function for learning with multi-relational data. Machine Learning, 94(2):233-259, 2014.","title":"A semantic matching energy function for learning with multi-relational data","authors":"Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bottou, 2010] Léon Bottou. Large-scale machine learning with stochastic gradient descent. In COMPSTAT, pages 177-186. Springer, 2010.","title":"Large-scale machine learning with stochastic gradient descent","authors":"Léon Bottou, Bottou","year":"2010","venue":"COMPSTAT","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dunteman, 1989] George H Dunteman. Principal compo- nents analysis. Number 69. Sage, 1989.","title":"Principal components analysis","authors":"George H Dunteman","year":"1989","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Feng et al., 2016] Jun Feng, Minlie Huang, Yang Yang, et al. Gake: Graph aware knowledge embedding. In COL- ING, pages 641-651, 2016.","title":"Gake: Graph aware knowledge embedding","authors":"Jun Feng, Minlie Huang, Yang Yang","year":"2016","venue":"COL-ING","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guan et al., 2015] Ziyu Guan, Shengqi Yang, Huan Sun, Mudhakar Srivatsa, and Xifeng Yan. Fine-grained knowl- edge sharing in collaborative environments. TKDE, 27(8):2163-2174, 2015.","title":"Fine-grained knowledge sharing in collaborative environments","authors":"Shengqi Ziyu Guan, Huan Yang, Mudhakar Sun, Xifeng Srivatsa, Yan","year":"2015","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2015a] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpass- ing human-level performance on imagenet classification. In ICCV, pages 1026-1034, 2015.","title":"Delving deep into rectifiers: Surpassing human-level performance on imagenet classification","authors":"He","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2015b] Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. Learning to represent knowledge graphs with gaussian embedding. In CIKM, pages 623-632. ACM, 2015.","title":"Learning to represent knowledge graphs with gaussian embedding","authors":"He","year":"2015","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jenatton et al., 2012] Rodolphe Jenatton, Nicolas L Roux, Antoine Bordes, and Guillaume R Obozinski. A laten- t factor model for highly multi-relational data. In NIPS, pages 3167-3175, 2012.","title":"A latent factor model for highly multi-relational data","authors":"Rodolphe Jenatton, Nicolas L Roux, Antoine Bordes, Guillaume R Obozinski","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ji et al., 2015] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dy- namic mapping matrix. In ACL, pages 687-696, 2015.","title":"Knowledge graph embedding via dynamic mapping matrix","authors":"Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, Jun Zhao","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ji et al., 2016] Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. Knowledge graph completion with adaptive sparse transfer matrix. In AAAI, pages 985-991, 2016.","title":"Knowledge graph completion with adaptive sparse transfer matrix","authors":"Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kanojia et al., 2017] Vibhor Kanojia, Hideyuki Maeda, Riku Togashi, and Sumio Fujita. Enhancing knowledge graph embedding with probabilistic negative sampling. In WWW, pages 801-802. International World Wide Web Conferences Steering Committee, 2017.","title":"Enhancing knowledge graph embedding with probabilistic negative sampling","authors":"Vibhor Kanojia, Hideyuki Maeda, Riku Togashi, Sumio Fujita","year":"2017","venue":"International World Wide Web Conferences Steering Committee","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2015] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation em- beddings for knowledge graph completion. In AAAI, pages 2181-2187, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Miller, 1995] George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39-41, 1995.","title":"Wordnet: a lexical database for english","authors":"George A Miller","year":"1995","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pedregosa et al., 2011] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. Journal of Machine Learning Re- search, 12(Oct):2825-2830, 2011.","title":"Scikit-learn: Machine learning in python","authors":"Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Socher et al., 2013] Richard Socher, Danqi Chen, Christo- pher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. In NIPS, pages 926-934, 2013.","title":"Reasoning with neural tensor networks for knowledge base completion","authors":"Richard Socher, Danqi Chen, Christopher D Manning, Andrew Ng","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In AAAI, pages 1112-1119, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2017] Chao-Yuan Wu, R Manmatha, Alexander J Smola, and Philipp Krähenbühl. Sampling matters in deep embedding learning. arXiv, 2017.","title":"Sampling matters in deep embedding learning","authors":"Chao-Yuan, R Wu, Alexander J Manmatha, Philipp Smola, Krähenbühl","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang, 2017] Wen Zhang. Knowledge graph embedding with diversity of structures. In WWW, pages 747-753. In- ternational World Wide Web Conferences Steering Com- mittee, 2017.","title":"Knowledge graph embedding with diversity of structures","authors":"Wen Zhang","year":"2017","venue":"ternational World Wide Web Conferences Steering Committee","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu et al., 2016] Yu Zhu, Ziyu Guan, Shulong Tan, Haifeng Liu, Deng Cai, and Xiaofei He. Heterogeneous hyper- graph embedding for document recommendation. Neuro- computing, 216:150-162, 2016.","title":"Heterogeneous hypergraph embedding for document recommendation","authors":"Yu Zhu, Ziyu Guan, Shulong Tan, Haifeng Liu, Deng Cai, Xiaofei He","year":"2016","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2017] Yu Zhu, Hao Li, Yikang Liao, Beidou Wang, Ziyu Guan, Haifeng Liu, and Deng Cai. What to do next: modeling user behaviors by time-lstm. In IJCAI, pages 3602-3608, 2017.","title":"What to do next: modeling user behaviors by time-lstm","authors":"Yu Zhu, Hao Li, Yikang Liao, Beidou Wang, Ziyu Guan, Haifeng Liu, Deng Cai","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/504"},{"position":27,"raw":"[Zou et al., 2013] Will Y Zou, Richard Socher, Daniel Cer, and Christopher D Manning. Bilingual word embeddings for phrase-based machine translation. In EMNLP, pages 1393-1398, 2013.","title":"Bilingual word embeddings for phrase-based machine translation","authors":"Will Y Zou, Richard Socher, Daniel Cer, Christopher D Manning","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}