{"type":"ScholarlyArticle","title":"TransMS: Knowledge Graph Embedding for Complex Relations by Multidirectional Semantics","authors":[{"name":"Shihui Yang","slug":"shihui-yang","given":"Shihui","family":"Yang","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; MoE Key Lab of Artificial Intelligence , AI Institute , Shanghai Jiao Tong University"},{"name":"Jidong Tian","slug":"jidong-tian","given":"Jidong","family":"Tian","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; MoE Key Lab of Artificial Intelligence , AI Institute , Shanghai Jiao Tong University"},{"name":"Honglun Zhang","slug":"honglun-zhang","given":"Honglun","family":"Zhang","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; MoE Key Lab of Artificial Intelligence , AI Institute , Shanghai Jiao Tong University"},{"name":"Junchi Yan","slug":"junchi-yan","given":"Junchi","family":"Yan","affiliation":"MoE Key Lab of Artificial Intelligence , AI Institute , Shanghai Jiao Tong University; Department of Computer Science and Engineering , Shanghai Jiao Tong University"},{"name":"Hao He","slug":"hao-he","given":"Hao","family":"He","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; MoE Key Lab of Artificial Intelligence , AI Institute , Shanghai Jiao Tong University"},{"name":"Yaohui Jin","slug":"yaohui-jin","given":"Yaohui","family":"Jin","affiliation":"State Key Lab of Advanced Optical Communication System and Network , Shanghai Jiao Tong University; MoE Key Lab of Artificial Intelligence , AI Institute , Shanghai Jiao Tong University"}],"abstract":"Knowledge graph embedding, which projects the symbolic relations and entities onto low-dimension continuous spaces, is essential to knowledge graph completion. Recently, translation-based embedding models (e.g. TransE) have aroused increasing attention for their simplicity and effectiveness. These models attempt to translate semantics from head entities to tail entities with the relations and infer richer facts outside the knowledge graph. In this paper, we propose a novel knowledge graph embedding method named TransMS, which translates and transmits multidirectional semantics: i) the semantics of head/tail entities and relations to tail/head entities with nonlinear functions and ii) the semantics from entities to relations with linear bias vectors. Our model has merely one additional parameter α than TransE for each triplet, which results in its better scalability in large-scale knowledge graph. Experiments show that TransMS achieves substantial improvements against state-of-the-art baselines, especially the Hit@10s of head entity prediction for N-1 relations and tail entity prediction for 1-N relations improved by about 27.1% and 24.8% on FB15K database respectively.","doi":"10.24963/ijcai.2019/268","url":"https://ijcai.org/proceedings/2019/268","pdf":"https://ijcai.org/proceedings/2019/0268.pdf","markdown":"https://ijcai.org/proceedings/2019/268.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/268","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Knowledge Representation and Reasoning","track":"Main track","number":268,"pages":{"first":1935,"last":1942},"keywords":["Knowledge Representation and Reasoning: Knowledge Representation Languages","Natural Language Processing: Embeddings","Machine Learning: Knowledge-based Learning","Natural Language Processing: Natural Language Semantics"],"videos":[],"references":[{"position":1,"raw":"[Bollacker et al., 2008] Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring hu- man knowledge. In Proceedings of the ACM SIGMOD International Conference on Management of Data, SIG- MOD 2008, Vancouver, BC, Canada, June 10-12, 2008, pages 1247-1250, 2008.","title":"Freebase: a collaboratively created graph database for structuring human knowledge","authors":"Kurt D Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, Jamie Taylor","year":"2008","venue":"Proceedings of the ACM SIGMOD International Conference on Management of Data, SIG-MOD 2008","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes et al., 2011] Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. Learning structured em- beddings of knowledge bases. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2011, 2011.","title":"Learning structured embeddings of knowledge bases","authors":"Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio","year":"2011","venue":"Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2011","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bordes et al., 2012] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2012, pages 127-135, 2012.","title":"Joint learning of words and meaning representations for open-text semantic parsing","authors":"Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio","year":"2012","venue":"Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2012","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi- relational data. In 27th Annual Conference on Neural Information Processing Systems 2013, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multirelational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"27th Annual Conference on Neural Information Processing Systems 2013","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bordes et al., 2014] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. A semantic matching energy function for learning with multi-relational data -applica- tion to word-sense disambiguation. Machine Learning, 94(2):233-259, 2014.","title":"A semantic matching energy function for learning with multi-relational data -application to word-sense disambiguation","authors":"Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dettmers et al., 2017] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. CoRR, abs/1707.01476, 2017.","title":"Convolutional 2d knowledge graph embeddings","authors":"Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Glorot and Bengio, 2010] Xavier Glorot and Yoshua Ben- gio. Understanding the difficulty of training deep feed- forward neural networks. In Proceedings of the Thir- teenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, pages 249-256, 2010.","title":"Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks","authors":"Bengio Glorot","year":"2010","venue":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jenatton et al., 2012] Rodolphe Jenatton, Nicolas Le Roux, Antoine Bordes, and Guillaume Obozinski. A latent fac- tor model for highly multi-relational data. In 26th An- nual Conference on Neural Information Processing Sys- tems 2012., pages 3176-3184, 2012.","title":"A latent factor model for highly multi-relational data","authors":"Rodolphe Jenatton, Nicolas Le Roux, Antoine Bordes, Guillaume Obozinski","year":"2012","venue":"26th Annual Conference on Neural Information Processing Systems 2012","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ji et al., 2015] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dy- namic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics ACL 2015, pages 687-696, 2015.","title":"Knowledge graph embedding via dynamic mapping matrix","authors":"Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, Jun Zhao","year":"2015","venue":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics ACL 2015","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ji et al., 2016] Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. Knowledge graph completion with adaptive sparse transfer matrix. In Thirtieth AAAI Conference on Artificial Intelligence, pages 985-991, 2016.","title":"Knowledge graph completion with adaptive sparse transfer matrix","authors":"Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao","year":"2016","venue":"Thirtieth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2015] Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu. Learning entity and relation embeddings for knowledge graph completion. In Twenty-Ninth AAAI Conference on Artificial Intelligence, pages 2181-2187, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Y Lin, Z Liu, M Sun, Y Liu, X Zhu","year":"2015","venue":"Twenty-Ninth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Miller, 1995] George A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38(11):39-41, 1995.","title":"Wordnet: A lexical database for english","authors":"George A Miller","year":"1995","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Socher et al., 2013] R. Socher, D. Chen, C. D. Manning, and A. Y. Ng. Reasoning with neural tensor networks for knowledge base completion. In International Conference on Neural Information Processing Systems, pages 926- 934, 2013.","title":"Reasoning with neural tensor networks for knowledge base completion","authors":"R Socher, D Chen, C D Manning, A Y Ng","year":"2013","venue":"International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tan et al., 2018] Zhen Tan, Xiang Zhao, Yang Fang, and Weidong Xiao. GTrans: generic knowledge graph embed- ding via multi-state entities and dynamic relation spaces. IEEE Access, 6(99):8232-8244, 2018.","title":"GTrans: generic knowledge graph embedding via multi-state entities and dynamic relation spaces","authors":"Zhen Tan, Xiang Zhao, Yang Fang, Weidong Xiao","year":"2018","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Toutanova and Chen, 2015] Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. 2015.","title":"Observed versus latent features for knowledge base and text inference","authors":"Kristina Toutanova, Danqi Chen","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Twenty-Eighth AAAI Con- ference on Artificial Intelligence, pages 1112-1119, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen","year":"2014","venue":"Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhang, 2017] Wen Zhang. Knowledge graph embedding with diversity of structures. In Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia, April 3-7, 2017, pages 747-753, 2017.","title":"Knowledge graph embedding with diversity of structures","authors":"Wen Zhang","year":"2017","venue":"Proceedings of the 26th International Conference on World Wide Web Companion","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}