{"type":"ScholarlyArticle","title":"Knowledge Graph Representation with Jointly Structural and Textual Encoding","authors":[{"name":"Jiacheng Xu","slug":"jiacheng-xu","given":"Jiacheng","family":"Xu","affiliation":"♠ Software School , Fudan University , Shanghai , China"},{"name":"Xipeng Qiu","slug":"xipeng-qiu","given":"Xipeng","family":"Qiu","affiliation":"♦ School of Computer Science , Fudan University , Shanghai , China"},{"name":"Kan Chen","slug":"kan-chen","given":"Kan","family":"Chen","affiliation":"♦ School of Computer Science , Fudan University , Shanghai , China; Shanghai Key Laboratory of Intelligent Information Processing , Fudan University , Shanghai , China"},{"name":"Xuanjing Huang","slug":"xuanjing-huang","given":"Xuanjing","family":"Huang","affiliation":"♦ School of Computer Science , Fudan University , Shanghai , China"}],"abstract":"The objective of knowledge graph embedding is to encode both entities and relations of knowledge graphs into continuous low-dimensional vector spaces. Previously, most works focused on symbolic representation of knowledge graph with structure information, which can not handle  new entities or entities with few facts well. In this paper, we propose a novel deep architecture to utilize both structural and textual information of entities. Specifically, we introduce three neural models to encode the valuable information from text description of entity, among which an attentive model can select related information as needed. Then, a gating mechanism is applied to integrate representations of structure and text into a unified architecture. Experiments show that our models outperform baseline and obtain state-of-the-art results on link prediction and triplet classification tasks.","doi":"10.24963/ijcai.2017/183","url":"https://ijcai.org/proceedings/2017/183","pdf":"https://ijcai.org/proceedings/2017/0183.pdf","markdown":"https://ijcai.org/proceedings/2017/183.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/183","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Knowledge Representation, Reasoning, and Logic","track":"Main track","number":183,"pages":{"first":1318,"last":1324},"keywords":["Knowledge Representation, Reasoning, and Logic: Knowledge Representation Languages","Machine Learning: Deep Learning","Natural Language Processing: NLP Applications and Tools","Machine Learning: Knowledge-based Learning"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 1994] Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. Neural Networks, IEEE Transactions on, 5(2):157-166, 1994.","title":"Learning long-term dependencies with gradient descent is difficult","authors":"Yoshua Bengio, Patrice Simard, Paolo Frasconi","year":"1994","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bollacker et al., 2008] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247-1250. ACM, 2008.","title":"Freebase: a collaboratively created graph database for structuring human knowledge","authors":"Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, Jamie Taylor","year":"2008","venue":"Proceedings of the 2008 ACM SIGMOD international conference on Management of data","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bordes et al., 2012] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. Joint learning of words and meaning representations for open-text semantic parsing. In AISTATS, 2012.","title":"Joint learning of words and meaning representations for open-text semantic parsing","authors":"Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio","year":"2012","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bordes et al., 2014] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. A semantic matching energy function for learning with multi-relational data. Machine Learning, 94(2):233-259, 2014.","title":"A semantic matching energy function for learning with multi-relational data","authors":"Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceedings of EMNLP, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. JMLR, 12:2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Duchi et al., 2011] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. JMLR, 12:2121-2159, 2011.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"John Duchi, Elad Hazan, Yoram Singer","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Elman, 1990] Jeffrey L Elman. Finding structure in time. Cognitive science, 14(2):179-211, 1990.","title":"Finding structure in time","authors":"Jeffrey L Elman","year":"1990","venue":"Cognitive science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Graves and Schmidhuber, 2005] Alex Graves and Jürgen Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Networks, 18(5):602-610, 2005.","title":"Framewise phoneme classification with bidirectional lstm and other neural network architectures","authors":"Schmidhuber Graves, Alex Graves, Jürgen Schmidhuber","year":"2005","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Graves, 2013] Alex Graves. Generating sequences with re- current neural networks. arXiv preprint arXiv:1308.0850, 2013.","title":"Generating sequences with recurrent neural networks","authors":"Alex Graves","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hochreiter et al., 2001] Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, and Jürgen Schmidhuber. Gradient flow in recurrent nets: the difficulty of learning long-term de- pendencies, 2001.","title":"Gradient flow in recurrent nets: the difficulty of learning long-term dependencies","authors":"Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ji et al., 2015] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dy- namic mapping matrix. In Proceedings of ACL, 2015.","title":"Knowledge graph embedding via dynamic mapping matrix","authors":"Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, Jun Zhao","year":"2015","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ji et al., 2016] Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. Knowledge graph completion with adaptive sparse transfer matrix. In AAAI, 2016.","title":"Knowledge graph completion with adaptive sparse transfer matrix","authors":"Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kalchbrenner et al., 2014] Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. In ACL, 2014.","title":"A convolutional neural network for modelling sentences","authors":"Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lao et al., 2012] Ni Lao, Amarnag Subramanya, Fernando Pereira, and William W Cohen. Reading the web with learned syntactic-semantic inference rules. In Proceed- ings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Nat- ural Language Learning, pages 1017-1026. Association for Computational Linguistics, 2012.","title":"Reading the web with learned syntactic-semantic inference rules","authors":"Ni Lao, Amarnag Subramanya, Fernando Pereira, William W Cohen","year":"2012","venue":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2015] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In AAAI, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Miller, 1995] G.A. Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11), 1995.","title":"Wordnet: a lexical database for english","authors":"G A Miller","year":"1995","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Neelakantan et al., 2015] Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. Compositional vector space models for knowledge base completion. arXiv preprint arXiv:1504.06662, 2015.","title":"Compositional vector space models for knowledge base completion","authors":"Arvind Neelakantan, Benjamin Roth, Andrew Mccallum","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schuster and Paliwal, 1997] Mike Schuster and Kuldip K Paliwal. Bidirectional recurrent neural networks. Signal Processing, IEEE Transactions on, 1997.","title":"Bidirectional recurrent neural networks","authors":"Mike Schuster, Kuldip K Paliwal","year":"1997","venue":"Signal Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Socher et al., 2013] Richard Socher, Danqi Chen, Christo- pher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. In Ad- vances in NIPS, pages 926-934, 2013.","title":"Reasoning with neural tensor networks for knowledge base completion","authors":"Richard Socher, Danqi Chen, Christopher D Manning, Andrew Ng","year":"2013","venue":"Advances in NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. Sequence to sequence learning with neural networks. In Advances in NIPS, pages 3104-3112, 2014.","title":"Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Taskar et al., 2005] Ben Taskar, Vassil Chatalbashev, Daphne Koller, and Carlos Guestrin. Learning struc- tured prediction models: A large margin approach. In Proceedings of the ICML, 2005.","title":"Learning structured prediction models: A large margin approach","authors":"Ben Taskar, Vassil Chatalbashev, Daphne Koller, Carlos Guestrin","year":"2005","venue":"Proceedings of the ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Toutanova et al., 2015] Kristina Toutanova, Danqi Chen, Patrick Pantel, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowl- edge bases. In Proceedings of EMNLP, 2015.","title":"Representing text for joint embedding of text and knowledge bases","authors":"Kristina Toutanova, Danqi Chen, Patrick Pantel, Pallavi Choudhury, Michael Gamon","year":"2015","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2014a] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph and text jointly embedding. In Proceedings of EMNLP, 2014.","title":"Knowledge graph and text jointly embedding","authors":"Wang","year":"2014","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2014b] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of AAAI, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Wang","year":"2014","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2016] Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. Representation learning of knowledge graphs with entity descriptions. In Proceed- ings of IJCAI, 2016.","title":"Representation learning of knowledge graphs with entity descriptions","authors":"Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, Maosong Sun","year":"2016","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2015] Dongxu Zhang, Bin Yuan, Dong Wang, and Rong Liu. Joint semantic relevance learning with text data and graph knowledge. ACL-IJCNLP 2015, 2015.","title":"Joint semantic relevance learning with text data and graph knowledge","authors":"Zhang","year":"2015","venue":"ACL-IJCNLP 2015","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhong et al., 2015] Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, and Zheng Chen. Aligning knowledge and text embeddings by entity descriptions. In Proceed- ings of EMNLP, 2015.","title":"Aligning knowledge and text embeddings by entity descriptions","authors":"Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, Zheng Chen","year":"2015","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}