{"type":"ScholarlyArticle","title":"Knowledge Transfer for Out-of-Knowledge-Base Entities : A Graph Neural Network Approach","authors":[{"name":"Takuo Hamaguchi","slug":"takuo-hamaguchi","given":"Takuo","family":"Hamaguchi","affiliation":"Nara Institute of Science and Technology , Ikoma , Nara , Japan"},{"name":"Hidekazu Oiwa","slug":"hidekazu-oiwa","given":"Hidekazu","family":"Oiwa","affiliation":"Recruit Institute of Technology"},{"name":"Masashi Shimbo","slug":"masashi-shimbo","given":"Masashi","family":"Shimbo","affiliation":"Nara Institute of Science and Technology , Ikoma , Nara , Japan"},{"name":"Yuji Matsumoto","slug":"yuji-matsumoto","given":"Yuji","family":"Matsumoto","affiliation":"Nara Institute of Science and Technology , Ikoma , Nara , Japan"}],"abstract":"Knowledge base completion (KBC) aims to predict missing information in a knowledge base. In this paper, we address the out-of-knowledge-base (OOKB) entity problem in KBC: how to answer queries concerning test entities not observed at training time. Existing embedding-based KBC models assume that all test entities are available at training time, making it unclear how to obtain embeddings for new entities without costly retraining. To solve the OOKB entity problem without retraining, we use graph neural networks (Graph-NNs) to compute the embeddings of OOKB entities, exploiting the limited auxiliary knowledge provided at test time. The experimental results show the effectiveness of our proposed model in the OOKB setting. Additionally, in the standard KBC setting in which OOKB entities are not involved, our model achieves state-of-the-art performance on the WordNet dataset.","doi":"10.24963/ijcai.2017/250","url":"https://ijcai.org/proceedings/2017/250","pdf":"https://ijcai.org/proceedings/2017/0250.pdf","markdown":"https://ijcai.org/proceedings/2017/250.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/250","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":250,"pages":{"first":1802,"last":1808},"keywords":["Machine Learning: Deep Learning","Knowledge Representation, Reasoning, and Logic: Non-classical logics for Knowledge Representation","Natural Language Processing: Information Extraction","Machine Learning: Neural Networks"],"videos":[],"references":[{"position":1,"raw":"[Bollacker et al., 2008] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pages 1247-1250, 2008.","title":"Freebase: A collaboratively created graph database for structuring human knowledge","authors":"Bollacker","year":"2008","venue":"Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2016] Shaosheng Cao, Wei Lu, and Qiongkai Xu. Deep neural networks for learning graph represen- tations. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2016.","title":"Deep neural networks for learning graph representations","authors":"Cao","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural net- works on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems 29, pages 3844-3852, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fang et al., 2016] Wei Fang, Jianwen Zhang, Dilin Wang, Zheng Chen, and Ming Li. Entity disambiguation by knowledge and text jointly embedding. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 260-269, 2016.","title":"Entity disambiguation by knowledge and text jointly embedding","authors":"Wei Fang, Jianwen Zhang, Dilin Wang, Zheng Chen, Ming Li","year":"2016","venue":"Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guu et al., 2015] Kelvin Guu, John Miller, and Percy Liang. Traversing knowledge graphs in vector space. In Proceed- ings of the 2015 Conference on Empirical Methods in Nat- ural Language Processing, pages 318-327, 2015.","title":"Traversing knowledge graphs in vector space","authors":"Kelvin Guu, John Miller, Percy Liang","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, pages 448-456, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"Proceedings of the 32nd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ji et al., 2015] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dy- namic mapping matrix. In Proceedings of the 53rd An- nual Meeting of the Association for Computational Lin- guistics and the 7th International Joint Conference on Nat- ural Language Processing, volume 1: Long Papers, pages 687-696, 2015.","title":"Knowledge graph embedding via dynamic mapping matrix","authors":"Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, Jun Zhao","year":"2015","venue":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ji et al., 2016] Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. Knowledge graph completion with adaptive sparse transfer matrix. In Proceedings of the 30th AAAI Confer- ence on Artificial Intelligence, 2016.","title":"Knowledge graph completion with adaptive sparse transfer matrix","authors":"Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2015] Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. Gated graph sequence neural networks. CoRR, abs/1511.05493, 2015.","title":"Gated graph sequence neural networks","authors":"Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard S Zemel","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2015] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Proceed- ings of the 29th AAAI Conference on Artificial Intelligence, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Lin","year":"2015","venue":"Proceedings of the 29th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Miller, 1995] George A. Miller. Wordnet: A lexical database for English. Communication of the ACM, 38(11):39-41, 1995.","title":"Wordnet: A lexical database for English","authors":"George A Miller","year":"1995","venue":"Communication of the ACM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nguyen et al., 2016] Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. Neighborhood mixture model for knowledge base completion. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 40-50, 2016.","title":"Neighborhood mixture model for knowledge base completion","authors":"Kairit Dat Quoc Nguyen, Lizhen Sirts, Mark Qu, Johnson","year":"2016","venue":"Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nickel et al., 2016] M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104:11- 33, 2016.","title":"A review of relational machine learning for knowledge graphs","authors":"M Nickel, K Murphy, V Tresp, E Gabrilovich","year":"2016","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Scarselli et al., 2009] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20:61-80, 2009.","title":"The graph neural network model","authors":"F Scarselli, M Gori, A C Tsoi, M Hagenbuchner, G Monfardini","year":"2009","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Socher et al., 2013] Richard Socher, Danqi Chen, Christo- pher D Manning, and Andrew Ng. Reasoning with neu- ral tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems 26, pages 926-934, 2013.","title":"Reasoning with neural tensor networks for knowledge base completion","authors":"Richard Socher, Danqi Chen, Christopher D Manning, Andrew Ng","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vincent et al., 2010] Pascal Vincent, Hugo Larochelle, Is- abelle Lajoie, Yoshua Bengio, and Pierre-Antoine Man- zagol. Stacked denoising autoencoders: Learning use- ful representations in a deep network with a local de- noising criterion. Journal of Machine Learning Research, 11:3371-3408, 2010.","title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","authors":"Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2014a] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph and text jointly embedding. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1591-1601, 2014.","title":"Knowledge graph and text jointly embedding","authors":"Wang","year":"2014","venue":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2014b] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Wang","year":"2014","venue":"Proceedings of the 28th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xiao et al., 2016a] Han Xiao, Minlie Huang, and Xiaoyan Zhu. From one point to a manifold: Knowledge graph embedding for precise link prediction. In Proceedings of the 25th International Joint Conference on Artificial Intel- ligence, pages 1315-1321, 2016.","title":"From one point to a manifold: Knowledge graph embedding for precise link prediction","authors":"Xiao","year":"2016","venue":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/190"},{"position":23,"raw":"[Xiao et al., 2016b] Han Xiao, Minlie Huang, and Xiaoyan Zhu. TransG: A generative model for knowledge graph embedding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, volume 1: Long Papers, pages 2316-2325, 2016.","title":"TransG: A generative model for knowledge graph embedding","authors":"Xiao","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yoon et al., 2016] Hee-Geun Yoon, Hyun-Je Song, Seong- Bae Park, and Se-Young Park. A translation-based knowl- edge graph embedding preserving logical property of rela- tions. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 907- 916, 2016.","title":"A translation-based knowledge graph embedding preserving logical property of relations","authors":"Yoon","year":"2016","venue":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhong et al., 2015] Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, and Zheng Chen. Aligning knowledge and text embeddings by entity descriptions. In Proceed- ings of the 2015 Conference on Empirical Methods in Nat- ural Language Processing, pages 267-272, 2015.","title":"Aligning knowledge and text embeddings by entity descriptions","authors":"Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, Zheng Chen","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}