{"type":"ScholarlyArticle","title":"Augmenting Knowledge Graphs for Better Link Prediction","authors":[{"name":"Jiang Wang","slug":"jiang-wang","given":"Jiang","family":"Wang","affiliation":"USC Information Sciences Institute"},{"name":"Filip Ilievski","slug":"filip-ilievski","given":"Filip","family":"Ilievski","affiliation":"USC Information Sciences Institute"},{"name":"Pedro Szekely","slug":"pedro-szekely","given":"Pedro","family":"Szekely","affiliation":"USC Information Sciences Institute"},{"name":"Ke-Thia Yao","slug":"ke-thia-yao","given":"Ke-Thia","family":"Yao","affiliation":"USC Information Sciences Institute"}],"abstract":"Embedding methods have demonstrated robust performance on the task of link prediction in knowledge graphs, by mostly encoding entity relationships. Recent methods propose to enhance the loss function with a literal-aware term. In this paper, we propose KGA: a knowledge graph augmentation method that incorporates literals in an embedding model without modifying its loss function. KGA discretizes quantity and year values into bins, and chains these bins both horizontally, modeling neighboring values, and vertically, modeling multiple levels of granularity. KGA is scalable and can be used as a pre-processing step for any existing knowledge graph embedding model. Experiments on legacy benchmarks and a new large benchmark, DWD, show that augmenting the knowledge graph with quantities and years is beneficial for predicting both entities and numbers, as KGA outperforms the vanilla models and other relevant baselines. Our ablation studies confirm that both quantities and years contribute to KGA's performance, and that its performance depends on the discretization and binning settings. We make the code, models, and the DWD benchmark publicly available to facilitate reproducibility and future research.","doi":"10.24963/ijcai.2022/316","url":"https://ijcai.org/proceedings/2022/316","pdf":"https://ijcai.org/proceedings/2022/0316.pdf","markdown":"https://ijcai.org/proceedings/2022/316.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/316","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":316,"pages":{"first":2277,"last":2283},"keywords":["Natural Language Processing: Embeddings","Knowledge Representation and Reasoning: Semantic Web","Data Mining: Knowledge Graphs and Knowledge Base Completion","Knowledge Representation and Reasoning: Learning and reasoning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986581","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984724","length_sec":359,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Balažević et al., 2019] Ivana Balažević, Carl Allen, and Timothy M Hospedales. Tucker: Tensor factoriza- tion for knowledge graph completion. arXiv preprint arXiv:1901.09590, 2019.","title":"Tucker: Tensor factorization for knowledge graph completion","authors":"Ivana Balažević, Carl Allen, Timothy M Hospedales","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bayram et al., 2021] Eda Bayram, Alberto García-Durán, and Robert West. Node attribute completion in knowledge graphs with multi-relational propagation. In ICASSP 2021. IEEE, 2021.","title":"Node attribute completion in knowledge graphs with multi-relational propagation","authors":"Bayram","year":"2021","venue":"ICASSP 2021","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bollacker et al., 2008] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247-1250, 2008.","title":"Freebase: a collaboratively created graph database for structuring human knowledge","authors":"Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, Jamie Taylor","year":"2008","venue":"Proceedings of the 2008 ACM SIGMOD international conference on Management of data","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dehaene, 2011] Stanislas Dehaene. The number sense: How the mind creates mathematics. OUP USA, 2011.","title":"The number sense: How the mind creates mathematics","authors":"Stanislas Dehaene, Dehaene","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dettmers et al., 2018] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Thirty-second AAAI conference on artificial intelligence, 2018.","title":"Convolutional 2d knowledge graph embeddings","authors":"Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel","year":"2018","venue":"Thirty-second AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Feng et al., 2019] Ming-Han Feng, Chin-Chi Hsu, Cheng- Te Li, Mi-Yen Yeh, and Shou-De Lin. Marine: Multi- relational network embeddings with relational proximity and node attributes. In The World Wide Web Conference, pages 470-479, 2019.","title":"Marine: Multirelational network embeddings with relational proximity and node attributes","authors":"Ming-Han Feng, Chin-Chi Hsu, Cheng-Te Li, Mi-Yen Yeh, Shou-De Lin","year":"2019","venue":"The World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[García-Durán and Niepert, 2017] Alberto García-Durán and Mathias Niepert. Kblrn: End-to-end learning of knowledge base representations with latent, relational, and numerical features. arXiv preprint arXiv:1709.04676, 2017.","title":"Kblrn: End-to-end learning of knowledge base representations with latent, relational, and numerical features","authors":"Alberto García-Durán, Mathias Niepert","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gesese et al., 2019] Genet Asefa Gesese, Russa Biswas, Mehwish Alam, and Harald Sack. A survey on knowl- edge graph embeddings with literals: Which model links better literal-ly? Semantic Web, (Preprint):1-31, 2019.","title":"A survey on knowledge graph embeddings with literals: Which model links better literal-ly? Semantic Web","authors":"Gesese","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kotnis and García-Durán, 2019] Bhushan Kotnis and Al- berto García-Durán. Learning numerical attributes in knowledge bases. In Automated Knowledge Base Con- struction (AKBC), 2019.","title":"Learning numerical attributes in knowledge bases","authors":"García-Durán ; Kotnis, García-Durán","year":"2019","venue":"Automated Knowledge Base Construction (AKBC)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kristiadi et al., 2019] Agustinus Kristiadi, Mohammad Asif Khan, Denis Lukovnikov, Jens Lehmann, and Asja Fis- cher. Incorporating literals into knowledge graph embed- dings. In International Semantic Web Conference, pages 347-363. Springer, 2019.","title":"Incorporating literals into knowledge graph embeddings","authors":"Agustinus Kristiadi, Mohammad Asif Khan, Denis Lukovnikov, Jens Lehmann, Asja Fischer","year":"2019","venue":"International Semantic Web Conference","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lacroix et al., 2020] Timothée Lacroix, Guillaume Obozin- ski, and Nicolas Usunier. Tensor decompositions for temporal knowledge base completion. arXiv preprint arXiv:2004.04926, 2020.","title":"Tensor decompositions for temporal knowledge base completion","authors":"Timothée Lacroix, Guillaume Obozinski, Nicolas Usunier","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lerer et al., 2019] Adam Lerer, Ledell Wu, Jiajun Shen, Timothee Lacroix, Luca Wehrstedt, Abhijit Bose, and Alex Peysakhovich. Pytorch-biggraph: A large scale graph embedding system. In A. Talwalkar, V. Smith, and M. Zaharia, editors, Proceedings of Machine Learning and Systems, volume 1, pages 120-131, 2019.","title":"Pytorch-biggraph: A large scale graph embedding system","authors":"Adam Lerer, Ledell Wu, Jiajun Shen, Timothee Lacroix, Luca Wehrstedt, Abhijit Bose, Alex Peysakhovich","year":"2019","venue":"Proceedings of Machine Learning and Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2019] Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, and David S Rosenblum. Mmkg: multi-modal knowledge graphs. In European Semantic Web Conference, pages 459-474. Springer, 2019.","title":"Mmkg: multi-modal knowledge graphs","authors":"Liu","year":"2019","venue":"European Semantic Web Conference","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lloyd, 1982] S. Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129- 137, 1982.","title":"Least squares quantization in pcm","authors":"S Lloyd","year":"1982","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lu et al., 2020] Fengyuan Lu, Peijin Cong, and Xinli Huang. Utilizing textual information in knowledge graph embedding: A survey of methods and applications. IEEE Access, 8:92072-92088, 2020.","title":"Utilizing textual information in knowledge graph embedding: A survey of methods and applications","authors":"Lu","year":"2020","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Suchanek et al., 2007] Fabian M Suchanek, Gjergji Kas- neci, and Gerhard Weikum. Yago: a core of semantic knowledge. In Proceedings of the 16th international con- ference on World Wide Web, pages 697-706, 2007.","title":"Yago: a core of semantic knowledge","authors":"Gjergji Fabian M Suchanek, Gerhard Kasneci, Weikum","year":"2007","venue":"Proceedings of the 16th international conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sun et al., 2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197, 2019.","title":"Rotate: Knowledge graph embedding by relational rotation in complex space","authors":"Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tay et al., 2017] Yi Tay, Luu Anh Tuan, Minh C Phan, and Siu Cheung Hui. Multi-task neural network for non- discrete attribute prediction in knowledge graphs. In Pro- ceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 1029-1038, 2017.","title":"Multi-task neural network for nondiscrete attribute prediction in knowledge graphs","authors":"Yi Tay, Anh Luu, Minh C Tuan, Siu Phan, Hui Cheung","year":"2017","venue":"Proceedings of the 2017 ACM on Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Toutanova and Chen, 2015] Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositional- ity, pages 57-66, 2015.","title":"Observed versus latent features for knowledge base and text inference","authors":"Chen ; Toutanova, Kristina Toutanova, Danqi Chen","year":"2015","venue":"Proceedings of the 3rd workshop on continuous vector space models and their compositionality","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Trouillon et al., 2016] Théo Trouillon, Johannes Welbl, Se- bastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In In- ternational conference on machine learning, pages 2071- 2080. PMLR, 2016.","title":"Complex embeddings for simple link prediction","authors":"Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vrandečić and Krötzsch, 2014] Denny Vrandečić and Markus Krötzsch. Wikidata: a free collaborative knowl- edgebase. Communications of the ACM, 57(10):78-85, 2014.","title":"Wikidata: a free collaborative knowledgebase","authors":"Krötzsch Vrandečić, Denny Vrandečić, Markus Krötzsch","year":"2014","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2021] Meihong Wang, Linling Qiu, and Xiaoli Wang. A survey on knowledge graph embeddings for link prediction. Symmetry, 13(3):485, 2021.","title":"A survey on knowledge graph embeddings for link prediction","authors":"Wang","year":"2021","venue":"Symmetry","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu and Wang, 2018] Yanrong Wu and Zhichun Wang. Knowledge graph embedding with numeric attributes of entities. In Proceedings of The Third Workshop on Repre- sentation Learning for NLP, pages 132-136, 2018.","title":"Knowledge graph embedding with numeric attributes of entities","authors":"Wang ; Wu, Yanrong Wu, Zhichun Wang","year":"2018","venue":"Proceedings of The Third Workshop on Representation Learning for NLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xiao et al., 2017] Han Xiao, Minlie Huang, Lian Meng, and Xiaoyan Zhu. Ssp: semantic space projection for knowl- edge graph embedding with text descriptions. In Thirty- First AAAI conference on artificial intelligence, 2017.","title":"Ssp: semantic space projection for knowledge graph embedding with text descriptions","authors":"Han Xiao, Minlie Huang, Lian Meng, Xiaoyan Zhu","year":"2017","venue":"Thirty-First AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2014] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575, 2014.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, Li Deng","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}