{"type":"ScholarlyArticle","title":"Type-aware Embeddings for Multi-Hop Reasoning over Knowledge Graphs","authors":[{"name":"Zhiwei Hu","slug":"zhiwei-hu","given":"Zhiwei","family":"Hu","affiliation":"School of Computer and Information Technology , Shanxi University , China"},{"name":"Victor Gutierrez Basulto","slug":"victor-gutierrez-basulto","given":"Víctor","family":"Gutiérrez-Basulto","affiliation":"School of Computer Science and Informatics , Cardiff University , UK"},{"name":"Zhiliang Xiang","slug":"zhiliang-xiang","given":"Zhiliang","family":"Xiang","affiliation":"School of Computer Science and Informatics , Cardiff University , UK"},{"name":"Xiaoli Li","slug":"xiaoli-li","given":"Xiaoli","family":"Li","affiliation":"Institute for Infocomm Research / Centre for Frontier AI Research , A *"},{"name":"Ru Li","slug":"ru-li","given":"Ru","family":"Li","affiliation":"School of Computer and Information Technology , Shanxi University , China; STAR , Singapore"},{"name":"Jeff Z. Pan","slug":"jeff-z-pan","given":"Jeff Z.","family":"Pan","affiliation":"ILCC , School of Informatics , University of Edinburgh , UK"}],"abstract":"Multi-hop reasoning over real-life knowledge graphs (KGs) is a highly challenging  problem as traditional subgraph matching methods are not capable to deal with noise and missing information. Recently, to address this problem a promising approach based on jointly embedding logical queries and KGs into a low-dimensional space to identify answer entities has emerged. However,  existing proposals ignore critical semantic knowledge inherently available in KGs, such  as  type  information. To  leverage type  information, we  propose a novel type-aware  model, TypE-aware Message Passing (TEMP), which enhances the entity and relation representation in queries, and simultaneously improves generalization, and deductive and inductive reasoning. Remarkably, TEMP is a plug-and-play model that can be easily incorporated into existing embedding-based models to improve their performance. Extensive experiments on three real-world datasets demonstrate TEMP’s effectiveness.","doi":"10.24963/ijcai.2022/427","url":"https://ijcai.org/proceedings/2022/427","pdf":"https://ijcai.org/proceedings/2022/0427.pdf","markdown":"https://ijcai.org/proceedings/2022/427.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/427","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":427,"pages":{"first":3078,"last":3084},"keywords":["Knowledge Representation and Reasoning: Reasoning about Knowledge and Belief","Machine Learning: Knowledge Aided Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986640","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984783","length_sec":332,"provider":"slideslive"}],"references":[{"position":1,"raw":"a) Generalization on FB15k-237 (Q2B datasets) FB15k NELL Method 1p 2p 3p 2i 3i pi ip 2u up Avg Avg Avg GQE 41.3 21.5 15.2 26.5 38.5 16.7 8.8 17.1 15.8 22.4 40.1 23.5 +TEMP 47.6 29.6 24.7 36.3 48.4 25.5 13.4 30.2 21.0 30.7 56.6 38.2 Q2B 47.1 24.9 19.4 33.2 46.4 21.8 11.3 25.3 19.3 27.6 51.2 31.1 +TEMP 45.7 27.8 23.4 36.9 49.6 22.9 11.7 27.6 18.9 29.4 55.4 37.3 BETAE 42.6 25.4 21.6 30.2 43.3 20.7 9.2 24.2 18.3 26.2 50.6 33.4 +TEMP 43.3 27.2 22.7 35.3 47.5 24.3 10.6 26.7 18.2 28.4 53.6 34.5 LOGICE 45.6 27.8 24.1 34.7 46.5 23.5 12.0 27.1 20.8 29.1 54.2 39.1 +TEMP 46.6 29.2 25.0 35.8 47.9 24.9 13.5 28.7 21.0 30.3 55.7 39.1 (b) Deductive Reasoning on FB15k-237 (Q2B datasets) FB15k NELL GQE 56.4 30.1 24.5 35.9 51.2 25.1 13.0 25.8 22.0 31.6 43.7 49.8 +TEMP 76.3 48.6 39.0 49.7 60.4 36.9 22.1 59.0 36.3 47.6 71.4 75.5 Q2B 58.5 34.3 28.1 44.7 62.1 23.9 11.7 40.5 22.0 36.2 43.7 51.1 +TEMP 87.2 59.6 47.9 67.2 72.7 49.1 29.8 78.6 43.4 59.5 69.6 90.4 BETAE 77.9 52.6 44.5 59.0 67.8 42.2 23.5 63.7 35.1 51.8 60.6 80.2 +TEMP 84.7 58.3 49.4 62.3 68.8 45.3 28.5 74.5 41.1 57.0 60.6 81.5 LOGICE 81.5 54.2 46.0 58.1 67.1 44.0 28.5 66.6 40.8 54.1 65.5 85.3 +TEMP 84.5 59.8 51.9 59.3 68.1 47.0 33.4 70.8 45.4 57.8 67.1 84.6","title":"a) Generalization on FB15k-237 (Q2B datasets) FB15k NELL Method 1p 2p 3p 2i 3i pi ip 2u up","authors":null,"year":null,"venue":"Avg Avg Avg GQE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arakelyan et al., 2021] Erik Arakelyan, Daniel Daza, Pasquale Minervini, and Michael Cochez. Complex query answering with neural link predictors. In ICLR, 2021.","title":"Complex query answering with neural link predictors","authors":"Arakelyan","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NeurIPS, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Jiajun Chen, Huarui He, Feng Wu, and Jie Wang. Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In AAAI, pages 6271-6278, 2021.","title":"Topology-aware correlations between relations for inductive link prediction in knowledge graphs","authors":"Jiajun Chen, Huarui He, Feng Wu, Jie Wang","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choudhary et al., 2021a] Nurendra Choudhary, Nikhil Rao, Sumeet Katariya, Karthik Subbian, and Chandan Reddy. Probabilistic entity representation model for reasoning over knowledge graphs. In NeurIPS, 2021.","title":"Probabilistic entity representation model for reasoning over knowledge graphs","authors":"Choudhary","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choudhary et al., 2021b] Nurendra Choudhary, Nikhil Rao, Sumeet Katariya, Karthik Subbian, and Chandan K Reddy. Self-supervised hyperboloid representations from logical queries over knowledge graphs. In WWW, 2021.","title":"Self-supervised hyperboloid representations from logical queries over knowledge graphs","authors":"Choudhary","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Daza et al., 2021] Daniel Daza, Michael Cochez, and Paul Groth. Inductive entity representations from text via link prediction. In WWW, 2021.","title":"Inductive entity representations from text via link prediction","authors":"Daniel Daza, Michael Cochez, Paul Groth","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hamilton et al., 2018] William L Hamilton, Payal Bajaj, Marinka Zitnik, Dan Jurafsky, and Jure Leskovec. Em- bedding logical queries on knowledge graphs. In NeurIPS, 2018.","title":"Embedding logical queries on knowledge graphs","authors":"Payal William L Hamilton, Marinka Bajaj, Dan Zitnik, Jure Jurafsky, Leskovec","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Luus et al., 2021] Francois Luus, Prithviraj Sen, Pavan Ka- panipathi, Ryan Riegel, Ndivhuwo Makondo, Thabang Lebese, and Alexander Gray. Logic embeddings for com- plex query answering. In NeurIPS, 2021.","title":"Logic embeddings for complex query answering","authors":"Francois Luus, Prithviraj Sen, Pavan Kapanipathi, Ryan Riegel, Ndivhuwo Makondo, Thabang Lebese, Alexander Gray","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Niu et al., 2020] Guanglin Niu, Bo Li, Yongfei Zhang, Shil- iang Pu, and Jingyang Li. Autoeter: Automated entity type representation for knowledge graph embedding. In EMNLP, 2020.","title":"Autoeter: Automated entity type representation for knowledge graph embedding","authors":"Guanglin Niu, Bo Li, Yongfei Zhang, Shiliang Pu, Jingyang Li","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Pan et al., 2016] J.Z. Pan, G. Vetere, J.M. Gomez-Perez, and H. Wu. Exploiting Linked Data and Knowledge Graphs for Large Organisations. Springer, 2016.","title":"Exploiting Linked Data and Knowledge Graphs for Large Organisations","authors":"J Z Pan, G Vetere, J M Gomez-Perez, H Wu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Pan et al., 2021] Weiran Pan, Wei Wei, and Xianling Mao. Context-aware entity typing in knowledge graphs. In EMNLP, 2021.","title":"Context-aware entity typing in knowledge graphs","authors":"Pan","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Pan, 2009] Jeff Z. Pan. Resource description framework. In Handbook on Ontologies, pages 71-90. 2009.","title":"Resource description framework","authors":"Jeff Z Pan","year":"2009","venue":"Handbook on Ontologies","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ren and Leskovec, 2020] Hongyu Ren and Jure Leskovec. Beta embeddings for multi-hop logical reasoning in knowledge graphs. In NeurIPS, 2020.","title":"Beta embeddings for multi-hop logical reasoning in knowledge graphs","authors":"Leskovec ; ] Ren, Jure Hongyu Ren, Leskovec","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ren et al., 2020] Hongyu Ren, Weihua Hu, and Jure Leskovec. Query2box: Reasoning over knowledge graphs in vector space using box embeddings. In ICLR, 2020.","title":"Query2box: Reasoning over knowledge graphs in vector space using box embeddings","authors":"Weihua Hongyu Ren, Jure Hu, Leskovec","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Srivastava et al., 2015] Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. Highway networks. arXiv preprint arXiv:1505.00387, 2015.","title":"Highway networks","authors":"Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2020] Haitian Sun, Andrew O. Arnold, Tania Bedrax-Weiss, Fernando Pereira, and William W. Co- hen. Faithful embeddings for knowledge base queries. In NeurIPS, 2020.","title":"Faithful embeddings for knowledge base queries","authors":"Sun","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Christopher D. Manning. Improved semantic represen- tations from tree-structured long short-term memory net- works. In ACL, pages 1556-1566, 2015.","title":"Improved semantic representations from tree-structured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Christopher D Manning","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Teru et al., 2020] Komal Teru, Etienne Denis, and Will Hamilton. Inductive relation prediction by subgraph rea- soning. In ICML, pages 9448-9457, 2020.","title":"Inductive relation prediction by subgraph reasoning","authors":"Komal Teru, Etienne Denis, Will Hamilton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Toutanova and Chen, 2015] Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In ACL, pages 57-66, 2015.","title":"Observed versus latent features for knowledge base and text inference","authors":"Chen ; Toutanova, Kristina Toutanova, Danqi Chen","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2021] Hongwei Wang, Hongyu Ren, and Jure Leskovec. Relational message passing for knowledge graph completion. In SIGKDD, pages 1697-1707, 2021.","title":"Relational message passing for knowledge graph completion","authors":"Hongwei Wang, Hongyu Ren, Jure Leskovec","year":"2021","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wiharja et al., 2020] Kemas Wiharja, Jeff Z. Pan, Martin J. Kollingbaum, and Yu Deng. Schema Aware Iterative Knowledge Graph Completion. Journal of Web Seman- tics, 2020.","title":"Schema Aware Iterative Knowledge Graph Completion","authors":"Kemas Wiharja, Jeff Z Pan, Martin J Kollingbaum, Yu Deng","year":"2020","venue":"Journal of Web Semantics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiong et al., 2017] Wenhan Xiong, Thien Hoang, and William Yang Wang. Deeppath: A reinforcement learn- ing method for knowledge graph reasoning. In EMNLP, 2017.","title":"Deeppath: A reinforcement learning method for knowledge graph reasoning","authors":"Xiong","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yao et al., 2019] Liang Yao, Chengsheng Mao, and Yuan Luo. Kg-bert: Bert for knowledge graph completion. arXiv preprint arXiv:1909.03193, 2019.","title":"Kg-bert: Bert for knowledge graph completion","authors":"Liang Yao, Chengsheng Mao, Yuan Luo","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2020] Shuailiang Zhang, Hai Zhao, Yuwei Wu, Zhuosheng Zhang, Xi Zhou, and Xiang Zhou. Dcmn+: Dual co-matching network for multi-choice read- ing comprehension. In AAAI, 2020.","title":"Dcmn+: Dual co-matching network for multi-choice reading comprehension","authors":"Shuailiang Zhang, Hai Zhao, Yuwei Wu, Zhuosheng Zhang, Xi Zhou, Xiang Zhou","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2021] Zhanqiu Zhang, Jie Wang, Jiajun Chen, Shuiwang Ji, and Feng Wu. Cone: Cone embeddings for multi-hop reasoning over knowledge graphs. In NeurIPS, 2021.","title":"Cone: Cone embeddings for multi-hop reasoning over knowledge graphs","authors":"Zhang","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2020] Yu Zhao, Anxiang Zhang, Ruobing Xie, Kang Liu, and Xiaojie Wang. Connecting embeddings for knowledge graph entity typing. In ACL, 2020.","title":"Connecting embeddings for knowledge graph entity typing","authors":"Yu Zhao, Anxiang Zhang, Ruobing Xie, Kang Liu, Xiaojie Wang","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}