{"type":"ScholarlyArticle","title":"Subgraph Neighboring Relations Infomax for Inductive Link Prediction on Knowledge Graphs","authors":[{"name":"Xiaohan Xu","slug":"xiaohan-xu","given":"Xiaohan","family":"Xu","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences; School of Cyber Security , University of Chinese Academy of Sciences {xuxiaohan ,"},{"name":"Peng Zhang","slug":"peng-zhang","given":"Peng","family":"Zhang","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences"},{"name":"Yongquan He","slug":"yongquan-he","given":"Yongquan","family":"He","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences; School of Cyber Security , University of Chinese Academy of Sciences {xuxiaohan ,"},{"name":"Chengpeng Chao","slug":"chengpeng-chao","given":"Chengpeng","family":"Chao","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences; School of Cyber Security , University of Chinese Academy of Sciences {xuxiaohan ,"},{"name":"Chaoyang Yan","slug":"chaoyang-yan","given":"Chaoyang","family":"Yan","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences; School of Cyber Security , University of Chinese Academy of Sciences {xuxiaohan ,"}],"abstract":"Inductive link prediction for knowledge graph aims at predicting missing links between unseen entities, those not shown in training stage. Most previous works learn entity-specific embeddings of entities, which cannot handle unseen entities. Recent several methods utilize enclosing subgraph to obtain inductive ability. However, all these works only consider the enclosing part of subgraph without complete neighboring relations, which leads to the issue that partial neighboring relations are neglected, and sparse subgraphs are hard to be handled. To address that, we propose Subgraph Neighboring Relations Infomax, SNRI, which sufficiently exploits complete neighboring relations from two aspects: neighboring relational feature for node feature and neighboring relational path for sparse subgraph. To further model neighboring relations in a global way, we innovatively apply mutual information (MI) maximization for knowledge graph. Experiments show that SNRI outperforms existing state-of-art methods by a large margin on inductive link prediction task, and verify the effectiveness of exploring complete neighboring relations in a global way to characterize node features and reason on sparse subgraphs.","doi":"10.24963/ijcai.2022/325","url":"https://ijcai.org/proceedings/2022/325","pdf":"https://ijcai.org/proceedings/2022/0325.pdf","markdown":"https://ijcai.org/proceedings/2022/325.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/325","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":325,"pages":{"first":2341,"last":2347},"keywords":["Machine Learning: Self-supervised Learning","Knowledge Representation and Reasoning: Reasoning about Knowledge and Belief","Data Mining: Knowledge Graphs and Knowledge Base Completion","Knowledge Representation and Reasoning: Learning and reasoning","Machine Learning: Representation learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986988","length_sec":89,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985131","length_sec":706,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bollacker et al., 2008] Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, Tim Sturge, and Jamie Taylor. Free- base: a collaboratively created graph database for structur- ing human knowledge. In SIGMOD, 2008.","title":"Freebase: a collaboratively created graph database for structuring human knowledge","authors":"Kurt D Bollacker, Colin Evans, Praveen K Paritosh, Tim Sturge, Jamie Taylor","year":"2008","venue":"SIGMOD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NeurIPS, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021] Jiajun Chen, Huarui He, Feng Wu, and Jie Wang. Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In AAAI, 2021.","title":"Topology-aware correlations between relations for inductive link prediction in knowledge graphs","authors":"Jiajun Chen, Huarui He, Feng Wu, Jie Wang","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, C ¸aglar Gülc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, 2014.","title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dettmers et al., 2018] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In AAAI, 2018.","title":"Convolutional 2d knowledge graph embeddings","authors":"Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In NAACL, 2019.","title":"BERT: pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In ICML, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, George E Dahl","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross B. Girshick. Momentum contrast for un- supervised visual representation learning. In CVPR, 2020.","title":"Momentum contrast for unsupervised visual representation learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross B Girshick","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2019] Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. Knowledge graph em- bedding based question answering. In WSDM, 2019.","title":"Knowledge graph embedding based question answering","authors":"Xiao Huang, Jingyuan Zhang, Dingcheng Li, Ping Li","year":"2019","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liang et al., 2021] Shuang Liang, Jie Shao, Dongyang Zhang, Jiasheng Zhang, and Cui. Drgi: Deep relational graph infomax for knowledge graph completion. IEEE Transactions on Knowledge and Data Engineering, 2021.","title":"Drgi: Deep relational graph infomax for knowledge graph completion","authors":"Shuang Liang, Jie Shao, Dongyang Zhang, Jiasheng Zhang, Cui","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mai et al., 2021] Sijie Mai, Shuangjia Zheng, Yuedong Yang, and Haifeng Hu. Communicative message passing for inductive relation reasoning. In AAAI, 2021.","title":"Communicative message passing for inductive relation reasoning","authors":"Sijie Mai, Shuangjia Zheng, Yuedong Yang, Haifeng Hu","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nickel et al., 2012] Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. Factorizing YAGO: scalable machine learning for linked data. In WWW, 2012.","title":"Factorizing YAGO: scalable machine learning for linked data","authors":"Maximilian Nickel, Hans-Peter Volker Tresp, Kriegel","year":"2012","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Qiu et al., 2020] Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. GCC: graph contrastive coding for graph neural network pre-training. In KDD, 2020.","title":"GCC: graph contrastive coding for graph neural network pre-training","authors":"Qiu","year":"2020","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sadeghian et al., 2019] Ali Sadeghian, Mohammadreza Ar- mandpour, Patrick Ding, and Daisy Zhe Wang. DRUM: end-to-end differentiable rule mining on knowledge graphs. In NeurIPS, 2019.","title":"DRUM: end-to-end differentiable rule mining on knowledge graphs","authors":"Ali Sadeghian, Mohammadreza Armandpour, Patrick Ding, Daisy Zhe Wang","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Schlichtkrull et al., 2018] Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In ESWC, 2018.","title":"Modeling relational data with graph convolutional networks","authors":"Sejr Michael, Thomas N Schlichtkrull, Peter Kipf, Rianne Bloem, Van Den, Ivan Berg, Max Titov, Welling","year":"2018","venue":"ESWC","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2021] Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Yuanxing Ning, Philip S. Yu, and Lifang He. SUGAR: subgraph neural network with reinforcement pooling and self-supervised mutual information mecha- nism. In WWW, 2021.","title":"SUGAR: subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism","authors":"Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Yuanxing Ning, Philip S Yu, Lifang He","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Teru et al., 2020] Komal K. Teru, Etienne Denis, and Will Hamilton. Inductive relation prediction by subgraph rea- soning. In ICML, 2020.","title":"Inductive relation prediction by subgraph reasoning","authors":"K Komal, Etienne Teru, Will Denis, Hamilton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Toutanova et al., 2015] Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In EMNLP, 2015.","title":"Representing text for joint embedding of text and knowledge bases","authors":"Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, Michael Gamon","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Trivedi et al., 2017] Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. Know-evolve: Deep temporal rea- soning for dynamic knowledge graphs. In ICML, 2017.","title":"Know-evolve: Deep temporal reasoning for dynamic knowledge graphs","authors":"Rakshit Trivedi, Hanjun Dai, Yichen Wang, Le Song","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vashishth et al., 2020] Shikhar Vashishth, Soumya Sanyal, and Vikram Nitin. Composition-based multi-relational graph convolutional networks. In ICLR, 2020.","title":"Composition-based multi-relational graph convolutional networks","authors":"Shikhar Vashishth, Soumya Sanyal, Vikram Nitin","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Velickovic et al., 2019] Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. Deep graph infomax. In ICLR, 2019.","title":"Deep graph infomax","authors":"Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, R, Devon Hjelm","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vrandecic, 2012] Denny Vrandecic. Wikidata: a new plat- form for collaborative data collection. In WWW, 2012.","title":"Wikidata: a new platform for collaborative data collection","authors":"Denny Vrandecic, Vrandecic","year":"2012","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In AAAI, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018] Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In CIKM, 2018.","title":"Ripplenet: Propagating user preferences on the knowledge graph for recommender systems","authors":"Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, Minyi Guo","year":"2018","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2019] Peifeng Wang, Jialong Han, and Chen- liang Li. Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In AAAI, 2019.","title":"Logic attention based neighborhood aggregation for inductive knowledge graph embedding","authors":"Wang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2015] Bishan Yang, Wen-tau Yih, Xiaodong He, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In ICLR, 2015.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen-Tau Yih, Xiaodong He, Li Deng","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2017] Fan Yang, Zhilin Yang, and William W. Cohen. Differentiable learning of logical rules for knowl- edge base reasoning. In NeurIPS, 2017.","title":"Differentiable learning of logical rules for knowledge base reasoning","authors":"Fan Yang, Zhilin Yang, William W Cohen","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Chen, 2018] Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. In NeurIPS, 2018.","title":"Link prediction based on graph neural networks","authors":"Chen ; ] Muhan Zhang, Yixin Zhang, Chen","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}