{"type":"ScholarlyArticle","title":"Simple and Effective Relation-based Embedding Propagation for Knowledge Representation Learning","authors":[{"name":"Huijuan Wang","slug":"huijuan-wang","given":"Huijuan","family":"Wang","affiliation":""},{"name":"Siming Dai","slug":"siming-dai","given":"Siming","family":"Dai","affiliation":""},{"name":"Weiyue Su","slug":"weiyue-su","given":"Weiyue","family":"Su","affiliation":""},{"name":"Hui Zhong","slug":"hui-zhong","given":"Hui","family":"Zhong","affiliation":""},{"name":"Zeyang Fang","slug":"zeyang-fang","given":"Zeyang","family":"Fang","affiliation":""},{"name":"Zhengjie Huang","slug":"zhengjie-huang","given":"Zhengjie","family":"Huang","affiliation":""},{"name":"Shikun Feng","slug":"shikun-feng","given":"Shikun","family":"Feng","affiliation":""},{"name":"Zeyu Chen","slug":"zeyu-chen","given":"Zeyu","family":"Chen","affiliation":""},{"name":"Yu Sun","slug":"yu-sun","given":"Yu","family":"Sun","affiliation":""},{"name":"Dianhai Yu","slug":"dianhai-yu","given":"Dianhai","family":"Yu","affiliation":""}],"abstract":"Relational graph neural networks have garnered particular attention to encode graph context in knowledge graphs (KGs). Although they achieved competitive performance on small KGs, how to efficiently and effectively utilize graph context for large KGs remains an open problem. To this end, we propose the Relation-based Embedding Propagation (REP) method. It is a post-processing technique to adapt pre-trained KG embeddings with graph context. As relations in KGs are directional, we model the incoming head context and the outgoing tail context separately. Accordingly, we design relational context functions with no external parameters. Besides, we use averaging to aggregate context information, making REP more computation-efficient. We theoretically prove that such designs can avoid information distortion during propagation. Extensive experiments also demonstrate that REP has significant scalability while improving or maintaining prediction quality. Particularly, it averagely brings about 10% relative improvement to triplet-based embedding methods on OGBL-WikiKG2 and takes 5%-83% time to achieve comparable results as the state-of-the-art GC-OTE.","doi":"10.24963/ijcai.2022/382","url":"https://ijcai.org/proceedings/2022/382","pdf":"https://ijcai.org/proceedings/2022/0382.pdf","markdown":"https://ijcai.org/proceedings/2022/382.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/382","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":382,"pages":{"first":2755,"last":2761},"keywords":["Machine Learning: Representation learning","Knowledge Representation and Reasoning: Applications","Knowledge Representation and Reasoning: Learning and reasoning","Knowledge Representation and Reasoning: Semantic Web","Knowledge Representation and Reasoning: Other"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986457","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984600","length_sec":324,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bian et al., 2021] Ning Bian, Xianpei Han, Bo Chen, and Le Sun. Benchmarking knowledge-enhanced common- sense question answering via knowledge-to-text transfor- mation. In AAAI, pages 12574-12582. AAAI Press, 2021.","title":"Benchmarking knowledge-enhanced commonsense question answering via knowledge-to-text transformation","authors":"Xianpei Ning Bian, Bo Han, Le Chen, Sun","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NeurIPS, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2019] Ling Cai, Bo Yan, Gengchen Mai, Krzysztof Janowicz, and Rui Zhu. Transgcn: Coupling transformation assumptions with graph convolutional net- works for link prediction. In K-CAP, pages 131-138, 2019.","title":"Transgcn: Coupling transformation assumptions with graph convolutional networks for link prediction","authors":"Ling Cai, Bo Yan, Gengchen Mai, Krzysztof Janowicz, Rui Zhu","year":"2019","venue":"K-CAP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chao et al., 2021] Linlin Chao, Jianshan He, Taifeng Wang, and Wei Chu. Pairre: Knowledge graph embeddings via paired relation vectors. In ACL/IJCNLP, pages 4360- 4369, 2021.","title":"Pairre: Knowledge graph embeddings via paired relation vectors","authors":"Linlin Chao, Jianshan He, Taifeng Wang, Wei Chu","year":"2021","venue":"ACL/IJCNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dettmers et al., 2018] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In AAAI, pages 1811-1818, 2018.","title":"Convolutional 2d knowledge graph embeddings","authors":"Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neu- ral message passing for quantum chemistry. In ICML, vol- ume 70, pages 1263-1272, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, George E Dahl","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hu et al., 2020] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.","title":"Open graph benchmark: Datasets for machine learning on graphs","authors":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, Jure Leskovec","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2021] Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. Ogb-lsc: A large-scale challenge for machine learning on graphs. arXiv preprint arXiv:2103.09430, 2021.","title":"Ogb-lsc: A large-scale challenge for machine learning on graphs","authors":"Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, Jure Leskovec","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2021] Qian Huang, Horace He, Abhay Singh, Ser-Nam Lim, and Austin R. Benson. Combining label propagation and simple models out-performs graph neural networks. In ICLR, 2021.","title":"Combining label propagation and simple models out-performs graph neural networks","authors":"Qian Huang, Horace He, Abhay Singh, Ser-Nam Lim, Austin R Benson","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Joseph and Jiang, 2019] Kevin Joseph and Hui Jiang. Con- tent based news recommendation via shortest entity dis- tance over knowledge graphs. In WWW, pages 690-699, 2019.","title":"Content based news recommendation via shortest entity distance over knowledge graphs","authors":"Kevin Joseph, Hui Jiang","year":"2019","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Klicpera et al., 2019] Johannes Klicpera, Aleksandar Bo- jchevski, and Stephan Günnemann. Predict then propa- gate: Graph neural networks meet personalized pagerank. In ICLR, 2019.","title":"Predict then propagate: Graph neural networks meet personalized pagerank","authors":"Klicpera","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Saxena et al., 2020] Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. Improving multi-hop question answer- ing over knowledge graphs using knowledge base embed- dings. In ACL, pages 4498-4507, 2020.","title":"Improving multi-hop question answering over knowledge graphs using knowledge base embeddings","authors":"Apoorv Saxena, Aditay Tripathi, Partha Talukdar","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Schlichtkrull et al., 2018] Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In ESWC, volume 10843, pages 593-607, 2018.","title":"Modeling relational data with graph convolutional networks","authors":"Sejr Michael, Thomas N Schlichtkrull, Peter Kipf, Rianne Bloem, Van Den, Ivan Berg, Max Titov, Welling","year":"2018","venue":"ESWC","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shang et al., 2019] Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, and Bowen Zhou. End-to- end structure-aware convolutional networks for knowledge base completion. In AAAI, pages 3060-3067, 2019.","title":"End-toend structure-aware convolutional networks for knowledge base completion","authors":"Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, Bowen Zhou","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sun et al., 2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR, 2019.","title":"Rotate: Knowledge graph embedding by relational rotation in complex space","authors":"Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tang et al., 2020] Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, and Bowen Zhou. Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In ACL, pages 2713-2722, 2020.","title":"Orthogonal relation transforms with graph context modeling for knowledge graph embedding","authors":"Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, Bowen Zhou","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Trouillon et al., 2016] Théo Trouillon, Johannes Welbl, Se- bastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In ICML, pages 2071-2080, 2016.","title":"Complex embeddings for simple link prediction","authors":"Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vashishth et al., 2020] Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. Composition-based multi-relational graph convolutional networks. In ICLR, 2020.","title":"Composition-based multi-relational graph convolutional networks","authors":"Shikhar Vashishth, Soumya Sanyal, Nitin Vikram, Partha P Talukdar","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2021] Shen Wang, Xiaokai Wei, Ci- cero Nogueira dos Santos, Zhiguo Wang, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang, Philip S. Yu, and Isabel F. Cruz. Mixed-curvature multi-relational graph neural network for knowledge graph completion. In WWW, pages 1761-1771, 2021.","title":"Mixed-curvature multi-relational graph neural network for knowledge graph completion","authors":"Shen Wang, Xiaokai Wei, Cicero Nogueira Dos Santos, Zhiguo Wang, Ramesh Nallapati, Andrew O Arnold, Bing Xiang, Philip S Yu, Isabel F Cruz","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2019] Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Wein- berger. Simplifying graph convolutional networks. In ICML, volume 97, pages 6861-6871, 2019.","title":"Simplifying graph convolutional networks","authors":"Felix Wu, Amauri H Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, Kilian Q Weinberger","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2015] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In ICLR, 2015.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, Li Deng","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2019] Shuai Zhang, Yi Tay, Lina Yao, and Qi Liu. Quaternion knowledge graph embeddings. In NeurIPS, pages 2731-2741, 2019.","title":"Quaternion knowledge graph embeddings","authors":"Shuai Zhang, Yi Tay, Lina Yao, Qi Liu","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}