{"type":"ScholarlyArticle","title":"Meta-Learning Based Knowledge Extrapolation for Knowledge Graphs in the Federated Setting","authors":[{"name":"Mingyang Chen","slug":"mingyang-chen","given":"Mingyang","family":"Chen","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Wen Zhang","slug":"wen-zhang","given":"Wen","family":"Zhang","affiliation":"School of Software Technology , Zhejiang University"},{"name":"Zhen Yao","slug":"zhen-yao","given":"Zhen","family":"Yao","affiliation":"School of Software Technology , Zhejiang University"},{"name":"Xiangnan Chen","slug":"xiangnan-chen","given":"Xiangnan","family":"Chen","affiliation":"School of Software Technology , Zhejiang University"},{"name":"Mengxiao Ding","slug":"mengxiao-ding","given":"Mengxiao","family":"Ding","affiliation":"Huawei Technologies Co. , Ltd."},{"name":"Fei Huang","slug":"fei-huang","given":"Fei","family":"Huang","affiliation":"Alibaba Group"},{"name":"Huajun Chen","slug":"huajun-chen","given":"Huajun","family":"Chen","affiliation":"College of Computer Science and Technology , Zhejiang University; ZJU-Hangzhou Global Scientific and Technological Innovation Center; Alibaba-Zhejiang University Joint Institute of Frontier Technologies {mingyangchen ,"}],"abstract":"We study the knowledge extrapolation problem to embed new components (i.e., entities and relations) that come with emerging knowledge graphs (KGs) in the federated setting. In this problem, a model trained on an existing KG needs to embed an emerging KG with unseen entities and relations. To solve this problem, we introduce the meta-learning setting, where a set of tasks are sampled on the existing KG to mimic the link prediction task on the emerging KG. Based on sampled tasks, we meta-train a graph neural network framework that can construct features for unseen components based on structural information and output embeddings for them. Experimental results show that our proposed method can effectively embed unseen components and outperforms models that consider inductive settings for KGs and baselines that directly use conventional KG embedding methods.","doi":"10.24963/ijcai.2022/273","url":"https://ijcai.org/proceedings/2022/273","pdf":"https://ijcai.org/proceedings/2022/0273.pdf","markdown":"https://ijcai.org/proceedings/2022/273.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/273","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":273,"pages":{"first":1966,"last":1972},"keywords":["Data Mining: Knowledge Graphs and Knowledge Base Completion"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986858","length_sec":76,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985001","length_sec":328,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019] Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. Meta relational learning for few-shot link prediction in knowledge graphs. In EMNLP, 2019.","title":"Meta relational learning for few-shot link prediction in knowledge graphs","authors":"Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, Huajun Chen","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021a] Jiajun Chen, Huarui He, Feng Wu, and Jie Wang. Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In AAAI, pages 6271-6278. AAAI Press, 2021.","title":"Topology-aware correlations between relations for inductive link prediction in knowledge graphs","authors":"Chen","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021b] Mingyang Chen, Wen Zhang, Zong- gang Yuan, Yantao Jia, and Huajun Chen. Fede: Embed- ding knowledge graphs in federated setting. In IJCKG, pages 80-88. ACM, 2021.","title":"Fede: Embedding knowledge graphs in federated setting","authors":"Chen","year":"2021","venue":"IJCKG","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hamaguchi et al., 2017] Takuo Hamaguchi, Hidekazu Oiwa, Masashi Shimbo, and Yuji Matsumoto. Knowledge transfer for out-of-knowledge-base entities : A graph neural network approach. In IJCAI, 2017.","title":"Knowledge transfer for out-of-knowledge-base entities : A graph neural network approach","authors":"Takuo Hamaguchi, Hidekazu Oiwa, Masashi Shimbo, Yuji Matsumoto","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/250"},{"position":7,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2021] Shuwen Liu, Bernardo Cuenca Grau, Ian Horrocks, and Egor V. Kostylev. Indigo: Gnn-based in- ductive knowledge graph completion using pair-wise en- coding. In NeurIPS, 2021.","title":"Indigo: Gnn-based inductive knowledge graph completion using pair-wise encoding","authors":"Shuwen Liu, Bernardo Cuenca Grau, Ian Horrocks, Egor V Kostylev","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In AISTATS, 2017.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera Y Arcas","year":"2017","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Peng et al., 2021] Hao Peng, Haoran Li, Yangqiu Song, Vincent W. Zheng, and Jianxin Li. Differentially private federated knowledge graphs embedding. In CIKM, 2021.","title":"Differentially private federated knowledge graphs embedding","authors":"Hao Peng, Haoran Li, Yangqiu Song, Vincent W Zheng, Jianxin Li","year":"2021","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Santoro et al., 2016] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lill- icrap. Meta-learning with memory-augmented neural networks. In ICML, 2016.","title":"Meta-learning with memory-augmented neural networks","authors":"Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Schlichtkrull et al., 2018] Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convo- lutional networks. In ESWC, 2018.","title":"Modeling relational data with graph convolutional networks","authors":"Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den, Ivan Berg, Max Titov, Welling","year":"2018","venue":"ESWC","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NIPS, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sun et al., 2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR, 2019.","title":"Rotate: Knowledge graph embedding by relational rotation in complex space","authors":"Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Teru et al., 2020] Komal Teru, Etienne Denis, and Will Hamilton. Inductive relation prediction by subgraph rea- soning. In ICML, 2020.","title":"Inductive relation prediction by subgraph reasoning","authors":"Komal Teru, Etienne Denis, Will Hamilton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Toutanova et al., 2015] Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In EMNLP, 2015.","title":"Representing text for joint embedding of text and knowledge bases","authors":"Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, Michael Gamon","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Trouillon et al., 2016] Théo Trouillon, Johannes Welbl, Se- bastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In ICML, 2016.","title":"Complex embeddings for simple link prediction","authors":"Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vashishth et al., 2020] Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. Composition-based multi-relational graph convolutional networks. In ICLR, 2020.","title":"Composition-based multi-relational graph convolutional networks","authors":"Shikhar Vashishth, Soumya Sanyal, Nitin Vikram, Partha P Talukdar","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim- othy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In NIPS, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2017] Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. Knowledge graph embedding: A survey of ap- proaches and applications. IEEE Transactions on Knowl- edge and Data Engineering, 29(12):2724-2743, 2017.","title":"Knowledge graph embedding: A survey of approaches and applications","authors":"Quan Wang, Zhendong Mao, Bin Wang, Li Guo","year":"2017","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2019] Peifeng Wang, Jialong Han, Chenliang Li, and Rong Pan. Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In AAAI, 2019.","title":"Logic attention based neighborhood aggregation for inductive knowledge graph embedding","authors":"Wang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xiong et al., 2017] Wenhan Xiong, Thien Hoang, and William Yang Wang. Deeppath: A reinforcement learn- ing method for knowledge graph reasoning. In EMNLP, 2017.","title":"Deeppath: A reinforcement learning method for knowledge graph reasoning","authors":"Xiong","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiong et al., 2018] Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. One-shot rela- tional learning for knowledge graphs. In EMNLP, 2018.","title":"One-shot relational learning for knowledge graphs","authors":"Xiong","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2015] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In ICLR, 2015.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, Li Deng","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yasunaga et al., 2021] Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. QA- GNN: reasoning with language models and knowledge graphs for question answering. In NAACL-HLT, 2021.","title":"QA-GNN: reasoning with language models and knowledge graphs for question answering","authors":"Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, Jure Leskovec","year":"2021","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2016] Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. Collaborative knowledge base embedding for recommender systems. In KDD, 2016.","title":"Collaborative knowledge base embedding for recommender systems","authors":"Zhang","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2021] Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu Ming Yiu. Subgraph federated learn- ing with missing neighbor generation. In NeurIPS, 2021.","title":"Subgraph federated learning with missing neighbor generation","authors":"Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, Siu Ming, Yiu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2021] Qi Zhu, Yidan Xu, Haonan Wang, Chao Zhang, Jiawei Han, and Carl Yang. Transfer learning of graph neural networks with ego-graph information maxi- mization. In NeurIPS, 2021.","title":"Transfer learning of graph neural networks with ego-graph information maximization","authors":"Qi Zhu, Yidan Xu, Haonan Wang, Chao Zhang, Jiawei Han, Carl Yang","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}