{"type":"ScholarlyArticle","title":"Modeling Precursors for Temporal Knowledge Graph Reasoning via Auto-encoder Structure","authors":[{"name":"Yifu Gao","slug":"yifu-gao","given":"Yifu","family":"Gao","affiliation":"Science and Technology on Parallel and Distributed Processing Laboratory , National University of Defense Technology , Changsha , China"},{"name":"Linhui Feng","slug":"linhui-feng","given":"Linhui","family":"Feng","affiliation":"Science and Technology on Parallel and Distributed Processing Laboratory , National University of Defense Technology , Changsha , China"},{"name":"Zhigang Kan","slug":"zhigang-kan","given":"Zhigang","family":"Kan","affiliation":"Science and Technology on Parallel and Distributed Processing Laboratory , National University of Defense Technology , Changsha , China"},{"name":"Yi Han","slug":"yi-han","given":"Yi","family":"Han","affiliation":"College of Meteorology and Oceanography , National University of Defense Technology {gaoyifu , linhuifeng ,"},{"name":"Linbo Qiao","slug":"linbo-qiao","given":"Linbo","family":"Qiao","affiliation":"Science and Technology on Parallel and Distributed Processing Laboratory , National University of Defense Technology , Changsha , China"},{"name":"Dongsheng Li","slug":"dongsheng-li","given":"Dongsheng","family":"Li","affiliation":"Science and Technology on Parallel and Distributed Processing Laboratory , National University of Defense Technology , Changsha , China"}],"abstract":"Temporal knowledge graph (TKG) reasoning that infers missing facts in the future is an essential and challenging task. When predicting a future event, there must be a narrative evolutionary process composed of closely related historical facts to support the event's occurrence, namely fact precursors. However, most existing models employ a sequential reasoning process in an auto-regressive manner, which cannot capture precursor information. This paper proposes a novel auto-encoder architecture that introduces a relation-aware graph attention layer into transformer (rGalT) to accommodate inference over the TKG. Specifically, we first calculate the correlation between historical and predicted facts through multiple attention mechanisms along intra-graph and inter-graph dimensions, then constitute these mutually related facts into diverse fact segments. Next, we borrow the translation generation idea to decode in parallel the precursor information associated with the given query, which enables our model to infer future unknown facts by progressively generating graph structures. Experimental results on four benchmark datasets demonstrate that our model outperforms other state-of-the-art methods, and precursor identification provides supporting evidence for prediction.","doi":"10.24963/ijcai.2022/284","url":"https://ijcai.org/proceedings/2022/284","pdf":"https://ijcai.org/proceedings/2022/0284.pdf","markdown":"https://ijcai.org/proceedings/2022/284.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/284","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":284,"pages":{"first":2044,"last":2051},"keywords":["Data Mining: Knowledge Graphs and Knowledge Base Completion","Knowledge Representation and Reasoning: Learning and reasoning","Machine Learning: Representation learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986569","length_sec":78,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984712","length_sec":353,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boschee et al., 2015] Elizabeth Boschee, Jennifer Lauten- schlager, Sean O'Brien, Steve Shellman, James Starz, and Michael Ward. Icews coded event data. Harvard Data- verse, 12, 2015.","title":"Icews coded event data","authors":"Elizabeth Boschee, Jennifer Lautenschlager, O' Sean, Steve Brien, James Shellman, Michael Starz, Ward","year":"2015","venue":"Harvard Dataverse","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, C ¸aglar Gülc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, pages 1724-1734, 2014.","title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dasgupta et al., 2018] Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha P. Talukdar. Hyte: Hyperplane-based temporally aware knowledge graph embedding. In EMNLP, pages 2001-2011, 2018.","title":"Hyte: Hyperplane-based temporally aware knowledge graph embedding","authors":"Swayambhu Nath Shib Sankar Dasgupta, Partha P Ray, Talukdar","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dettmers et al., 2018] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In AAAI, 2018.","title":"Convolutional 2d knowledge graph embeddings","authors":"Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Galárraga et al., 2015] Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. Fast rule mining in ontological knowledge bases with AMIE+. VLDB J., 24(6):707-730, 2015.","title":"Fast rule mining in ontological knowledge bases with AMIE+","authors":"Luis Galárraga, Christina Teflioudi, Katja Hose, Fabian M Suchanek","year":"2015","venue":"VLDB J","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[García-Durán et al., 2018] Alberto García-Durán, Sebasti- jan Dumancic, and Mathias Niepert. Learning sequence encoders for temporal knowledge graph completion. In EMNLP, pages 4816-4821, 2018.","title":"Learning sequence encoders for temporal knowledge graph completion","authors":"Alberto García-Durán, Sebastijan Dumancic, Mathias Niepert","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2018] Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. Knowledge graph embedding with iterative guidance from soft rules. In AAAI, pages 4816- 4823, 2018.","title":"Knowledge graph embedding with iterative guidance from soft rules","authors":"Shu Guo, Quan Wang, Lihong Wang, Bin Wang, Li Guo","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Han et al., 2021] Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp. Explainable subgraph reasoning for fore- casting on temporal knowledge graphs. In ICLR, 2021.","title":"Explainable subgraph reasoning for forecasting on temporal knowledge graphs","authors":"Zhen Han, Peng Chen, Yunpu Ma, Volker Tresp","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2021] Yongquan He, Peng Zhang, Luchen Liu, Qi Liang, Wenyuan Zhang, and Chuang Zhang. Hip net- work: Historical information passing network for extrapo- lation reasoning on temporal knowledge graph. In IJCAI, 2021.","title":"Hip network: Historical information passing network for extrapolation reasoning on temporal knowledge graph","authors":"Yongquan He, Peng Zhang, Luchen Liu, Qi Liang, Wenyuan Zhang, Chuang Zhang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/264"},{"position":11,"raw":"[Jiang et al., 2016] Tingsong Jiang, Tianyu Liu, Tao Ge, Lei Sha, Baobao Chang, Sujian Li, and Zhifang Sui. Towards time-aware knowledge graph completion. In COLING, pages 1715-1724, 2016.","title":"Towards time-aware knowledge graph completion","authors":"Jiang","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jin et al., 2020] Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In EMNLP, pages 6669-6683, 2020.","title":"Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs","authors":"Jin","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Leetaru and Schrodt, 2013] Kalev Leetaru and Philip A Schrodt. Gdelt: Global data on events, location, and tone. In ISA Annual Convention, 2013.","title":"Gdelt: Global data on events, location, and tone","authors":"Schrodt Leetaru, Kalev Leetaru, Philip A Schrodt","year":"2013","venue":"ISA Annual Convention","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2021a] Zixuan Li, Xiaolong Jin, Saiping Guan, Wei Li, Jiafeng Guo, Yuanzhuo Wang, and Xueqi Cheng. Search from history and reason for future: Two-stage rea- soning on temporal knowledge graphs. In ACL-IJCNLP, 2021.","title":"Search from history and reason for future: Two-stage reasoning on temporal knowledge graphs","authors":"Li","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2021b] Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. Temporal knowledge graph reasoning based on evolutional representation learning. In SIGIR, 2021.","title":"Temporal knowledge graph reasoning based on evolutional representation learning","authors":"Li","year":"2021","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mahdisoltani et al., 2015] Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. YAGO3: A knowledge base from multilingual wikipedias. In CIDR, 2015.","title":"YAGO3: A knowledge base from multilingual wikipedias","authors":"Farzaneh Mahdisoltani, Joanna Biega, Fabian M Suchanek","year":"2015","venue":"CIDR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Schlichtkrull et al., 2018] Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In ESWC, 2018.","title":"Modeling relational data with graph convolutional networks","authors":"Sejr Michael, Thomas N Schlichtkrull, Peter Kipf, Rianne Bloem, Van Den, Ivan Berg, Max Titov, Welling","year":"2018","venue":"ESWC","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sun et al., 2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR, 2019.","title":"Rotate: Knowledge graph embedding by relational rotation in complex space","authors":"Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Trivedi et al., 2017] Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. Know-evolve: Deep temporal rea- soning for dynamic knowledge graphs. In ICML, pages 3462-3471, 2017.","title":"Know-evolve: Deep temporal reasoning for dynamic knowledge graphs","authors":"Rakshit Trivedi, Hanjun Dai, Yichen Wang, Le Song","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Trivedi et al., 2019] Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. In ICLR, 2019.","title":"Dyrep: Learning representations over dynamic graphs","authors":"Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, Hongyuan Zha","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Trouillon et al., 2016] Théo Trouillon, Johannes Welbl, Se- bastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In ICML, pages 2071-2080, 2016.","title":"Complex embeddings for simple link prediction","authors":"Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiong et al., 2017] Wenhan Xiong, Thien Hoang, and William Yang Wang. Deeppath: A reinforcement learn- ing method for knowledge graph reasoning. In EMNLP, pages 564-573, 2017.","title":"Deeppath: A reinforcement learning method for knowledge graph reasoning","authors":"Xiong","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2015] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In ICLR, 2015.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, Li Deng","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2019] Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, and Huajun Chen. Iteratively learning embeddings and rules for knowledge graph reasoning. In WWWC, pages 2366-2377, 2019.","title":"Iteratively learning embeddings and rules for knowledge graph reasoning","authors":"Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, Huajun Chen","year":"2019","venue":"WWWC","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2021] Cunchao Zhu, Muhao Chen, Changjun Fan, Guangquan Cheng, and Yan Zhan. Learning from his- tory: Modeling temporal knowledge graphs with sequen- tial copy-generation networks. In AAAI, 2021.","title":"Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks","authors":"Zhu","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}