{"type":"ScholarlyArticle","title":"Local and Global: Temporal Question Answering via Information Fusion","authors":[{"name":"Yonghao Liu","slug":"yonghao-liu","given":"Yonghao","family":"Liu","affiliation":"The Key Laboratory for Symbol Computation and Knowledge Engineering of the Ministry of Education , College of Computer Science and Technology , Jilin University"},{"name":"Di Liang","slug":"di-liang","given":"Di","family":"Liang","affiliation":"Center for Natural Language Processing , Meituan Inc"},{"name":"Mengyu Li","slug":"mengyu-li","given":"Mengyu","family":"Li","affiliation":"The Key Laboratory for Symbol Computation and Knowledge Engineering of the Ministry of Education , College of Computer Science and Technology , Jilin University"},{"name":"Fausto Giunchiglia","slug":"fausto-giunchiglia","given":"Fausto","family":"Giunchiglia","affiliation":"University of Trento"},{"name":"Ximing Li","slug":"ximing-li","given":"Ximing","family":"Li","affiliation":"The Key Laboratory for Symbol Computation and Knowledge Engineering of the Ministry of Education , College of Computer Science and Technology , Jilin University"},{"name":"Sirui Wang","slug":"sirui-wang","given":"Sirui","family":"Wang","affiliation":"Center for Natural Language Processing , Meituan Inc"},{"name":"Wei Wu","slug":"wei-wu","given":"Wei","family":"Wu","affiliation":"Center for Natural Language Processing , Meituan Inc"},{"name":"Lan Huang","slug":"lan-huang","given":"Lan","family":"Huang","affiliation":"The Key Laboratory for Symbol Computation and Knowledge Engineering of the Ministry of Education , College of Computer Science and Technology , Jilin University"},{"name":"Xiaoyue Feng","slug":"xiaoyue-feng","given":"Xiaoyue","family":"Feng","affiliation":"The Key Laboratory for Symbol Computation and Knowledge Engineering of the Ministry of Education , College of Computer Science and Technology , Jilin University"},{"name":"Renchu Guan","slug":"renchu-guan","given":"Renchu","family":"Guan","affiliation":"The Key Laboratory for Symbol Computation and Knowledge Engineering of the Ministry of Education , College of Computer Science and Technology , Jilin University"}],"abstract":"Many models that leverage knowledge graphs (KGs) have recently demonstrated remarkable success in question answering (QA) tasks. In the real world, many facts contained in KGs are time-constrained thus temporal KGQA has received increasing attention. Despite the fruitful efforts of previous models in temporal KGQA, they still have several limitations. (I) They neither emphasize the graph structural information between entities in KGs nor explicitly utilize a multi-hop relation path through graph neural networks to enhance answer prediction. (II) They adopt pre-trained language models (LMs) to obtain question representations, focusing merely on the global information related to the question while not highlighting the local information of the entities in KGs. To address these limitations, we introduce a novel model that simultaneously explores both Local information and Global information for the task of temporal KGQA (LGQA). Specifically, we first introduce an auxiliary task in the temporal KG embedding procedure to make timestamp embeddings time-order aware. Then, we design information fusion layers that effectively incorporate local and global information to deepen question understanding. We conduct extensive experiments on two benchmarks, and LGQA significantly outperforms previous state-of-the-art models, especially in difficult questions. Moreover, LGQA can generate interpretable and trustworthy predictions.","doi":"10.24963/ijcai.2023/571","url":"https://ijcai.org/proceedings/2023/571","pdf":"https://ijcai.org/proceedings/2023/0571.pdf","markdown":"https://ijcai.org/proceedings/2023/571.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/571","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Natural Language Processing","track":"Main Track","number":571,"pages":{"first":5141,"last":5149},"keywords":["Natural Language Processing: NLP: Question answering"],"videos":[],"references":[{"position":1,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NeurIPS, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Feng et al., 2020] Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan, and Xiang Ren. Scalable multi-hop relational reasoning for knowledge-aware ques- tion answering. In EMNLP, 2020.","title":"Scalable multi-hop relational reasoning for knowledge-aware question answering","authors":"Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan, Xiang Ren","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Févry et al., 2020] Thibault Févry, Livio Baldini Soares, Nicholas Fitzgerald, Eunsol Choi, and Tom Kwiatkowski. Entities as experts: Sparse memory access with entity su- pervision. In EMNLP, 2020.","title":"Entities as experts: Sparse memory access with entity supervision","authors":"Thibault Févry, Baldini Livio, Nicholas Soares, Eunsol Fitzgerald, Tom Choi, Kwiatkowski","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang et al., 2019] Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. Knowledge graph em- bedding based question answering. In WSDM, 2019.","title":"Knowledge graph embedding based question answering","authors":"Xiao Huang, Jingyuan Zhang, Dingcheng Li, Ping Li","year":"2019","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jia et al., 2021] Zhen Jia, Soumajit Pramanik, Rishiraj Saha Roy, and Gerhard Weikum. Complex temporal ques- tion answering on knowledge graphs. In CIKM, 2021.","title":"Complex temporal question answering on knowledge graphs","authors":"Zhen Jia, Soumajit Pramanik, Rishiraj Saha Roy, Gerhard Weikum","year":"2021","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jiao et al., 2022] Songlin Jiao, Zhenfang Zhu, Wenqing Wu, Zicheng Zuo, Jiangtao Qi, Wenling Wang, Guangyuan Zhang, and Peiyu Liu. An improving reasoning network for complex question answering over temporal knowledge graphs. Applied Intelligence, pages 1-14, 2022.","title":"An improving reasoning network for complex question answering over temporal knowledge graphs","authors":"Songlin Jiao, Zhenfang Zhu, Wenqing Wu, Zicheng Zuo, Jiangtao Qi, Wenling Wang, Guangyuan Zhang, Peiyu Liu","year":"2022","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lacroix et al., 2020] Timothée Lacroix, Guillaume Obozin- ski, and Nicolas Usunier. Tensor decompositions for tem- poral knowledge base completion. In ICLR, 2020.","title":"Tensor decompositions for temporal knowledge base completion","authors":"Timothée Lacroix, Guillaume Obozinski, Nicolas Usunier","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liang et al., 2019a] Di Liang, Fubao Zhang, Qi Zhang, and Xuan-Jing Huang. Asynchronous deep interaction net- work for natural language inference. In EMNLP-IJCNLP, 2019.","title":"Asynchronous deep interaction network for natural language inference","authors":"Liang","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liang et al., 2019b] Di Liang, Fubao Zhang, Weidong Zhang, Qi Zhang, Jinlan Fu, Minlong Peng, Tao Gui, and Xuanjing Huang. Adaptive multi-attention network incor- porating answer information for duplicate question detec- tion. In SIGIR, 2019.","title":"Adaptive multi-attention network incorporating answer information for duplicate question detection","authors":"Liang","year":"2019","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2021a] Yonghao Liu, Renchu Guan, Xiaoyue Feng, and Ximing Li. Vpalg: Paper-publication predic- tion with graph neural networks. In CIKM, 2021.","title":"Vpalg: Paper-publication prediction with graph neural networks","authors":"Liu","year":"2021","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2021b] Yonghao Liu, Renchu Guan, Fausto Giunchiglia, Yanchun Liang, and Xiaoyue Feng. Deep attention diffusion graph neural networks for text classi- fication. In EMNLP, 2021.","title":"Deep attention diffusion graph neural networks for text classification","authors":"Liu","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2022] Yonghao Liu, Mengyu Li, Ximing Li, Fausto Giunchiglia, Xiaoyue Feng, and Renchu Guan. Few-shot node classification on attributed networks with graph meta-learning. In SIGIR, 2022.","title":"Few-shot node classification on attributed networks with graph meta-learning","authors":"Yonghao Liu, Mengyu Li, Ximing Li, Fausto Giunchiglia, Xiaoyue Feng, Renchu Guan","year":"2022","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2023] Yonghao Liu, Di Liang, Fang Fang, Sirui Wang, Wei Wu, and Rui Jiang. Time-aware multiway adaptive fusion network for temporal kgqa. In ICASSP, 2023.","title":"Time-aware multiway adaptive fusion network for temporal kgqa","authors":"Yonghao Liu, Di Liang, Fang Fang, Sirui Wang, Wei Wu, Rui Jiang","year":"2023","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lukovnikov et al., 2017] Denis Lukovnikov, Asja Fischer, Jens Lehmann, and Sören Auer. Neural network-based question answering over knowledge graphs on word and character level. In The Web Conference, 2017.","title":"Neural network-based question answering over knowledge graphs on word and character level","authors":"Denis Lukovnikov, Asja Fischer, Jens Lehmann, Sören Auer","year":"2017","venue":"The Web Conference","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mavromatis et al., 2021] Costas Mavromatis, Prasanna Lakkur Subramanyam, Vassilis N Ioanni- dis, Soji Adeshina, Phillip R Howard, Tetiana Grinberg, Nagib Hakim, and George Karypis. Tempoqr: Temporal question reasoning over knowledge graphs. In AAAI, 2021.","title":"Tempoqr: Temporal question reasoning over knowledge graphs","authors":"Costas Mavromatis, Lakkur Prasanna, Subramanyam, N Vassilis, Soji Ioannidis, Adeshina, Tetiana Phillip R Howard, Nagib Grinberg, George Hakim, Karypis","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Peters et al., 2019] Matthew E Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A Smith. Knowledge enhanced contextual word representations. In EMNLP-IJCNLP, 2019.","title":"Knowledge enhanced contextual word representations","authors":"Mark Matthew E Peters, Robert Neumann, Roy Logan, Vidur Schwartz, Sameer Joshi, Noah A Singh, Smith","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ren et al., 2020] Hongyu Ren, Weihua Hu, and Jure Leskovec. Query2box: Reasoning over knowledge graphs in vector space using box embeddings. In ICLR, 2020.","title":"Query2box: Reasoning over knowledge graphs in vector space using box embeddings","authors":"Weihua Hongyu Ren, Jure Hu, Leskovec","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Saxena et al., 2020] Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. Improving multi-hop question answer- ing over knowledge graphs using knowledge base embed- dings. In ACL, 2020.","title":"Improving multi-hop question answering over knowledge graphs using knowledge base embeddings","authors":"Apoorv Saxena, Aditay Tripathi, Partha Talukdar","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Saxena et al., 2021] Apoorv Saxena, Soumen Chakrabarti, and Partha Talukdar. Question answering over temporal knowledge graphs. In ACL-IJCNLP, 2021.","title":"Question answering over temporal knowledge graphs","authors":"Apoorv Saxena, Soumen Chakrabarti, Partha Talukdar","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shang et al., 2022] Chao Shang, Guangtao Wang, Peng Qi, and Jing Huang. Improving time sensitivity for question answering over temporal knowledge graphs. In ACL, 2022.","title":"Improving time sensitivity for question answering over temporal knowledge graphs","authors":"Chao Shang, Guangtao Wang, Peng Qi, Jing Huang","year":"2022","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Trouillon et al., 2017] Théo Trouillon, Christopher R Dance, Éric Gaussier, Johannes Welbl, Sebastian Riedel, and Guillaume Bouchard. Knowledge graph completion via complex tensor factorization. JMLR, 2017.","title":"Knowledge graph completion via complex tensor factorization","authors":"Théo Trouillon, Éric Christopher R Dance, Johannes Gaussier, Sebastian Welbl, Guillaume Riedel, Bouchard","year":"2017","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yasunaga et al., 2021] Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. Qa- gnn: Reasoning with language models and knowledge graphs for question answering. In NAACL, 2021.","title":"Qagnn: Reasoning with language models and knowledge graphs for question answering","authors":"Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, Jure Leskovec","year":"2021","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2018] Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J Smola, and Le Song. Variational reasoning for question answering with knowledge graph. In AAAI, 2018.","title":"Variational reasoning for question answering with knowledge graph","authors":"Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J Smola, Le Song","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}