{"type":"ScholarlyArticle","title":"Multi-hop Reading Comprehension across Documents with Path-based Graph Convolutional Network","authors":[{"name":"Zeyun Tang","slug":"zeyun-tang","given":"Zeyun","family":"Tang","affiliation":"College of Computer Science and Technology , Zhejiang University , China"},{"name":"Yongliang Shen","slug":"yongliang-shen","given":"Yongliang","family":"Shen","affiliation":"College of Computer Science and Technology , Zhejiang University , China"},{"name":"Xinyin Ma","slug":"xinyin-ma","given":"Xinyin","family":"Ma","affiliation":"College of Computer Science and Technology , Zhejiang University , China"},{"name":"Wei Xu","slug":"wei-xu","given":"Wei","family":"Xu","affiliation":"College of Computer Science and Technology , Zhejiang University , China"},{"name":"Jiale Yu","slug":"jiale-yu","given":"Jiale","family":"Yu","affiliation":"College of Computer Science and Technology , Zhejiang University , China"},{"name":"Weiming Lu","slug":"weiming-lu","given":"Weiming","family":"Lu","affiliation":"College of Computer Science and Technology , Zhejiang University , China"}],"abstract":"Multi-hop reading comprehension across multiple documents attracts much attentions recently. In this paper, we propose a novel approach to tackle this multi-hop reading comprehension problem. Inspired by the human reasoning processing, we introduce a path-based graph with reasoning paths which extracted from supporting documents. The path-based graph can combine both the idea of the graph-based  and path-based approaches, so it is better for multi-hop reasoning. Meanwhile, we propose Gated-GCN to accumulate evidences on the path-based graph, which contains a new question-aware gating mechanism to regulate the usefulness of information propagating across documents and add question information during reasoning. We evaluate our approach on WikiHop dataset, and our approach achieves the the-state-of-art accuracy against previous published approaches. Especially, our ensemble model surpasses the human performance by 4.2%.","doi":"10.24963/ijcai.2020/540","url":"https://ijcai.org/proceedings/2020/540","pdf":"https://ijcai.org/proceedings/2020/0540.pdf","markdown":"https://ijcai.org/proceedings/2020/540.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/540","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":540,"pages":{"first":3905,"last":3911},"keywords":["Natural Language Processing: Question Answering","Natural Language Processing: Natural Language Processing"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24336/file_24336.m3u8","length_sec":279,"provider":"hls"}],"references":[{"position":1,"raw":"[Bird, 2006] Steven Bird. Nltk: the natural language toolkit. In Proceedings of the COLING/ACL on Interactive pre- sentation sessions, pages 69-72. Association for Compu- tational Linguistics, 2006.","title":"Nltk: the natural language toolkit","authors":"Steven Bird, Bird","year":"2006","venue":"Proceedings of the COLING/ACL on Interactive presentation sessions","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2019] Yu Cao, Meng Fang, and Dacheng Tao. Bag: Bi-directional attention entity graph convolutional network for multi-hop reasoning question answering. In NAACL-HLT, 2019.","title":"Bag: Bi-directional attention entity graph convolutional network for multi-hop reasoning question answering","authors":"Yu Cao, Meng Fang, Dacheng Tao","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Das et al., 2017] Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Kr- ishnamurthy, Alexander J. Smola, and Andrew McCallum. Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. In ICLR, 2017.","title":"Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning","authors":"Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alexander J Smola, Andrew Mccallum","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[De Cao et al., 2019] Nicola De Cao, Wilker Aziz, and Ivan Titov. Question answering by reasoning across documents with graph convolutional networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Long and Short Papers), pages 2306-2317, 2019.","title":"Question answering by reasoning across documents with graph convolutional networks","authors":"Nicola De Cao, Wilker Aziz, Ivan Titov","year":"2019","venue":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dhingra et al., 2018] Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Neu- ral models for reasoning over multiple mentions using coreference. In NAACL-HLT, 2018.","title":"Neural models for reasoning over multiple mentions using coreference","authors":"Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W Cohen, Ruslan Salakhutdinov","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9:1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2017] Minghao Hu, Yuxing Peng, Zhen Huang, Xipeng Qiu, Furu Wei, and Ming Zhou. Reinforced m- nemonic reader for machine reading comprehension. In IJCAI, 2017.","title":"Reinforced mnemonic reader for machine reading comprehension","authors":"Minghao Hu, Yuxing Peng, Zhen Huang, Xipeng Qiu, Furu Wei, Ming Zhou","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/570"},{"position":9,"raw":"[Jiang et al., 2019] Yichen Jiang, N. Joshi, Yen-Chun Chen, and Mohit Bansal. Explore, propose, and assemble: An interpretable model for multi-hop reading comprehension. In ACL, 2019.","title":"Explore, propose, and assemble: An interpretable model for multi-hop reading comprehension","authors":"Yichen Jiang, N Joshi, Yen-Chun Chen, Mohit Bansal","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khashabi et al., 2018] Daniel Khashabi, Snigdha Chaturve- di, Michael Roth, Shyam Upadhyay, and Dan Roth. Look- ing beyond the surface: A challenge set for reading com- prehension over multiple sentences. In NAACL-HLT, 2018.","title":"Looking beyond the surface: A challenge set for reading comprehension over multiple sentences","authors":"Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, Dan Roth","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kociský et al., 2017] Tomás Kociský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317-328, 2017.","title":"The narrativeqa reading comprehension challenge","authors":"Tomás Kociský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, Edward Grefenstette","year":"2017","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kundu et al., 2019] Souvik Kundu, Tushar Khot, Ashish S- abharwal, and Peter Clark. Exploiting explicit paths for multi-hop reading comprehension. In ACL (1), pages 2737-2747. Association for Computational Linguistics, 2019.","title":"Exploiting explicit paths for multi-hop reading comprehension","authors":"Souvik Kundu, Tushar Khot, Ashish Sabharwal, Peter Clark","year":"2019","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lv et al., 2019] Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, and Zhiyuan Liu. Adapting meta knowledge graph information for multi-hop reasoning over few-shot relations. ArXiv, abs/1908.11513, 2019.","title":"Adapting meta knowledge graph information for multi-hop reasoning over few-shot relations","authors":"Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Manning et al., 2014] Christopher D. Manning, Mihai Sur- deanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. The stanford corenlp natural lan- guage processing toolkit. In ACL, 2014.","title":"The stanford corenlp natural language processing toolkit","authors":"D Christopher, Mihai Manning, John Surdeanu, Jenny Rose Bauer, Steven Finkel, David Bethard, Mcclosky","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mihaylov et al., 2018] Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answer- ing. In EMNLP, 2018.","title":"Can a suit of armor conduct electricity? a new dataset for open book question answering","authors":"Todor Mihaylov, Peter Clark, Tushar Khot, Ashish Sabharwal","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Peters et al., 2018] Matthew E Peters, Mark Neumann, Mo- hit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word repre- sentations. In Proceedings of NAACL-HLT, pages 2227- 2237, 2018.","title":"Deep contextualized word representations","authors":"Mark Matthew E Peters, Mohit Neumann, Matt Iyyer, Christopher Gardner, Kenton Clark, Luke Lee, Zettlemoyer","year":"2018","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Qiu et al., 2019] Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. Dynamical- ly fused graph network for multi-hop reasoning. In ACL, 2019.","title":"Dynamically fused graph network for multi-hop reasoning","authors":"Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, Yong Yu","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Seo et al., 2017] Minjoon Seo, Aniruddha Kembhavi, Al- i Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. In ICLR, 2017.","title":"Bidirectional attention flow for machine comprehension","authors":"Seo","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen et al., 2017] Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. Reasonet: Learning to stop read- ing in machine comprehension. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1047-1055. ACM, 2017.","title":"Reasonet: Learning to stop reading in machine comprehension","authors":"Shen","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Song et al., 2018] Linfeng Song, Zhiguo Wang, Mo Yu, Yue Zhang, Radu Florian, and Daniel Gildea. Exploring graph- structured passage representation for multi-hop reading comprehension with graph neural networks. ArXiv, ab- s/1809.02040, 2018.","title":"Exploring graphstructured passage representation for multi-hop reading comprehension with graph neural networks","authors":"Song","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tu et al., 2019] Ming Tu, Guangtao Wang, Jing Huang, Yun Shu Tang, Xiaodong He, and Bowen Zhou. Multi- hop reading comprehension across multiple documents by reasoning over heterogeneous graphs. In ACL, 2019.","title":"Multihop reading comprehension across multiple documents by reasoning over heterogeneous graphs","authors":"Ming Tu, Guangtao Wang, Jing Huang, Yun Shu Tang, Xiaodong He, Bowen Zhou","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Welbl et al., 2018] Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. Constructing datasets for multi-hop reading comprehension across documents. Transactions of the Association for Computational Linguistics, 6:287- 302, 2018.","title":"Constructing datasets for multi-hop reading comprehension across documents","authors":"Welbl","year":"2018","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Williams, 1992] Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine Learning, 8:229-256, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Ronald J Williams","year":"1992","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2018] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdi- nov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In EMNLP, 2018.","title":"Hotpotqa: A dataset for diverse, explainable multi-hop question answering","authors":"Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, Christopher D Manning","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhong et al., 2019] Victor Zhong, Caiming Xiong, Ni- tish Shirish Keskar, and Richard Socher. Coarse-grain fine-grain coattention network for multi-evidence question answering. In ICLR, 2019.","title":"Coarse-grain fine-grain coattention network for multi-evidence question answering","authors":"Victor Zhong, Caiming Xiong, Nitish Shirish Keskar, Richard Socher","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhuang and Wang, 2019] Yimeng Zhuang and Huadong Wang. Token-level dynamic self-attention network for multi-passage reading comprehension. In ACL, 2019.","title":"Token-level dynamic self-attention network for multi-passage reading comprehension","authors":"Wang ; ] Yimeng Zhuang, Huadong Zhuang, Wang","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}