{"type":"ScholarlyArticle","title":"Towards Reading Comprehension for Long Documents","authors":[{"name":"Yuanxing Zhang","slug":"yuanxing-zhang","given":"Yuanxing","family":"Zhang","affiliation":"Peking University , China"},{"name":"Yangbin Zhang","slug":"yangbin-zhang","given":"Yangbin","family":"Zhang","affiliation":"Peking University , China"},{"name":"Kaigui Bian","slug":"kaigui-bian","given":"Kaigui","family":"Bian","affiliation":"Peking University , China"},{"name":"Xiaoming Li","slug":"xiaoming-li","given":"Xiaoming","family":"Li","affiliation":"Peking University , China"}],"abstract":"Machine reading comprehension has gained attention from both industry and academia. It is a very challenging task that involves various domains such as language comprehension, knowledge inference, summarization, etc. Previous studies mainly focus on reading comprehension on short paragraphs, and these approaches fail to perform well on the documents. In this paper, we propose a hierarchical match attention model to instruct the machine to extract answers from a specific short span of passages for the long document reading comprehension (LDRC) task. The model takes advantages from hierarchical-LSTM to learn the paragraph-level representation, and implements the match mechanism (i.e., quantifying the relationship between two contexts) to find the most appropriate paragraph that includes the hint of answers. Then the task can be decoupled into reading comprehension task for short paragraph, such that the answer can be produced. Experiments on the modified SQuAD dataset show that our proposed model outperforms existing reading comprehension models by at least 20% regarding exact match (EM), F1 and the proportion of identified paragraphs which are exactly the short paragraphs where the original answers locate.","doi":"10.24963/ijcai.2018/638","url":"https://ijcai.org/proceedings/2018/638","pdf":"https://ijcai.org/proceedings/2018/0638.pdf","markdown":"https://ijcai.org/proceedings/2018/638.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/638","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":638,"pages":{"first":4588,"last":4594},"keywords":["Natural Language Processing: Question Answering","Natural Language Processing: Natural Language Processing","Natural Language Processing: Coreference Resolution"],"videos":[],"references":[{"position":1,"raw":"[Botha et al., 2017] Jan A. Botha, Emily Pitler, Ji Ma, An- ton Bakalov, Alex Salcianu, David Weiss, Ryan Mc- Donald, and Slav Petrov. Natural language processing with small feed-forward networks. In Proceedings of the 2017 EMNLP, pages 2879-2885, Copenhagen, Denmark, September 2017. ACL.","title":"Natural language processing with small feed-forward networks","authors":"Jan A Botha, Emily Pitler, Ji Ma, Anton Bakalov, Alex Salcianu, David Weiss, Ryan Mc-Donald, Slav Petrov","year":"2017","venue":"Proceedings of the 2017 EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cui et al., 2017] Yiming Cui, Zhipeng Chen, Si Wei, Shi- jin Wang, Ting Liu, and Guoping Hu. Attention-over- attention neural networks for reading comprehension. In Proceedings of the 55th ACL (Volume 1: Long Papers), pages 593-602, Vancouver, Canada, July 2017. ACL.","title":"Attention-overattention neural networks for reading comprehension","authors":"Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, Guoping Hu","year":"2017","venue":"Proceedings of the 55th ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Glorot and Bengio, 2010] Xavier Glorot and Yoshua Ben- gio. Understanding the difficulty of training deep feed- forward neural networks. In Proceedings of the 13th AIS- TATS, volume 9, pages 249-256, Sardinia, Italy, 13-15 May 2010. PMLR.","title":"Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks","authors":"Bengio Glorot","year":"2010","venue":"Proceedings of the 13th AIS-TATS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hermann et al., 2015] Karl Moritz Hermann, Tomas Ko- cisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In NIPS, pages 1693-1701, 2015.","title":"Teaching machines to read and comprehend","authors":"Hermann","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hill et al., 2016] Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading chil- dren's books with explicit memory representations. ICLR, 2016.","title":"The goldilocks principle: Reading children's books with explicit memory representations","authors":"Felix Hill, Antoine Bordes, Sumit Chopra, Jason Weston","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2014] Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. Convolutional neural network architectures for matching natural language sentences. In NIPS, pages 2042-2050, 2014.","title":"Convolutional neural network architectures for matching natural language sentences","authors":"Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. In Proceedings of the 2014 EMNLP, pages 1746-1751, Doha, Qatar, October 2014. Association for Computational Linguistics.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":"Proceedings of the 2014 EMNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2015] Jiwei Li, Thang Luong, and Dan Jurafsky. A hierarchical neural autoencoder for paragraphs and docu- ments. In Proceedings of the 53rd ACL and the 7th IJC- NLP (Volume 1: Long Papers), pages 1106-1115, Beijing, China, July 2015. ACL.","title":"A hierarchical neural autoencoder for paragraphs and documents","authors":"Li","year":"2015","venue":"Proceedings of the 53rd ACL and the 7th IJC-NLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[MacCartney, 2009] Bill MacCartney. Natural language in- ference. Stanford University, 2009.","title":"Natural language inference","authors":"Maccartney","year":"2009","venue":"Bill MacCartney","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"Proceedings of the 2014 EMNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100,000+ ques- tions for machine comprehension of text. In Proceedings of the 2016 EMNLP, pages 2383-2392, Austin, Texas, November 2016. ACL.","title":"Squad: 100,000+ questions for machine comprehension of text","authors":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang","year":"2016","venue":"Proceedings of the 2016 EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Resnick and Varian, 1997] Paul Resnick and Hal R Varian. Recommender systems. Communications of the ACM, 40(3):56-58, 1997.","title":"Recommender systems","authors":"Varian Resnick, Paul Resnick, Hal R Varian","year":"1997","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Richardson et al., 2013] Matthew Richardson, Christo- pher JC Burges, and Erin Renshaw. Mctest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP, volume 3, page 4, 2013.","title":"Mctest: A challenge dataset for the open-domain machine comprehension of text","authors":"Matthew Richardson, J C Christopher, Erin Burges, Renshaw","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Seo et al., 2016] Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. International Confer- ence on Learning Representations, 2016.","title":"Bidirectional attention flow for machine comprehension","authors":"Seo","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vinyals et al., 2015] Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In NIPS, pages 2692- 2700, 2015.","title":"Pointer networks","authors":"Oriol Vinyals, Meire Fortunato, Navdeep Jaitly","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang and Jiang, 2016] Shuohang Wang and Jing Jiang. Learning natural language inference with lstm. In Pro- ceedings of the 2016 NAACL: Human Language Technolo- gies, pages 1442-1451, San Diego, California, June 2016. ACL.","title":"Learning natural language inference with lstm","authors":"Wang, Jiang, Shuohang Wang, Jing Jiang","year":"2016","venue":"Proceedings of the 2016 NAACL: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang and Jiang, 2017] Shuohang Wang and Jing Jiang. Machine comprehension using match-lstm and answer pointer. ICLR, 2017.","title":"Machine comprehension using match-lstm and answer pointer","authors":"Wang, Jiang, Shuohang Wang, Jing Jiang","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2017] Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. Gated self-matching net- works for reading comprehension and question answering. In Proceedings of the 55th ACL (Volume 1: Long Papers), pages 189-198, Vancouver, Canada, July 2017. ACL.","title":"Gated self-matching networks for reading comprehension and question answering","authors":"Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, Ming Zhou","year":"2017","venue":"Proceedings of the 55th ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Weston et al., 2014] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. International Confer- ence on Learning Representations, 2014.","title":"Antoine Bordes","authors":"Jason Weston, Sumit Chopra","year":"2014","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zilly et al., 2017] Julian Georg Zilly, Rupesh Kumar Srivas- tava, Jan Koutník, and Jürgen Schmidhuber. Recurrent highway networks. In Proceedings of the 34th ICML, vol- ume 70, pages 4189-4198, Sydney, Australia, 06-11 Aug 2017. PMLR.","title":"Recurrent highway networks","authors":"Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, Jürgen Schmidhuber","year":"2017","venue":"Proceedings of the 34th ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}