{"type":"ScholarlyArticle","title":"Reinforced Mnemonic Reader for Machine Reading Comprehension","authors":[{"name":"Minghao Hu","slug":"minghao-hu","given":"Minghao","family":"Hu","affiliation":"† College of Computer , National University of Defense Technology , Changsha , China"},{"name":"Yuxing Peng","slug":"yuxing-peng","given":"Yuxing","family":"Peng","affiliation":"† College of Computer , National University of Defense Technology , Changsha , China; Fudan University and Mi- crosoft Research."},{"name":"Zhen Huang","slug":"zhen-huang","given":"Zhen","family":"Huang","affiliation":"† College of Computer , National University of Defense Technology , Changsha , China"},{"name":"Xipeng Qiu","slug":"xipeng-qiu","given":"Xipeng","family":"Qiu","affiliation":"‡ School of Computer Science , Fudan University , Shanghai , China"},{"name":"Furu Wei","slug":"furu-wei","given":"Furu","family":"Wei","affiliation":""},{"name":"Ming Zhou","slug":"ming-zhou","given":"Ming","family":"Zhou","affiliation":"§ Microsoft Research , Beijing , China"}],"abstract":"In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects. First, a reattention mechanism is proposed to refine current attentions by directly accessing to past attentions that are temporally memorized in a multi-round alignment architecture, so as to avoid the problems of attention redundancy and attention deficiency. Second, a new optimization approach, called dynamic-critical reinforcement learning, is introduced to extend the standard supervised method. It always encourages to predict a more acceptable answer so as to address the convergence suppression problem occurred in traditional reinforcement learning algorithms. Extensive experiments on the Stanford Question Answering Dataset (SQuAD) show that our model achieves state-of-the-art results. Meanwhile, our model outperforms previous systems by over 6% in terms of both Exact Match and F1 metrics on two adversarial SQuAD datasets.","doi":"10.24963/ijcai.2018/570","url":"https://ijcai.org/proceedings/2018/570","pdf":"https://ijcai.org/proceedings/2018/0570.pdf","markdown":"https://ijcai.org/proceedings/2018/570.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/570","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":570,"pages":{"first":4099,"last":4106},"keywords":["Machine Learning: Reinforcement Learning","Machine Learning: Deep Learning","Natural Language Processing: Question Answering"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2017] Danqi Chen, Adam Fisch, Jason We- ston, and Antoine Bordes. Reading wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051, 2017.","title":"Reading wikipedia to answer open-domain questions","authors":"Chen","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dzmitry Bahdanau, 2015] Yoshua Bengio Dzmitry Bah- danau, Kyunghyun Cho. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Yoshua Bengio, Dzmitry Bahdanau, Kyunghyun Cho","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016.","title":"Deep learning","authors":"Ian Goodfellow, Yoshua Bengio, Aaron Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hermann et al., 2015] Karl Moritz Hermann, Tomas Ko- cisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, , and Phil Blunsom. Teaching ma- chines to read and comprehend. In Proceedings of NIPS, 2015.","title":"Teaching machines to read and comprehend","authors":"Hermann","year":"2015","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hill et al., 2016] Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading chil- dren's books with explicit memory representations. In Pro- ceedings of ICLR, 2016.","title":"The goldilocks principle: Reading children's books with explicit memory representations","authors":"Felix Hill, Antoine Bordes, Sumit Chopra, Jason Weston","year":"2016","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735--1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2017] Hsin-Yuan Huang, Chenguang Zhu, Ye- long Shen, and Weizhu Chen. Fusionnet: Fusing via fully- aware attention with application to machine comprehen- sion. arXiv preprint arXiv:1711.07341, 2017.","title":"Fusionnet: Fusing via fullyaware attention with application to machine comprehension","authors":"Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, Weizhu Chen","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jia and Liang, 2017] Robin Jia and Percy Liang. Adversar- ial examples for evaluating reading comprehension sys- tems. In Proceedings of EMNLP, 2017.","title":"Adversarial examples for evaluating reading comprehension systems","authors":"Liang Jia, Robin Jia, Percy Liang","year":"2017","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kendall et al., 2017] Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. arXiv preprint arXiv:1705.07115, 2017.","title":"Multi-task learning using uncertainty to weigh losses for scene geometry and semantics","authors":"Alex Kendall, Yarin Gal, Roberto Cipolla","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Lei Jimmy Ba. Adam: A method for stochastic optimization. In CoRR, abs/1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Kingma, Jimmy Lei, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2017a] Rui Liu, Junjie Hu, Wei Wei, Zi Yang, and Eric Nyberg. Structural embedding of syntac- tic trees for machine comprehension. arXiv preprint arXiv:1703.00572, 2017.","title":"Structural embedding of syntactic trees for machine comprehension","authors":"Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017b] Xiaodong Liu, Yelong Shen, Kevin Duh, and Jianfeng Gao. Stochastic answer networks for machine reading comprehension. arXiv preprint arXiv:1712.03556, 2017.","title":"Stochastic answer networks for machine reading comprehension","authors":"Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In Proceedings of EMNLP, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Peters et al., 2018] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word prepresentations. In Proceedings of NAACL, 2018.","title":"Deep contextualized word prepresentations","authors":"Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer","year":"2018","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100,000+ ques- tions for machine comprehension of text. In Proceedings of EMNLP, 2016.","title":"Squad: 100,000+ questions for machine comprehension of text","authors":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rennie et al., 2016] Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. Self- critical sequence training for image captioning. arXiv preprint arXiv:1612.00563, 2016.","title":"Selfcritical sequence training for image captioning","authors":"Etienne Steven J Rennie, Youssef Marcheret, Jarret Mroueh, Vaibhava Ross, Goel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Seo et al., 2017] Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hananneh Hajishirzi. Bidirectional attention flow for machine comprehension. In Proceedings of ICLR, 2017.","title":"Bidirectional attention flow for machine comprehension","authors":"Seo","year":"2017","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shen et al., 2016] Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. Reasonet: Learning to stop reading in machine comprehension. arXiv preprint arXiv:1609.05284, 2016.","title":"Reasonet: Learning to stop reading in machine comprehension","authors":"Shen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Re- search, pages 1929-1958, 2014.","title":"Dropout: A simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Srivastava et al., 2015] RupeshKumar Srivastava, Klaus Gr- eff, and Jurgen Schmidhuber. Highway networks. arXiv preprint arXiv:1505.00387, 2015.","title":"Highway networks","authors":"Srivastava","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sutton and Barto, 1998] Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. MIT Press, 1998.","title":"Reinforcement learning: An introduction","authors":"Richard S Sutton, Andrew G Barto","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vinyals et al., 2015] Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In Proceedings of NIPS, 2015.","title":"Pointer networks","authors":"Oriol Vinyals, Meire Fortunato, Navdeep Jaitly","year":"2015","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang and Jiang, 2017] Shuohang Wang and Jing Jiang. Machine comprehension using match-lstm and answer pointer. In Proceedings of ICLR, 2017.","title":"Machine comprehension using match-lstm and answer pointer","authors":"Wang, Jiang ; ] Shuohang Wang, Jing Jiang","year":"2017","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017] Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. Gated self-matching net- works for reading comprehension and question answering. In Proceedings of ACL, 2017.","title":"Gated self-matching networks for reading comprehension and question answering","authors":"Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, Ming Zhou","year":"2017","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Weissenborn et al., 2017] Dirk Weissenborn, Georg Wiese, and Laura Seiffe. Making neural qa as simple as possible but not simpler. In Proceedings of CoNLL, pages 271-280, 2017.","title":"Making neural qa as simple as possible but not simpler","authors":"Weissenborn","year":"2017","venue":"Proceedings of CoNLL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xiong et al., 2017a] Caiming Xiong, Victor Zhong, and Richard Socher. Dcn+: Mixed objective and deep resid- ual coattention for question answering. arXiv preprint arXiv:1711.00106, 2017.","title":"Dcn+: Mixed objective and deep residual coattention for question answering","authors":"Xiong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xiong et al., 2017b] Caiming Xiong, Victor Zhong, and Richard Socher. Dynamic coattention networks for ques- tion answering. In Proceedings of ICLR, 2017.","title":"Dynamic coattention networks for question answering","authors":"Xiong","year":"2017","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}