{"type":"ScholarlyArticle","title":"ElimiNet: A Model for Eliminating Options for Reading Comprehension with Multiple Choice Questions","authors":[{"name":"Soham Parikh","slug":"soham-parikh","given":"Soham","family":"Parikh","affiliation":"Indian Institute of Technology , Madras"},{"name":"Ananya Sai","slug":"ananya-sai","given":"Ananya","family":"Sai","affiliation":"Indian Institute of Technology , Madras"},{"name":"Preksha Nema","slug":"preksha-nema","given":"Preksha","family":"Nema","affiliation":"Indian Institute of Technology , Madras"},{"name":"Mitesh Khapra","slug":"mitesh-khapra","given":"Mitesh","family":"Khapra","affiliation":"Indian Institute of Technology , Madras"}],"abstract":"The task of Reading Comprehension with Multiple Choice Questions, requires a human (or machine) to read a given {passage, question} pair and select one of the n given options. The current state of the art model for this task first computes a question-aware representation for the passage and then selects the option which has the maximum similarity with this representation. However, when humans perform this task they do not just focus on option selection but use a combination of elimination and selection. Specifically, a human would first try to eliminate the most irrelevant option and then read the passage again in the light of this new information (and perhaps ignore portions corresponding to the eliminated option). This process could be repeated multiple times till the reader is finally ready to select the correct option. We propose ElimiNet, a neural network-based model which tries to mimic this process. Specifically, it has gates which decide whether an option can be eliminated given the {passage, question} pair and if so it tries to make the passage representation orthogonal to this eliminated option (akin to ignoring portions of the passage corresponding to the eliminated option). The model makes multiple rounds of partial elimination to refine the passage representation and finally uses a selection module to pick the best option. We evaluate our model on the recently released large scale RACE dataset and show that it outperforms the current state of the art model on 7 out of the 13 question types in this dataset. Further, we show that taking an ensemble of our elimination-selection based method with a selection based method gives us an improvement of 3.1% over the best-reported performance on this dataset.","doi":"10.24963/ijcai.2018/594","url":"https://ijcai.org/proceedings/2018/594","pdf":"https://ijcai.org/proceedings/2018/0594.pdf","markdown":"https://ijcai.org/proceedings/2018/594.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/594","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":594,"pages":{"first":4272,"last":4278},"keywords":["Natural Language Processing: Question Answering","Machine Learning: Deep Learning","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2016] Danqi Chen, Jason Bolton, and Christo- pher D. Manning. A thorough examination of the cnn/daily mail reading comprehension task. In ACL (1). The Asso- ciation for Computer Linguistics, 2016.","title":"A thorough examination of the cnn/daily mail reading comprehension task","authors":"Chen","year":"2016","venue":"The Association for Computer Linguistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chung et al., 2014] Junyoung Chung, C ¸aglar Gülc ¸ehre, KyungHyun Cho, and Yoshua Bengio. Empirical evalua- tion of gated recurrent neural networks on sequence mod- eling. CoRR, abs/1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cui et al., 2017] Yiming Cui, Zhipeng Chen, Si Wei, Shi- jin Wang, Ting Liu, and Guoping Hu. Attention-over- attention neural networks for reading comprehension. In ACL (1), pages 593-602. Association for Computational Linguistics, 2017.","title":"Attention-overattention neural networks for reading comprehension","authors":"Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, Guoping Hu","year":"2017","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dhingra et al., 2017] Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Gated-attention readers for text comprehension. In ACL (1), pages 1832-1846. Association for Computational Lin- guistics, 2017.","title":"Gated-attention readers for text comprehension","authors":"Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William W Cohen, Ruslan Salakhutdinov","year":"2017","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hermann et al., 2015] Karl Moritz Hermann, Tomás Ko- ciský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In NIPS, pages 1693-1701, 2015.","title":"Teaching machines to read and comprehend","authors":"Hermann","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hill et al., 2015] Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading chil- dren's books with explicit memory representations. CoRR, abs/1511.02301, 2015.","title":"The goldilocks principle: Reading children's books with explicit memory representations","authors":"Felix Hill, Antoine Bordes, Sumit Chopra, Jason Weston","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hu et al., 2017] Minghao Hu, Yuxing Peng, and Xipeng Qiu. Mnemonic reader for machine comprehension. CoRR, abs/1705.02798, 2017.","title":"Mnemonic reader for machine comprehension","authors":"Minghao Hu, Yuxing Peng, Xipeng Qiu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Joshi et al., 2017] Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale dis- tantly supervised challenge dataset for reading compre- hension. In ACL (1), pages 1601-1611. Association for Computational Linguistics, 2017.","title":"Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension","authors":"Mandar Joshi, Eunsol Choi, Daniel S Weld, Luke Zettlemoyer","year":"2017","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kadlec et al., 2016] Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, and Jan Kleindienst. Text understanding with the attention sum reader network. In ACL (1). The Association for Computer Linguistics, 2016.","title":"Text understanding with the attention sum reader network","authors":"Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, Jan Kleindienst","year":"2016","venue":"The Association for Computer Linguistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khashabi et al., 2016] Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Peter Clark, Oren Etzioni, and Dan Roth. Question answering via integer programming over semi-structured knowledge. In IJCAI, pages 1145-1152. IJCAI/AAAI Press, 2016.","title":"Question answering via integer programming over semi-structured knowledge","authors":"Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Peter Clark, Oren Etzioni, Dan Roth","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/166"},{"position":11,"raw":"[Kociský et al., 2017] Tomás Kociský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. CoRR, abs/1712.07040, 2017.","title":"The narrativeqa reading comprehension challenge","authors":"Tomás Kociský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, Edward Grefenstette","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lai et al., 2017] Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard H. Hovy. RACE: large-scale reading comprehension dataset from examinations. In EMNLP, pages 785-794. Association for Computational Linguistics, 2017.","title":"RACE: large-scale reading comprehension dataset from examinations","authors":"Lai","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nguyen et al., 2016] Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. Ms marco: A human generated machine reading comprehension dataset. November 2016.","title":"Ms marco: A human generated machine reading comprehension dataset","authors":"Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, Li Deng","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Onishi et al., 2016] Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, and David A. McAllester. Who did what: A large-scale person-centered cloze dataset. In EMNLP, pages 2230-2235. The Association for Compu- tational Linguistics, 2016.","title":"Who did what: A large-scale person-centered cloze dataset","authors":"Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, David A Mcallester","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543. ACL, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100, 000+ ques- tions for machine comprehension of text. In EMNLP, pages 2383-2392. The Association for Computational Linguistics, 2016.","title":"Squad: 100, 000+ questions for machine comprehension of text","authors":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Richardson et al., 2013] Matthew Richardson, Christopher J. C. Burges, and Erin Renshaw. Mctest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP, pages 193-203. ACL, 2013.","title":"Mctest: A challenge dataset for the open-domain machine comprehension of text","authors":"Matthew Richardson, J C Christopher, Erin Burges, Renshaw","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Seo et al., 2016] Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. CoRR, abs/1611.01603, 2016.","title":"Bidirectional attention flow for machine comprehension","authors":"Seo","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Trischler et al., 2016] Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. Newsqa: A machine comprehension dataset. CoRR, abs/1611.09830, 2016.","title":"Newsqa: A machine comprehension dataset","authors":"Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, Kaheer Suleman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2017] Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. Gated self-matching net- works for reading comprehension and question answer- ing. In ACL (1), pages 189-198. Association for Com- putational Linguistics, 2017.","title":"Gated self-matching networks for reading comprehension and question answering","authors":"Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, Ming Zhou","year":"2017","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xiong et al., 2016] Caiming Xiong, Victor Zhong, and Richard Socher. Dynamic coattention networks for ques- tion answering. CoRR, abs/1611.01604, 2016.","title":"Dynamic coattention networks for question answering","authors":"Xiong","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}