{"type":"ScholarlyArticle","title":"Conditional Generative Adversarial Networks for Commonsense Machine Comprehension","authors":[{"name":"Bingning Wang","slug":"bingning-wang","given":"Bingning","family":"Wang","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences , Beijing , 100190 , China; University of Chinese Academy of Sciences , Beijing , 100049 , China"},{"name":"Kang Liu","slug":"kang-liu","given":"Kang","family":"Liu","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences , Beijing , 100190 , China"},{"name":"Jun Zhao","slug":"jun-zhao","given":"Jun","family":"Zhao","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences , Beijing , 100190 , China; University of Chinese Academy of Sciences , Beijing , 100049 , China"}],"abstract":"Recently proposed Story Cloze Test [Mostafazadeh et al., 2016] is a commonsense machine comprehension application to deal with natural language understanding problem. This dataset contains a lot of story tests which require commonsense inference ability. Unfortunately, the training data is almost unsupervised where each context document followed with only one positive sentence that can be inferred from the context. However, in the testing period, we must make inference from two candidate sentences. To tackle this problem, we employ the generative adversarial networks (GANs) to generate fake sentence. We proposed a Conditional GANs in which the generator is conditioned by the context. Our experiments show the advantage of the CGANs in discriminating sentence and achieve state-of-the-art results in commonsense story reading comprehension task compared with previous feature engineering and deep learning methods.","doi":"10.24963/ijcai.2017/576","url":"https://ijcai.org/proceedings/2017/576","pdf":"https://ijcai.org/proceedings/2017/0576.pdf","markdown":"https://ijcai.org/proceedings/2017/576.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/576","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":576,"pages":{"first":4123,"last":4129},"keywords":["Natural Language Processing: Question Answering","Natural Language Processing: Information Extraction"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky and Bottou, 2017] Martín Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. CoRR, abs/1701.04862, 2017.","title":"Towards principled methods for training generative adversarial networks","authors":"Bottou Arjovsky, Martín Arjovsky, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowman et al., 2016] Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, and Christo- pher Potts. A fast unified model for parsing and sentence under- standing. CoRR, abs/1603.06021, 2016.","title":"A fast unified model for parsing and sentence understanding","authors":"R Samuel, Jon Bowman, Abhinav Gauthier, Raghav Rastogi, Christopher D Gupta, Christopher Manning, Potts","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chambers and Jurafsky, 2008] Nathanael Chambers and Daniel Jurafsky. Unsupervised learning of narrative event chains. In ACL, 2008.","title":"Unsupervised learning of narrative event chains","authors":"Jurafsky Chambers, Nathanael Chambers, Daniel Jurafsky","year":"2008","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Inter- pretable representation learning by information maximizing gen- erative adversarial nets. NIPS, 2016.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. arXiv:1409.1259, 2014.","title":"On the properties of neural machine translation: Encoder-decoder approaches","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Denton et al., 2015] Emily L Denton, Soumith Chintala, Rob Fer- gus, et al. Deep generative image models using a laplacian pyra- mid of adversarial networks. In NIPS, pages 1486-1494, 2015.","title":"Deep generative image models using a laplacian pyramid of adversarial networks","authors":"Soumith Emily L Denton, Rob Chintala, Fergus","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Donahue et al., 2016] Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. Adversarial feature learning. arXiv preprint arXiv:1605.09782, 2016.","title":"Adversarial feature learning","authors":"Jeff Donahue, Philipp Krähenbühl, Trevor Darrell","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hermann et al., 2015] Karl Moritz Hermann, Tomas Kocisky, Ed- ward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In NIPS, pages 1684-1692, 2015.","title":"Teaching machines to read and comprehend","authors":"Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, Phil Blunsom","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hill et al., 2015] Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading children's books with explicit memory representations. ICLR, 2015.","title":"The goldilocks principle: Reading children's books with explicit memory representations","authors":"Felix Hill, Antoine Bordes, Sumit Chopra, Jason Weston","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2013] Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. In Pro- ceedings of the 22nd CIKM, pages 2333-2338. ACM, 2013.","title":"Learning deep structured semantic models for web search using clickthrough data","authors":"Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck","year":"2013","venue":"Proceedings of the 22nd CIKM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jang et al., 2016] Eric Jang, Shixiang Gu, and Ben Poole. Cate- gorical reparameterization with gumbel-softmax. arXiv, 2016.","title":"Categorical reparameterization with gumbel-softmax","authors":"Eric Jang, Shixiang Gu, Ben Poole","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jozefowicz et al., 2015] Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. An empirical exploration of recurrent network ar- chitectures. In Proceedings of ICML-15, pages 2342-2350, 2015.","title":"An empirical exploration of recurrent network architectures","authors":"Rafal Jozefowicz, Wojciech Zaremba, Ilya Sutskever","year":"2015","venue":"Proceedings of ICML-15","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kiros et al., 2015] Ryan Kiros, Yukun Zhu, Ruslan R Salakhut- dinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In NIPS, 2015.","title":"Skip-thought vectors","authors":"Ryan Kiros, Yukun Zhu, R Ruslan, Richard Salakhutdinov, Raquel Zemel, Antonio Urtasun, Sanja Torralba, Fidler","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kusner and Hernández-Lobato, 2016] Matt J. Kusner and José Miguel Hernández-Lobato. Gans for sequences of discrete elements with the gumbel-softmax distribution. CoRR, abs/1611.04051, 2016.","title":"Gans for sequences of discrete elements with the gumbel-softmax distribution","authors":"Hernández-Lobato ; Kusner, J Matt, José Kusner, Miguel Hernández-Lobato","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lee et al., 2015] Moontae Lee, Xiaodong He, Wen-tau Yih, Jian- feng Gao, Li Deng, and Paul Smolensky. Reasoning in vec- tor space: An exploratory study of question answering. arXiv preprint arXiv:1511.06426, 2015.","title":"Reasoning in vector space: An exploratory study of question answering","authors":"Lee","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin and Och, 2004] Chin Yew Lin and Franz Josef Och. Auto- matic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. ACL, 2004.","title":"Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics","authors":"Lin, Och ; ] Chin, Yew Lin, Franz Josef, Och","year":"2004","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Luong et al., 2015] Minh-Thang Luong, Hieu Pham, and Christo- pher D. Manning. Effective approaches to attention-based neural machine translation. In 2015 EMNLP, 2015.","title":"Effective approaches to attention-based neural machine translation","authors":"Minh-Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Manning et al., 2014] Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc- closky. The stanford corenlp natural language processing toolkit. In ACL: System Demos, 2014.","title":"The stanford corenlp natural language processing toolkit","authors":"D Christopher, Mihai Manning, John Surdeanu, Jenny Bauer, Steven J Finkel, David Bethard, Mcclosky","year":"2014","venue":"ACL: System Demos","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mostafazadeh et al., 2016] Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. Proceedings of NAACL HLT, 2016.","title":"A corpus and cloze evaluation for deeper understanding of commonsense stories","authors":"Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, James Allen","year":"2016","venue":"Proceedings of NAACL HLT","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for ma- chine comprehension of text. arXiv:1606.05250, 2016.","title":"Squad: 100,000+ questions for machine comprehension of text","authors":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Richardson et al., 2013] Matthew Richardson, Christopher JC Burges, and Erin Renshaw. Mctest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP, 2013.","title":"Mctest: A challenge dataset for the open-domain machine comprehension of text","authors":"Matthew Richardson, J C Christopher, Erin Burges, Renshaw","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Santos et al., 2016] Cicero dos Santos, Ming Tan, Bing Xiang, and Bowen Zhou. Attentive pooling networks. arXiv, 2016.","title":"Attentive pooling networks","authors":"Santos","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Seo et al., 2016] Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. CoRR, abs/1611.01603, 2016.","title":"Bidirectional attention flow for machine comprehension","authors":"Seo","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Trischler et al., 2016] Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Ka- heer Suleman. Newsqa: A machine comprehension dataset. arXiv:1611.09830, 2016.","title":"Newsqa: A machine comprehension dataset","authors":"Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, Kaheer Suleman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2016a] Bingning Wang, Shangmin Guo, Kang Liu, Shizhu He, and Jun Zhao. Employing external rich knowledge for machine comprehension. In IJCAI, pages 2929-2935, 2016.","title":"Employing external rich knowledge for machine comprehension","authors":"Wang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/416"},{"position":30,"raw":"[Wang et al., 2016b] Bingning Wang, Kang Liu, and Jun Zhao. In- ner attention based recurrent neural networks for answer selec- tion. In ACL, 2016.","title":"Inner attention based recurrent neural networks for answer selection","authors":"Wang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"Weston et al., ] Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. arXiv.","title":"Memory networks","authors":"Weston","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Weston et al., 2015] Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. Towards ai-complete question answering: a set of prerequisite toy tasks. arXiv, 2015.","title":"Towards ai-complete question answering: a set of prerequisite toy tasks","authors":"Jason Weston, Antoine Bordes, Sumit Chopra, Tomas Mikolov","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Williams, 1992] Ronald J Williams. Simple statistical gradient- following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256, 1992.","title":"Simple statistical gradientfollowing algorithms for connectionist reinforcement learning","authors":"Williams Ronald","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yu et al., 2017] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, 2017.","title":"Seqgan: Sequence generative adversarial nets with policy gradient","authors":"Lantao Yu, Weinan Zhang, Jun Wang, Yong Yu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}