{"type":"ScholarlyArticle","title":"SentiGAN: Generating Sentimental Texts via Mixture Adversarial Networks","authors":[{"name":"Ke Wang","slug":"ke-wang","given":"Ke","family":"Wang","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"},{"name":"Xiaojun Wan","slug":"xiaojun-wan","given":"Xiaojun","family":"Wan","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"}],"abstract":"Generating texts of different sentiment labels is getting more and more attention in the area of natural language generation. Recently, Generative Adversarial Net (GAN) has shown promising results in text generation. However, the texts generated by GAN usually suffer from the problems of poor quality, lack of diversity and mode collapse. In this paper, we propose a novel framework - SentiGAN, which has multiple generators and one multi-class discriminator, to address the above problems. In our framework, multiple generators are trained simultaneously, aiming at generating texts of different sentiment labels without supervision. We propose a penalty based objective in the generators to force each of them to generate diversified examples of a specific sentiment label. Moreover, the use of multiple generators and one multi-class discriminator can make each generator focus on generating its own examples of a specific sentiment label accurately. Experimental results on four datasets demonstrate that our model consistently outperforms several state-of-the-art text generation methods in the sentiment accuracy and quality of generated texts.","doi":"10.24963/ijcai.2018/618","url":"https://ijcai.org/proceedings/2018/618","pdf":"https://ijcai.org/proceedings/2018/0618.pdf","markdown":"https://ijcai.org/proceedings/2018/618.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/618","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":618,"pages":{"first":4446,"last":4452},"keywords":["Natural Language Processing: Natural Language Generation","Natural Language Processing: Sentiment Analysis and Text Mining","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martín Arjovsky, Soumith Chin- tala, and Léon Bottou. Wasserstein GAN. CoRR, abs/1701.07875, 2017.","title":"Martín Arjovsky, Soumith Chintala","authors":"Arjovsky","year":"2017","venue":"Wasserstein GAN","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bachman and Precup, 2015] Philip Bachman and Doina Precup. Data generation as sequential decision making. NIPS, 2015.","title":"Data generation as sequential decision making","authors":"Precup Bachman, Philip Bachman, Doina Precup","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2015] Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for se- quence prediction with recurrent neural networks. In NIPS, pages 1171-1179, 2015.","title":"Scheduled sampling for sequence prediction with recurrent neural networks","authors":"Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Denton et al., 2015] Emily L Denton, Soumith Chintala, Arthur Szlam, and Robert D Fergus. Deep generative im- age models using a laplacian pyramid of adversarial net- works. NIPS, pages 1486-1494, 2015.","title":"Deep generative image models using a laplacian pyramid of adversarial networks","authors":"Soumith Emily L Denton, Arthur Chintala, Robert D Szlam, Fergus","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2017] Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, and Ke Xu. Learning to gen- erate product reviews from attributes. In EACL, volume 1, pages 623-632, 2017.","title":"Learning to generate product reviews from attributes","authors":"Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, Ke Xu","year":"2017","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] Ian J Goodfellow, Jean Pougetabadie, Mehdi Mirza, Bing Xu, David Warde- farley, Sherjil Ozair, Aaron C Courville, and Yoshua Bengio. Generative adversarial networks. arXiv: Machine Learning, 2014.","title":"Generative adversarial networks","authors":"Ian J Goodfellow, Jean Pougetabadie, Mehdi Mirza, Bing Xu, David Wardefarley, Sherjil Ozair, Aaron C Courville, Yoshua Bengio","year":"2014","venue":"arXiv: Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guo et al., 2017] Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Yong Yu, and Jun Wang. Long text generation via adversarial training with leaked information. CoRR, 2017.","title":"Long text generation via adversarial training with leaked information","authors":"Guo","year":"2017","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jurgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu and Liu, 2004] Minqing Hu and Bing Liu. Mining and summarizing customer reviews. pages 168-177, 2004.","title":"Mining and summarizing customer reviews","authors":"Liu Hu, Minqing Hu, Bing Liu","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2016] Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard H Hovy, and Eric P Xing. Harnessing deep neural networks with logic rules. ACL, pages 2410-2420, 2016.","title":"Harnessing deep neural networks with logic rules","authors":"Hu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2017] Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. Toward controlled generation of text. In ICML, pages 1587-1596, 2017.","title":"Toward controlled generation of text","authors":"Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, Eric P Xing","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Isola et al., 2016] Phillip Isola, Junyan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with con- ditional adversarial networks. CVPR, 2016.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Junyan Zhu, Tinghui Zhou, Alexei A Efros","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kingma and Welling, 2014] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma et al., 2014] Diederik P Kingma, Danilo J Rezende, Shakir Mohamed, and Max Welling. Semi- supervised learning with deep generative models. NIPS, 4:3581-3589, 2014.","title":"Semisupervised learning with deep generative models","authors":"Danilo J Diederik P Kingma, Shakir Rezende, Max Mohamed, Welling","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kusner and Hernández-Lobato, 2016] Matt J. Kusner and José Miguel Hernández-Lobato. GANS for sequences of discrete elements with the gumbel-softmax distribution. CoRR, abs/1611.04051, 2016.","title":"GANS for sequences of discrete elements with the gumbel-softmax distribution","authors":"Hernández-Lobato ; Kusner, J Matt, José Kusner, Miguel Hernández-Lobato","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lamb et al., 2016] Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. NIPS, pages 4601-4609, 2016.","title":"Professor forcing: A new algorithm for training recurrent networks","authors":"Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron C Courville, Yoshua Bengio","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2017] Kevin Lin, Dianqi Li, Xiaodong He, Ming- Ting Sun, and Zhengyou Zhang. Adversarial ranking for language generation. In NIPS, pages 3158-3168, 2017.","title":"Adversarial ranking for language generation","authors":"Kevin Lin, Dianqi Li, Xiaodong He, Ming-Ting Sun, Zhengyou Zhang","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lipton et al., 2015] Zachary C Lipton, Sharad Vikram, and Julian McAuley. Generative concatenative nets jointly learn to write and classify reviews. arXiv preprint arXiv:1511.03683, 2015.","title":"Generative concatenative nets jointly learn to write and classify reviews","authors":"Sharad Zachary C Lipton, Julian Vikram, Mcauley","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mcauley and Leskovec, 2013] Julian Mcauley and Jure Leskovec. From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. WWW, pages 897-908, 2013.","title":"From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews","authors":"Leskovec Mcauley, Julian Mcauley, Jure Leskovec","year":"2013","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mikolov et al., 2011] Tomas Mikolov, Stefan Kombrink, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. Extensions of recurrent neural network language model. pages 5528-5531, 2011.","title":"Extensions of recurrent neural network language model","authors":"Mikolov","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mirza and Osindero, 2014] Mehdi Mirza and Simon Osin- dero. Conditional generative adversarial nets. arXiv: Learning, 2014.","title":"Conditional generative adversarial nets","authors":"Mehdi Mirza, Simon Osindero","year":"2014","venue":"Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Salimans et al., 2016] Tim Salimans, Ian J Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. NIPS, pages 2234-2242, 2016.","title":"Improved techniques for training gans","authors":"Tim Salimans, Ian J Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Socher et al., 2013] Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for se- mantic compositionality over a sentiment treebank. 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, Christopher Potts","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Stolcke, 2002] Andreas Stolcke. Srilm -an extensible lan- guage modeling toolkit. 2002.","title":"Srilm -an extensible language modeling toolkit","authors":"Andreas Stolcke","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Theis et al., 2016] Lucas Theis, Aaron Van Den Oord, and Matthias Bethge. A note on the evaluation of generative models. ICLR, 2016.","title":"A note on the evaluation of generative models","authors":"Lucas Theis, Aaron Van Den, Matthias Oord, Bethge","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2017] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, 2017.","title":"Seqgan: Sequence generative adversarial nets with policy gradient","authors":"Lantao Yu, Weinan Zhang, Jun Wang, Yong Yu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zang and Wan, 2017] Hongyu Zang and Xiaojun Wan. To- wards automatic generation of product reviews from aspect-sentiment scores. In INLG, pages 168-177, 2017.","title":"Towards automatic generation of product reviews from aspect-sentiment scores","authors":"Wan ; ] Zang, Zang Hongyu, Xiaojun Wan","year":"2017","venue":"INLG","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Lecun, 2015] Xiang Zhang and Yann Lecun. Text understanding from scratch. arXiv: Learning, 2015.","title":"Text understanding from scratch","authors":"Xiang Zhang, Yann Lecun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou and Wang, 2017] Xianda Zhou and William Yang Wang. Mojitalk: Generating emotional responses at scale. CoRR, abs/1711.04090, 2017.","title":"Mojitalk: Generating emotional responses at scale","authors":"Wang ; Zhou, Xianda Zhou, William Yang, Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhou et al., 2017] Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. Emotional chatting machine: Emotional conversation generation with internal and external memory. CoRR, abs/1704.01074, 2017.","title":"Emotional chatting machine: Emotional conversation generation with internal and external memory","authors":"Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, Bing Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}