{"type":"ScholarlyArticle","title":"Adaptively Multi-Objective Adversarial Training for Dialogue Generation","authors":[{"name":"Xuemiao Zhang","slug":"xuemiao-zhang","given":"Xuemiao","family":"Zhang","affiliation":"School of Software & Microelectronics , Peking University"},{"name":"Zhouxing Tan","slug":"zhouxing-tan","given":"Zhouxing","family":"Tan","affiliation":"School of Software & Microelectronics , Peking University"},{"name":"Xiaoning Zhang","slug":"xiaoning-zhang","given":"Xiaoning","family":"Zhang","affiliation":"Changchun Institute of Optics , Fine Mechanics and Physics , Chinese Academy of Sciences"},{"name":"Yang Cao","slug":"yang-cao","given":"Yang","family":"Cao","affiliation":"SenseTime Research"},{"name":"Rui Yan","slug":"rui-yan","given":"Rui","family":"Yan","affiliation":"Wangxuan Institute of Computer Technology , Peking University"}],"abstract":"Naive neural dialogue generation models tend to produce repetitive and dull utterances. The promising adversarial models train the generator against a well-designed discriminator to push it to improve towards the expected direction. However, assessing dialogues requires consideration of many aspects of linguistics, which are difficult to be fully covered by a single discriminator. To address it, we reframe the dialogue generation task as a multi-objective optimization problem and propose a novel adversarial dialogue generation framework with multiple discriminators that excel in different objectives for multiple linguistic aspects, called AMPGAN, whose feasibility is proved by theoretical derivations. Moreover, we design an adaptively adjusted sampling distribution to balance the discriminators and promote the overall improvement of the generator by continuing to focus on these objectives that the generator is not performing well relatively. Experimental results on two real-world datasets show a significant improvement over the baselines.","doi":"10.24963/ijcai.2020/397","url":"https://ijcai.org/proceedings/2020/397","pdf":"https://ijcai.org/proceedings/2020/0397.pdf","markdown":"https://ijcai.org/proceedings/2020/397.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/397","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":397,"pages":{"first":2872,"last":2878},"keywords":["Natural Language Processing: Dialogue","Machine Learning: Reinforcement Learning","Natural Language Processing: Natural Language Generation","Machine Learning: Adversarial Machine Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24054/file_24054.m3u8","length_sec":286,"provider":"hls"}],"references":[{"position":1,"raw":"[Albuquerque et al., 2019] Isabela Albuquerque, João Mon- teiro, Thang Doan, Breandan Considine, Tiago H. Falk, and Ioannis Mitliagkas. Multi-objective training of gener- ative adversarial networks with multiple discriminators. In ICML, 2019.","title":"Multi-objective training of generative adversarial networks with multiple discriminators","authors":"Albuquerque","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowman et al., 2016] Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from a continuous space. In SIGNLL, 2016.","title":"Generating sentences from a continuous space","authors":"R Samuel, Luke Bowman, Oriol Vilnis, Andrew Vinyals, Rafal Dai, Samy Jozefowicz, Bengio","year":"2016","venue":"SIGNLL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cui et al., 2019] Shaobo Cui, Rongzhong Lian, Di Jiang, Yuanfeng Song, Siqi Bao, and Yong Jiang. DAL: Dual adversarial learning for dialogue generation. In NAACL Workshop on NeuralGen, 2019.","title":"DAL: Dual adversarial learning for dialogue generation","authors":"Shaobo Cui, Rongzhong Lian, Di Jiang, Yuanfeng Song, Siqi Bao, Yong Jiang","year":"2019","venue":"NAACL Workshop on NeuralGen","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deb, 2001] Kalyanmoy Deb. Multi-objective optimization using evolutionary algorithms, volume 16. John Wiley & Sons, 2001.","title":"Multi-objective optimization using evolutionary algorithms","authors":"Deb Kalyanmoy","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Durugkar et al., 2017] Ishan P. Durugkar, Ian Gemp, and Sridhar Mahadevan. Generative multi-adversarial net- works. In ICLR, 2017.","title":"Generative multi-adversarial networks","authors":"P Ishan, Ian Durugkar, Sridhar Gemp, Mahadevan","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NIPS, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In ICML, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2016] Jiwei Li, Michel Galley, Chris Brockett, and Jianfeng Gao. A diversity-promoting objective function for neural conversation models. In NAACL, 2016.","title":"A diversity-promoting objective function for neural conversation models","authors":"Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2017] Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. Adversarial learning for neural dialogue generation. In EMNLP, 2017.","title":"Adversarial learning for neural dialogue generation","authors":"Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, Dan Jurafsky","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Luan et al., 2016] Yi Luan, Yangfeng Ji, and Mari Osten- dorf. Lstm based conversation models. arXiv preprint arXiv:1603.09457, 2016.","title":"Lstm based conversation models","authors":"Yi Luan, Yangfeng Ji, Mari Ostendorf","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Luong et al., 2015] Thang Luong, Hieu Pham, and Christo- pher D Manning. Effective approaches to attention-based neural machine translation. In EMNLP, 2015.","title":"Effective approaches to attention-based neural machine translation","authors":"Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Manning et al., 2014] Christopher Manning, Mihai Sur- deanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. The Stanford CoreNLP natural language processing toolkit. In ACL: System Demonstrations, 2014.","title":"The Stanford CoreNLP natural language processing toolkit","authors":"Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, David Mcclosky","year":"2014","venue":"ACL: System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Neyshabur et al., 2017] Behnam Neyshabur, Srinadh Bho- janapalli, and Ayan Chakrabarti. Stabilizing gan train- ing with multiple random projections. arXiv preprint arXiv:1705.07831, 2017.","title":"Stabilizing gan training with multiple random projections","authors":"Srinadh Behnam Neyshabur, Ayan Bhojanapalli, Chakrabarti","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nguyen et al., 2017] Tu Nguyen, Trung Le, Hung Vu, and Dinh Phung. Dual discriminator generative adversarial nets. In NIPS, 2017.","title":"Dual discriminator generative adversarial nets","authors":"Tu Nguyen, Trung Le, Hung Vu, Dinh Phung","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In ACL, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Serban et al., 2016] Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. Building end-to-end dialogue systems using generative hi- erarchical neural network models. In AAAI, 2016.","title":"Building end-to-end dialogue systems using generative hierarchical neural network models","authors":"Alessandro Iulian V Serban, Yoshua Sordoni, Aaron Bengio, Joelle Courville, Pineau","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Somesh Das Gupta, 2000] Jun Shao. Somesh Das Gupta. Mathematical statistics. Springer, 2000.","title":"Mathematical statistics","authors":"Somesh Das Gupta ;, Jun Shao, Das Somesh, Gupta","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sutskever et al., 2014] I Sutskever, O Vinyals, and QV Le. Sequence to sequence learning with neural networks. NIPS, 2014.","title":"QV Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sutton et al., 2000] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approx- imation. In NIPS, 2000.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"David A Richard S Sutton, Mcallester, P Satinder, Yishay Singh, Mansour","year":"2000","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tao et al., 2018] Chongyang Tao, Shen Gao, Mingyue Shang, Wei Wu, Dongyan Zhao, and Rui Yan. Get the point of my utterance! learning towards effective re- sponses with multi-head attention mechanism. In IJCAI, 2018.","title":"Get the point of my utterance! learning towards effective responses with multi-head attention mechanism","authors":"Chongyang Tao, Shen Gao, Mingyue Shang, Wei Wu, Dongyan Zhao, Rui Yan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/614"},{"position":23,"raw":"[Tiedemann, 2009] Jörg Tiedemann. News from opus-a col- lection of multilingual parallel corpora with tools and in- terfaces. 2009.","title":"News from opus-a collection of multilingual parallel corpora with tools and interfaces","authors":"Jörg Tiedemann, Tiedemann","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vashishth et al., 2019] Shikhar Vashishth, Manik Bhandari, Prateek Yadav, Piyush Rai, Chiranjib Bhattacharyya, and Partha Talukdar. Incorporating syntactic and semantic in- formation in word embeddings using graph convolutional networks. In ACL, 2019.","title":"Incorporating syntactic and semantic information in word embeddings using graph convolutional networks","authors":"Shikhar Vashishth, Manik Bhandari, Prateek Yadav, Piyush Rai, Chiranjib Bhattacharyya, Partha Talukdar","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2018] Jingjing Xu, Xuancheng Ren, Junyang Lin, and Xu Sun. Diversity-promoting GAN: A cross-entropy based generative adversarial network for diversified text generation. In EMNLP, 2018.","title":"Diversity-promoting GAN: A cross-entropy based generative adversarial network for diversified text generation","authors":"Jingjing Xu, Xuancheng Ren, Junyang Lin, Xu Sun","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2017] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, 2017.","title":"Seqgan: Sequence generative adversarial nets with policy gradient","authors":"Lantao Yu, Weinan Zhang, Jun Wang, Yong Yu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2018] Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. Generating informative and diverse conversational responses via adversarial information maximization. In NeurIPS, 2018.","title":"Generating informative and diverse conversational responses via adversarial information maximization","authors":"Zhang","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}