{"type":"ScholarlyArticle","title":"Exemplar Guided Neural Dialogue Generation","authors":[{"name":"Hengyi Cai","slug":"hengyi-cai","given":"Hengyi","family":"Cai","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Hongshen Chen","slug":"hongshen-chen","given":"Hongshen","family":"Chen","affiliation":"Data Science Lab , JD.com , China"},{"name":"Yonghao Song","slug":"yonghao-song","given":"Yonghao","family":"Song","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China"},{"name":"Xiaofang Zhao","slug":"xiaofang-zhao","given":"Xiaofang","family":"Zhao","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China"},{"name":"Dawei Yin","slug":"dawei-yin","given":"Dawei","family":"Yin","affiliation":"Baidu Inc. , China"}],"abstract":"Humans benefit from previous experiences when taking actions.\r\nSimilarly, related examples from the training data also provide exemplary information for neural dialogue models when responding to a given input message.\r\nHowever, effectively fusing such exemplary information into dialogue generation is non-trivial: useful exemplars are required to be not only literally-similar, but also topic-related with the given context.\r\nNoisy exemplars impair the neural dialogue models understanding the conversation topics and even corrupt the response generation.\r\nTo address the issues, we propose an exemplar guided neural dialogue generation model where exemplar responses are retrieved in terms of both the text similarity and the topic proximity through a two-stage exemplar retrieval model.\r\nIn the first stage, a small subset of conversations is retrieved from a training set given a dialogue context. \r\nThese candidate exemplars are then finely ranked regarding the topical proximity to choose the best-matched exemplar response. \r\nTo further induce the neural dialogue generation model consulting the exemplar response and the conversation topics more faithfully, we introduce a multi-source sampling mechanism to provide the dialogue model with both local exemplary semantics and global topical guidance during decoding.\r\nEmpirical evaluations on a large-scale conversation dataset show that the proposed approach significantly outperforms the state-of-the-art in terms of both the quantitative metrics and human evaluations.","doi":"10.24963/ijcai.2020/498","url":"https://ijcai.org/proceedings/2020/498","pdf":"https://ijcai.org/proceedings/2020/0498.pdf","markdown":"https://ijcai.org/proceedings/2020/498.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/498","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":498,"pages":{"first":3601,"last":3607},"keywords":["Natural Language Processing: Natural Language Generation","Machine Learning: Deep Learning: Sequence Modeling","Natural Language Processing: Natural Language Processing","Natural Language Processing: Dialogue"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24716/file_24716.m3u8","length_sec":248,"provider":"hls"}],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2019] Hengyi Cai, Hongshen Chen, Cheng Zhang, Yonghao Song, Xiaofang Zhao, and Dawei Yin. Adaptive parameterization for neural dialogue generation. In EMNLP-IJCNLP, 2019.","title":"Adaptive parameterization for neural dialogue generation","authors":"Hengyi Cai, Hongshen Chen, Cheng Zhang, Yonghao Song, Xiaofang Zhao, Dawei Yin","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Hongshen Chen, Zhaochun Ren, Jiliang Tang, Yihong Eric Zhao, and Dawei Yin. Hierarchical variational memory network for dialogue generation. In WWW, 2018.","title":"Hierarchical variational memory network for dialogue generation","authors":"Chen","year":"2018","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Clark and Cao, 2017] Stephen Clark and Kris Cao. Latent variable dialogue models and their diversity. In EACL, 2017.","title":"Latent variable dialogue models and their diversity","authors":"Clark, Cao, Stephen Clark, Kris Cao","year":"2017","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dixon and Mood, 1946] Wilfrid J Dixon and Alexander M Mood. The statistical sign test. Journal of the American Statistical Association, 41(236):557-566, 1946.","title":"The statistical sign test","authors":"J Wilfrid, Alexander M Dixon, Mood","year":"1946","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dodge et al., 2015] Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander H. Miller, Arthur Szlam, and Jason Weston. Evaluating pre- requisite qualities for learning end-to-end dialog systems. CoRR, abs/1511.06931, 2015.","title":"Evaluating prerequisite qualities for learning end-to-end dialog systems","authors":"Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander H Miller, Arthur Szlam, Jason Weston","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gu et al., 2016] Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. Incorporating copying mechanism in sequence-to-sequence learning. In ACL, 2016.","title":"Incorporating copying mechanism in sequence-to-sequence learning","authors":"Gu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Indyk and Motwani, 1998] Piotr Indyk and Rajeev Mot- wani. Approximate nearest neighbors: Towards removing the curse of dimensionality. In STOC, 1998.","title":"Approximate nearest neighbors: Towards removing the curse of dimensionality","authors":"Piotr Indyk, Rajeev Motwani","year":"1998","venue":"STOC","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2016] Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. In NAACL-HLT, 2016.","title":"A diversity-promoting objective function for neural conversation models","authors":"Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, Bill Dolan","year":"2016","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2016] Chia-Wei Liu, Ryan Lowe, Iulian Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. How NOT to evaluate your dialogue system: An empiri- cal study of unsupervised evaluation metrics for dialogue response generation. In EMNLP, 2016.","title":"How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation","authors":"Chia-Wei Liu, Ryan Lowe, Iulian Serban, Michael Noseworthy, Laurent Charlin, Joelle Pineau","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lowe et al., 2015] Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In SIGDIAL, 2015.","title":"The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems","authors":"Ryan Lowe, Nissan Pow, Iulian Serban, Joelle Pineau","year":"2015","venue":"SIGDIAL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Miao et al., 2017] Yishu Miao, Edward Grefenstette, and Phil Blunsom. Discovering discrete latent topics with neu- ral variational inference. In ICML, 2017.","title":"Discovering discrete latent topics with neural variational inference","authors":"Miao","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Miller et al., 2017] A. H. Miller, W. Feng, A. Fisch, J. Lu, D. Batra, A. Bordes, D. Parikh, and J. Weston. Par- lai: A dialog research software platform. arXiv preprint arXiv:1705.06476, 2017.","title":"Parlai: A dialog research software platform","authors":"A H Miller, W Feng, A Fisch, J Lu, D Batra, A Bordes, D Parikh, J Weston","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pandey et al., 2018] Gaurav Pandey, Danish Contractor, Vi- neet Kumar, and Sachindra Joshi. Exemplar encoder- decoder for neural conversation generation. In ACL, 2018.","title":"Exemplar encoderdecoder for neural conversation generation","authors":"Gaurav Pandey, Danish Contractor, Vineet Kumar, Sachindra Joshi","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In ACL, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Robertson and Zaragoza, 2009] Stephen E. Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Informa- tion Retrieval, 3(4):333-389, 2009.","title":"The probabilistic relevance framework: BM25 and beyond","authors":"Stephen E Robertson, Hugo Zaragoza","year":"2009","venue":"Foundations and Trends in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Serban et al., 2016] Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. Building end-to-end dialogue systems using generative hi- erarchical neural network models. In AAAI, 2016.","title":"Building end-to-end dialogue systems using generative hierarchical neural network models","authors":"V Iulian, Alessandro Serban, Yoshua Sordoni, Aaron Bengio, Joelle Courville, Pineau","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Serban et al., 2017] Iulian Vlad Serban, Alessandro Sor- doni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. A hierarchical latent variable encoder-decoder model for generating dia- logues. In AAAI, 2017.","title":"A hierarchical latent variable encoder-decoder model for generating dialogues","authors":"Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, Yoshua Bengio","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2017] Di Wang, Nebojsa Jojic, Chris Brockett, and Eric Nyberg. Steering output style and topic in neural response generation. In EMNLP, 2017.","title":"Steering output style and topic in neural response generation","authors":"Di Wang, Nebojsa Jojic, Chris Brockett, Eric Nyberg","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2018] Wenjie Wang, Minlie Huang, Xin-Shun Xu, Fumin Shen, and Liqiang Nie. Chat more: Deepen- ing and widening the chatting topic via A deep model. In SIGIR, 2018.","title":"Chat more: Deepening and widening the chatting topic via A deep model","authors":"Wenjie Wang, Minlie Huang, Xin-Shun Xu, Fumin Shen, Liqiang Nie","year":"2018","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xing et al., 2017] Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. Topic aware neural response generation. In AAAI, 2017.","title":"Topic aware neural response generation","authors":"Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yao et al., 2017] Lili Yao, Yaoyuan Zhang, Yansong Feng, Dongyan Zhao, and Rui Yan. Towards implicit content- introducing for generative short-text conversation systems. In EMNLP, 2017.","title":"Towards implicit contentintroducing for generative short-text conversation systems","authors":"Lili Yao, Yaoyuan Zhang, Yansong Feng, Dongyan Zhao, Rui Yan","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2018] Saizheng Zhang, Emily Dinan, Jack Ur- banek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In ACL, 2018.","title":"Personalizing dialogue agents: I have a dog, do you have pets too? In ACL","authors":"Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao et al., 2017] Tiancheng Zhao, Ran Zhao, and Maxine Eskénazi. Learning discourse-level diversity for neural di- alog models using conditional variational autoencoders. In ACL, 2017.","title":"Learning discourse-level diversity for neural dialog models using conditional variational autoencoders","authors":"Zhao","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhao et al., 2018] Tiancheng Zhao, Kyusong Lee, and Max- ine Eskénazi. Unsupervised discrete sentence representa- tion learning for interpretable neural dialog generation. In ACL, 2018.","title":"Unsupervised discrete sentence representation learning for interpretable neural dialog generation","authors":"Zhao","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}