{"type":"ScholarlyArticle","title":"Modeling Topical Relevance for Multi-Turn Dialogue Generation","authors":[{"name":"Hainan Zhang","slug":"hainan-zhang","given":"Hainan","family":"Zhang","affiliation":"JD.com , Beijing , China"},{"name":"Yanyan Lan","slug":"yanyan-lan","given":"Yanyan","family":"Lan","affiliation":"CAS Key Lab of Network Data Science and Technology , Institute of Computing Technology , CAS"},{"name":"Liang Pang","slug":"liang-pang","given":"Liang","family":"Pang","affiliation":"CAS Key Lab of Network Data Science and Technology , Institute of Computing Technology , CAS; Baidu.com , Beijing , China"},{"name":"Hongshen Chen","slug":"hongshen-chen","given":"Hongshen","family":"Chen","affiliation":"JD.com , Beijing , China"},{"name":"Zhuoye Ding","slug":"zhuoye-ding","given":"Zhuoye","family":"Ding","affiliation":"JD.com , Beijing , China"},{"name":"Dawei Yin","slug":"dawei-yin","given":"Dawei","family":"Yin","affiliation":"University of Chinese Academy of Sciences"}],"abstract":"Topic drift is a common phenomenon in multi-turn dialogue. Therefore, an ideal dialogue generation models should be able to capture the topic information of each context, detect the relevant context, and produce appropriate responses accordingly. However, existing models usually use word or sentence level similarities to detect the relevant contexts, which fail to well capture the topical level relevance. In this paper, we propose a new model, named STAR-BTM, to tackle this problem. Firstly, the Biterm Topic Model is pre-trained on the whole training dataset. Then, the topic level attention weights are computed based on the topic representation of each context. Finally, the attention weights and the topic distribution are utilized in the decoding process to generate the corresponding responses. Experimental results on both Chinese customer services data and English Ubuntu dialogue data show that STAR-BTM significantly outperforms several state-of-the-art methods, in terms of both metric-based and human evaluations.","doi":"10.24963/ijcai.2020/517","url":"https://ijcai.org/proceedings/2020/517","pdf":"https://ijcai.org/proceedings/2020/0517.pdf","markdown":"https://ijcai.org/proceedings/2020/517.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/517","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":517,"pages":{"first":3737,"last":3743},"keywords":["Natural Language Processing: Dialogue","Natural Language Processing: Natural Language Generation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25125/file_25125.m3u8","length_sec":300,"provider":"hls"}],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2018] Hongshen Chen, Zhaochun Ren, Jiliang Tang, Yihong Eric Zhao, and Dawei Yin. Hierarchical variational memory network for dialogue generation. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, pages 1653-1662, 2018.","title":"Hierarchical variational memory network for dialogue generation","authors":"Chen","year":"2018","venue":"Proceedings of the 2018 World Wide Web Conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fleiss, 1971] Joseph L Fleiss. Measuring nominal scale agreement among many raters. American Psychological Association, 1971.","title":"Measuring nominal scale agreement among many raters","authors":"L Joseph, Fleiss","year":"1971","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Li et al., 2016a] Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting ob- jective function for neural conversation models. NAACL, 2016.","title":"A diversity-promoting objective function for neural conversation models","authors":"Li","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Li et al., 2016b] Jiwei Li, Will Monroe, Alan Ritter, and Galley et al. Deep reinforcement learning for dialogue generation. EMNLP, 2016.","title":"Deep reinforcement learning for dialogue generation","authors":"Li","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2017] Jiwei Li, Will Monroe, Tianlin Shi, Alan Ritter, and Dan Jurafsky. Adversarial learning for neural dialogue generation. EMNLP, 2017.","title":"Adversarial learning for neural dialogue generation","authors":"Jiwei Li, Will Monroe, Tianlin Shi, Alan Ritter, Dan Jurafsky","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lowe et al., 2015] Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. Computer Science, 2015.","title":"The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems","authors":"Ryan Lowe, Nissan Pow, Iulian Serban, Joelle Pineau","year":"2015","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Mou et al., 2017] Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation. ACL, 2017.","title":"Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation","authors":"Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, Zhi Jin","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Serban et al., 2016] Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. Building end-to-end dialogue systems using generative hi- erarchical neural network models. In Thirtieth AAAI Con- ference on Artificial Intelligence, 2016.","title":"Building end-to-end dialogue systems using generative hierarchical neural network models","authors":"Alessandro Iulian V Serban, Yoshua Sordoni, Aaron Bengio, Joelle Courville, Pineau","year":"2016","venue":"Thirtieth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Serban et al., 2017a] Iulian Vlad Serban, Tim Klinger, Ger- ald Tesauro, Kartik Talamadupula, Bowen Zhou, Yoshua Bengio, and Aaron Courville. Multiresolution recurrent neural networks: An application to dialogue response gen- eration. In Thirty-First AAAI Conference on Artificial In- telligence, 2017.","title":"Multiresolution recurrent neural networks: An application to dialogue response generation","authors":"Serban","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Serban et al., 2017b] Iulian Vlad Serban, Alessandro Sor- doni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. A hierarchical latent vari- able encoder-decoder model for generating dialogues. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.","title":"A hierarchical latent variable encoder-decoder model for generating dialogues","authors":"Serban","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Sordoni et al., 2015] Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simon- sen, and Jian-Yun Nie. A hierarchical recurrent encoder- decoder for generative context-aware query suggestion. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pages 553- 562, 2015.","title":"A hierarchical recurrent encoderdecoder for generative context-aware query suggestion","authors":"Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, Jian-Yun Nie","year":"2015","venue":"Proceedings of the 24th ACM International on Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tian et al., 2017] Zhiliang Tian, Rui Yan, Lili Mou, Yiping Song, Yansong Feng, and Dongyan Zhao. How to make context more useful? an empirical study on context-aware neural conversational models. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), volume 2, pages 231- 236, 2017.","title":"How to make context more useful? an empirical study on context-aware neural conversational models","authors":"Zhiliang Tian, Rui Yan, Lili Mou, Yiping Song, Yansong Feng, Dongyan Zhao","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wu et al., 2017] Yu Wu, Wei Wu, Chen Xing, Ming Zhou, and Zhoujun Li. Sequential matching network: A new architecture for multi-turn response selection in retrieval- based chatbots. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.","title":"Sequential matching network: A new architecture for multi-turn response selection in retrievalbased chatbots","authors":"Yu Wu, Wei Wu, Chen Xing, Ming Zhou, Zhoujun Li","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Xing et al., 2017] Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. Topic aware neural response generation. In AAAI, pages 3351-3357, 2017.","title":"Topic aware neural response generation","authors":"Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Xing et al., 2018] Chen Xing, Yu Wu, Wei Wu, Yalou Huang, and Ming Zhou. Hierarchical recurrent attention network for response generation. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"Hierarchical recurrent attention network for response generation","authors":"Chen Xing, Yu Wu, Wei Wu, Yalou Huang, Ming Zhou","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yan et al., 2013] Xiaohui Yan, Jiafeng Guo, Yanyan Lan, and Xueqi Cheng. A biterm topic model for short texts. In Proceedings of the 22nd international conference on World Wide Web, pages 1445-1456. ACM, 2013.","title":"A biterm topic model for short texts","authors":"Xiaohui Yan, Jiafeng Guo, Yanyan Lan, Xueqi Cheng","year":"2013","venue":"Proceedings of the 22nd international conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang et al., 2018a] Hainan Zhang, Yanyan Lan, Jiafeng Guo, Jun Xu, and Xueqi Cheng. Reinforcing coherence for sequence to sequence model in dialogue generation. In International Joint Conference on Artificial Intelligence, pages 4567-4573, 2018.","title":"Reinforcing coherence for sequence to sequence model in dialogue generation","authors":"Zhang","year":"2018","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/635"},{"position":20,"raw":"[Zhang et al., 2018b] Hainan Zhang, Yanyan Lan, Jiafeng Guo, Jun Xu, and Xueqi Cheng. Tailored sequence to se- quence models to different conversation scenarios. In Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol- ume 1, pages 1479-1488, 2018.","title":"Tailored sequence to sequence models to different conversation scenarios","authors":"Zhang","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2018c] Weinan Zhang, Yiming Cui, Yifa Wang, Qingfu Zhu, Lingzhi Li, Lianqiang Zhou, and Ting Liu. Context-sensitive generation of open-domain con- versational responses. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 2437-2447, 2018.","title":"Context-sensitive generation of open-domain conversational responses","authors":"Zhang","year":"2018","venue":"Proceedings of the 27th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2019] Hainan Zhang, Yanyan Lan, Liang Pang, Jiafeng Guo, and Xueqi Cheng. Recosa: Detect- ing the relevant contexts with self-attention for multi-turn dialogue generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3721-3730, 2019.","title":"Recosa: Detecting the relevant contexts with self-attention for multi-turn dialogue generation","authors":"Zhang","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 2016] Xiangyang Zhou, Daxiang Dong, Hua Wu, Shiqi Zhao, Dianhai Yu, Hao Tian, Xuan Liu, and Rui Yan. Multi-view response selection for human-computer conversation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 372-381, 2016.","title":"Multi-view response selection for human-computer conversation","authors":"Xiangyang Zhou, Daxiang Dong, Hua Wu, Shiqi Zhao, Dianhai Yu, Hao Tian, Xuan Liu, Rui Yan","year":"2016","venue":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}