{"type":"ScholarlyArticle","title":"A Weakly Supervised Method for Topic Segmentation and Labeling in Goal-oriented Dialogues via Reinforcement Learning","authors":[{"name":"Ryuichi Takanobu","slug":"ryuichi-takanobu","given":"Ryuichi","family":"Takanobu","affiliation":"Conversational AI Group, AI Lab. , Dept. of Computer Science , Tsinghua University ,"},{"name":"Minlie Huang","slug":"minlie-huang","given":"Minlie","family":"Huang","affiliation":"Conversational AI Group, AI Lab. , Dept. of Computer Science , Tsinghua University ,"},{"name":"Zhongzhou Zhao","slug":"zhongzhou-zhao","given":"Zhongzhou","family":"Zhao","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Fenglin Li","slug":"fenglin-li","given":"Fenglin","family":"Li","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Haiqing Chen","slug":"haiqing-chen","given":"Haiqing","family":"Chen","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Xiaoyan Zhu","slug":"xiaoyan-zhu","given":"Xiaoyan","family":"Zhu","affiliation":"Conversational AI Group, AI Lab. , Dept. of Computer Science , Tsinghua University ,; Beijing National Research Center for Information Science and Technology , China"},{"name":"Liqiang Nie","slug":"liqiang-nie","given":"Liqiang","family":"Nie","affiliation":"Shandong University , Jinan , China"}],"abstract":"Topic structure analysis plays a pivotal role in dialogue understanding. We propose a reinforcement learning (RL) method for topic segmentation and labeling in goal-oriented dialogues, which aims to detect topic boundaries among dialogue utterances and assign topic labels to the utterances. We address three common issues in the goal-oriented customer service dialogues: informality, local topic continuity, and global topic structure. We explore the task in a weakly supervised setting and formulate it as a sequential decision problem. The proposed method consists of a state representation network to address the informality issue, and a policy network with rewards to model local topic continuity and global topic structure. To train the two networks and offer a warm-start to the policy, we firstly use some keywords to annotate the data automatically. We then pre-train the networks on noisy data. Henceforth, the method continues to refine the data labels using the current policy to learn better state representations on the refined data for obtaining a better policy. Results demonstrate that this weakly supervised method obtains substantial improvements over state-of-the-art baselines.","doi":"10.24963/ijcai.2018/612","url":"https://ijcai.org/proceedings/2018/612","pdf":"https://ijcai.org/proceedings/2018/0612.pdf","markdown":"https://ijcai.org/proceedings/2018/612.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/612","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":612,"pages":{"first":4403,"last":4410},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Natural Language Processing","Natural Language Processing: Text Classification","Natural Language Processing: Dialogue","Machine Learning Applications: Applications of Reinforcement Learning"],"videos":[],"references":[{"position":1,"raw":"[Afantenos et al., 2015] Stergos Afantenos, Eric Kow, Nicholas Asher, and Jérémy Perret. Discourse parsing for multi-party chat dialogues. In EMNLP, pages 928-937, 2015.","title":"Discourse parsing for multi-party chat dialogues","authors":"Afantenos","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arguello and Rosé, 2006] Jaime Arguello and Carolyn Rosé. Topic segmentation of dialogue. In HLT-NAACL Workshop on Analyzing Conversations in Text and Speech, pages 42-49, 2006.","title":"Topic segmentation of dialogue","authors":"Jaime Arguello, Carolyn Rosé","year":"2006","venue":"HLT-NAACL Workshop on Analyzing Conversations in Text and Speech","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blei, 2012] David M Blei. Probabilistic topic models. Com- munications of the ACM, 55(4):77-84, 2012.","title":"Probabilistic topic models","authors":"M David, Blei","year":"2012","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bokaei et al., 2016] Mohammad Hadi Bokaei, Hossein Sameti, and Yang Liu. Extractive summarization of multi- party meetings through discourse segmentation. Natural Language Engineering, 22(1):41-72, 2016.","title":"Extractive summarization of multiparty meetings through discourse segmentation","authors":"Mohammad Hadi Bokaei, Hossein Sameti, Yang Liu","year":"2016","venue":"Natural Language Engineering","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chung et al., 2017] Junyoung Chung, Sungjin Ahn, and Yoshua Bengio. Hierarchical multiscale recurrent neural networks. In ICLR, 2017.","title":"Hierarchical multiscale recurrent neural networks","authors":"Chung","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Du et al., 2013] Lan Du, Wray L Buntine, and Mark John- son. Topic segmentation with a structured topic model. In HLT-NAACL, pages 190-200, 2013.","title":"Topic segmentation with a structured topic model","authors":"Lan Du, Mark Wray L Buntine, Johnson","year":"2013","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du et al., 2017] Wenchao Du, Pascal Poupart, and Wei Xu. Discovering conversational dependencies between mes- sages in dialogs. In AAAI, pages 4917-4918, 2017.","title":"Discovering conversational dependencies between messages in dialogs","authors":"Wenchao Du, Pascal Poupart, Wei Xu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Eisenstein and Barzilay, 2008] Jacob Eisenstein and Regina Barzilay. Bayesian unsupervised topic segmentation. In EMNLP, pages 334-343, 2008.","title":"Bayesian unsupervised topic segmentation","authors":"Barzilay Eisenstein","year":"2008","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guu et al., 2017] Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, and Percy Liang. From language to programs: Bridging reinforcement learning and max- imum marginal likelihood. In ACL, pages 1051-1062, 2017.","title":"From language to programs: Bridging reinforcement learning and maximum marginal likelihood","authors":"Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, Percy Liang","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Marti A Hearst. Texttiling: Segmenting text into multi-paragraph subtopic passages. Computational linguistics, 23(1):33-64, 1997.","title":"Texttiling: Segmenting text into multi-paragraph subtopic passages","authors":"A Marti, Hearst","year":"1997","venue":"Computational linguistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joty et al., 2013] Shafiq Joty, Giuseppe Carenini, and Ray- mond T Ng. Topic segmentation and labeling in asyn- chronous conversations. Journal of Artificial Intelligence Research, 47:521-573, 2013.","title":"Topic segmentation and labeling in asynchronous conversations","authors":"Shafiq Joty, Giuseppe Carenini, Raymond T Ng","year":"2013","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2016] Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. Deep reinforce- ment learning for dialogue generation. In EMNLP, pages 1192-1202, 2016.","title":"Deep reinforcement learning for dialogue generation","authors":"Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, Dan Jurafsky","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mehrotra et al., 2013] Rishabh Mehrotra, Scott Sanner, Wray Buntine, and Lexing Xie. Improving lda topic mod- els for microblogs via tweet pooling and automatic label- ing. In SIGIR, pages 889-892, 2013.","title":"Improving lda topic models for microblogs via tweet pooling and automatic labeling","authors":"Rishabh Mehrotra, Scott Sanner, Wray Buntine, Lexing Xie","year":"2013","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mei et al., 2007] Qiaozhu Mei, Xuehua Shen, and ChengX- iang Zhai. Automatic labeling of multinomial topic mod- els. In SIGKDD, pages 490-499, 2007.","title":"Qiaozhu Mei, Xuehua Shen, and ChengXiang Zhai. Automatic labeling of multinomial topic models","authors":"Mei","year":"2007","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pevzner and Hearst, 2002] Lev Pevzner and Marti A Hearst. A critique and improvement of an evaluation metric for text segmentation. Computational Linguistics, 28(1):19- 36, 2002.","title":"A critique and improvement of an evaluation metric for text segmentation","authors":"Hearst Pevzner","year":"2002","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Purver et al., 2006] Matthew Purver, Thomas L Griffiths, Konrad P Körding, and Joshua B Tenenbaum. Unsuper- vised topic modelling for multi-party spoken discourse. In ACL, pages 17-24, 2006.","title":"Unsupervised topic modelling for multi-party spoken discourse","authors":"Matthew Purver, L Thomas, Konrad P Griffiths, Joshua B Körding, Tenenbaum","year":"2006","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qin et al., 2017] Kechen Qin, Lu Wang, and Joseph Kim. Joint modeling of content and discourse relations in dia- logues. In ACL, pages 974-984, 2017.","title":"Joint modeling of content and discourse relations in dialogues","authors":"Qin","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ramage et al., 2009] Daniel Ramage, David Hall, Ramesh Nallapati, and Christopher D Manning. Labeled lda: A su- pervised topic model for credit attribution in multi-labeled corpora. In EMNLP, pages 248-256, 2009.","title":"Labeled lda: A supervised topic model for credit attribution in multi-labeled corpora","authors":"Daniel Ramage, David Hall, Ramesh Nallapati, Christopher D Manning","year":"2009","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Riedl and Biemann, 2012] Martin Riedl and Chris Bie- mann. Topictiling: a text segmentation algorithm based on lda. In ACL Student Research Workshop, pages 37-42, 2012.","title":"Topictiling: a text segmentation algorithm based on lda","authors":"Martin Riedl, Chris Biemann","year":"2012","venue":"ACL Student Research Workshop","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Soleimani and Miller, 2016] Hossein Soleimani and David J Miller. Semi-supervised multi-label topic models for document classification and sentence labeling. In CIKM, pages 105-114, 2016.","title":"Semi-supervised multi-label topic models for document classification and sentence labeling","authors":"Hossein Soleimani, David J Miller","year":"2016","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2016] Yiping Song, Lili Mou, Rui Yan, Li Yi, Zinan Zhu, Xiaohua Hu, and Ming Zhang. Dialogue ses- sion segmentation by embedding-enhanced texttiling. In INTERSPEECH, pages 2706-2710, 2016.","title":"Dialogue session segmentation by embedding-enhanced texttiling","authors":"Yiping Song, Lili Mou, Rui Yan, Li Yi, Zinan Zhu, Xiaohua Hu, Ming Zhang","year":"2016","venue":"INTERSPEECH","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sutton et al., 2000] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approx- imation. In NIPS, pages 1057-1063, 2000.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"David A Richard S Sutton, Mcallester, P Satinder, Yishay Singh, Mansour","year":"2000","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Webber et al., 2012] Bonnie Webber, Markus Egg, and Valia Kordoni. Discourse structure and language tech- nology. Natural Language Engineering, 18(4):437-490, 2012.","title":"Discourse structure and language technology","authors":"Webber","year":"2012","venue":"Natural Language Engineering","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Williams et al., 2017] Jason D Williams, Kavosh Asadi, and Geoffrey Zweig. Hybrid code networks: practical and ef- ficient end-to-end dialog control with supervised and rein- forcement learning. In ACL, pages 665-677, 2017.","title":"Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning","authors":"Jason D Williams, Kavosh Asadi, Geoffrey Zweig","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Williams, 1992] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8(3-4):229-256, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Williams Ronald","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhai and Williams, 2014] Ke Zhai and Jason D Williams. Discovering latent structure in task-oriented dialogues. In ACL, pages 36-46, 2014.","title":"Discovering latent structure in task-oriented dialogues","authors":"Ke Zhai, Jason D Williams","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2011] Wayne Xin Zhao, Jing Jiang, Jianshu Weng, Jing He, Ee-Peng Lim, Hongfei Yan, and Xiaoming Li. Comparing twitter and traditional media using topic models. In ECIR, pages 338-349, 2011.","title":"Comparing twitter and traditional media using topic models","authors":"Jing Wayne Xin Zhao, Jianshu Jiang, Jing Weng, Ee-Peng He, Hongfei Lim, Xiaoming Yan, Li","year":"2011","venue":"ECIR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}