{"type":"ScholarlyArticle","title":"Learning Conversational Systems that Interleave Task and Non-Task Content","authors":[{"name":"Zhou Yu","slug":"zhou-yu","given":"Zhou","family":"Yu","affiliation":"Carnegie Mellon University 5000 Forbes Avenue , Pittsburgh , PA , 15213"},{"name":"Alexander Rudnicky","slug":"alexander-rudnicky","given":"Alexander","family":"Rudnicky","affiliation":"Carnegie Mellon University 5000 Forbes Avenue , Pittsburgh , PA , 15213"},{"name":"Alan Black","slug":"alan-black","given":"Alan","family":"Black","affiliation":"Carnegie Mellon University 5000 Forbes Avenue , Pittsburgh , PA , 15213"}],"abstract":"Task-oriented dialog systems have been applied in various tasks, such as automated personal assistants, customer service providers and tutors. These systems work well when users have clear and explicit intentions that are well-aligned to the systems' capabilities. However, they fail if users intentions are not explicit.To address this shortcoming, we propose a framework to interleave non-task content (i.e.everyday social conversation) into task conversations. When the task content fails, the system can still keep the user engaged with the non-task content. We trained a policy using reinforcement learning algorithms to promote  long-turn conversation coherence and consistency, so that the system can have smooth transitions between task and non-task content.To test the effectiveness of the proposed framework, we developed a movie promotion dialog system. Experiments with human users indicate that a system that interleaves social and task content achieves a better task success rate and is also rated as more engaging compared to a pure task-oriented system.","doi":"10.24963/ijcai.2017/589","url":"https://ijcai.org/proceedings/2017/589","pdf":"https://ijcai.org/proceedings/2017/0589.pdf","markdown":"https://ijcai.org/proceedings/2017/589.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/589","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":589,"pages":{"first":4214,"last":4220},"keywords":["Natural Language Processing: Dialogue","Natural Language Processing: NLP Applications and Tools"],"videos":[],"references":[{"position":1,"raw":"[Banchs and Li, 2012] Rafael E Banchs and Haizhou Li. Iris: a chat-oriented dialogue system based on the vector space model. In Proceedings of the ACL, 2012.","title":"Iris: a chat-oriented dialogue system based on the vector space model","authors":"Li ; Banchs, E Rafael, Haizhou Banchs, Li","year":"2012","venue":"Proceedings of the ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes and Weston, 2016] Antoine Bordes and Jason We- ston. Learning end-to-end goal-oriented dialog. arXiv preprint arXiv:1605.07683, 2016.","title":"Learning end-to-end goal-oriented dialog","authors":"Weston Bordes","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dodge et al., 2015] Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander Miller, Arthur Szlam, and Jason Weston. Evaluating prerequisite qualities for learning end-to-end dialog systems. arXiv preprint arXiv:1511.06931, 2015.","title":"Evaluating prerequisite qualities for learning end-to-end dialog systems","authors":"Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander Miller, Arthur Szlam, Jason Weston","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He and Young, 1998] Agnes Weiyun He and Richard Young. Language proficiency interviews: A discourse ap- proach. Talking and testing: Discourse approaches to the assessment of oral proficiency, 14:1-24, 1998.","title":"Agnes Weiyun He and Richard Young. Language proficiency interviews: A discourse approach. Talking and testing: Discourse approaches to the assessment of oral proficiency","authors":"Young He","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Higashinaka et al., 2014] Ryuichiro Higashinaka, Kenji Imamura, Toyomi Meguro, Chiaki Miyazaki, Nozomi Kobayashi, Hiroaki Sugiyama, Toru Hirano, Toshiro Makino, and Yoshihiro Matsuo. Towards an open-domain conversational system fully based on natural language processing. In COLING, pages 928-939, 2014.","title":"Towards an open-domain conversational system fully based on natural language processing","authors":"Ryuichiro Higashinaka, Kenji Imamura, Toyomi Meguro, Chiaki Miyazaki, Nozomi Kobayashi, Hiroaki Sugiyama, Toru Hirano, Toshiro Makino, Yoshihiro Matsuo","year":"2014","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jia and Ruan, 2008] Jiyou Jia and Meixian Ruan. Use chat- bot csiec to facilitate the individual learning in english in- struction: A case study. In International Conference on In- telligent Tutoring Systems, pages 706-708. Springer, 2008.","title":"Use chatbot csiec to facilitate the individual learning in english instruction: A case study","authors":"Ruan Jia, Jiyou Jia, Meixian Ruan","year":"2008","venue":"International Conference on Intelligent Tutoring Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jiang et al., 2015] Jiepu Jiang, Ahmed Hassan Awadallah, Rosie Jones, Umut Ozertem, Imed Zitouni, Ranjitha Gu- runath Kulkarni, and Omar Zia Khan. Automatic online evaluation of intelligent assistants. In WWW, 2015.","title":"Automatic online evaluation of intelligent assistants","authors":"Jiang","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jurcıcek et al., 2011] Filip Jurcıcek, Simon Keizer, Milica Gašic, Francois Mairesse, Blaise Thomson, Kai Yu, and Steve Young. Real user evaluation of spoken dialogue sys- tems using amazon mechanical turk. In Proceedings of INTERSPEECH, volume 11, 2011.","title":"Real user evaluation of spoken dialogue systems using amazon mechanical turk","authors":"Filip Jurcıcek, Simon Keizer, Milica Gašic, Francois Mairesse, Blaise Thomson, Kai Yu, Steve Young","year":"2011","venue":"Proceedings of INTERSPEECH","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kiros et al., 2015] Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In Ad- vances in neural information processing systems, 2015.","title":"Skip-thought vectors","authors":"Ryan Kiros, Yukun Zhu, R Ruslan, Richard Salakhutdinov, Raquel Zemel, Antonio Urtasun, Sanja Torralba, Fidler","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2016] Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. Deep reinforce- ment learning for dialogue generation. arXiv, 2016.","title":"Deep reinforcement learning for dialogue generation","authors":"Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, Dan Jurafsky","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lison and Tiedemann, 2016] Pierre Lison and Jörg Tiede- mann. Opensubtitles2016: Extracting large parallel cor- pora from movie and tv subtitles. In LREC, 2016.","title":"Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles","authors":"Lison, Tiedemann, Pierre Lison, Jörg Tiedemann","year":"2016","venue":"LREC","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Martin, 2002] J. R. Martin. Meaning beyond the clause: area: self-perspectives. Annual Review of Applied Lin- guistics 22, 2002.","title":"Meaning beyond the clause: area: self-perspectives","authors":"J R Martin","year":"2002","venue":"Annual Review of Applied Linguistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mesnil et al., 2013] Grégoire Mesnil, Xiaodong He, Li Deng, and Yoshua Bengio. Investigation of recurrent- neural-network architectures and learning methods for spoken language understanding. In Interspeech, 2013.","title":"Investigation of recurrentneural-network architectures and learning methods for spoken language understanding","authors":"Grégoire Mesnil, Xiaodong He, Li Deng, Yoshua Bengio","year":"2013","venue":"Interspeech","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[MPAA, 2015] MPAA. Theatrical market statistics 2015. In Motion Picture of American Association, 2015.","title":"MPAA. Theatrical market statistics 2015","authors":"Mpaa","year":"2015","venue":"Motion Picture of American Association","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Raux et al., 2005] Antoine Raux, Brian Langner, Dan Bo- hus, Alan W Black, and Maxine Eskenazi. Lets go public! taking a spoken dialog system to the real world. In in Proc. of Interspeech 2005. Citeseer, 2005.","title":"Lets go public! taking a spoken dialog system to the real world","authors":"Raux","year":"2005","venue":"Proc. of Interspeech 2005","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ritter et al., 2011] Alan Ritter, Colin Cherry, and William B Dolan. Data-driven response generation in social media. In Proceedings of the conference on empirical methods in natural language processing, pages 583-593. Association for Computational Linguistics, 2011.","title":"Data-driven response generation in social media","authors":"Alan Ritter, Colin Cherry, William B Dolan","year":"2011","venue":"Proceedings of the conference on empirical methods in natural language processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rudnicky and Xu, 1999] Alexander Rudnicky and Wei Xu. An agenda-based dialog management architecture for spo- ken language systems. In IEEE Automatic Speech Recog- nition and Understanding Workshop, volume 13, 1999.","title":"An agenda-based dialog management architecture for spoken language systems","authors":"Alexander Rudnicky, Wei Xu","year":"1999","venue":"IEEE Automatic Speech Recognition and Understanding Workshop","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schegloff, 1968] Emanuel A Schegloff. Sequencing in con- versational openings1. American anthropologist, 1968.","title":"Sequencing in conversational openings1","authors":"A Emanuel, Schegloff","year":"1968","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Serban et al., 2015] Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. Building end-to-end dialogue systems using generative hi- erarchical neural network models. arXiv, 2015.","title":"Building end-to-end dialogue systems using generative hierarchical neural network models","authors":"Alessandro Iulian V Serban, Yoshua Sordoni, Aaron Bengio, Joelle Courville, Pineau","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sutton and Barto, 1998] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, vol- ume 1. MIT press Cambridge, 1998.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vinyals and Le, 2015] Oriol Vinyals and Quoc Le. A neural conversational model. In ICML Workshop, 2015.","title":"Oriol Vinyals and Quoc Le. A neural conversational model","authors":"Le Vinyals","year":"2015","venue":"ICML Workshop","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wendler, 2014] Daniel Wendler. Improve your social skills. CreateSpace Independent Publishing Platform, 2014.","title":"Improve your social skills","authors":"Daniel Wendler","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Williams and Young, 2007] Jason D Williams and Steve Young. Partially observable markov decision processes for spoken dialog systems. Computer Speech&Language, 2007.","title":"Partially observable markov decision processes for spoken dialog systems","authors":"Jason D Williams, Steve Young","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Williams et al., 2015] Jason D Williams, Eslam Kamal, Hani Amr Mokhtar Ashour, Jessica Miller, and Geoff Zweig. Fast and easy language understanding for dialog systems with microsoft language understanding intelligent service (luis). In SIGDIAL, page 159, 2015.","title":"Fast and easy language understanding for dialog systems with microsoft language understanding intelligent service (luis)","authors":"Jason D Williams, Eslam Kamal, Hani Amr Mokhtar, Jessica Ashour, Geoff Miller, Zweig","year":"2015","venue":"SIGDIAL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2015a] Zhou Yu, Dan Bohus, and Eric Horvitz. In- cremental coordination: Attention-centric speech produc- tion in a physically situated conversational agent. In SIG- DIAL, page 402, 2015.","title":"Incremental coordination: Attention-centric speech production in a physically situated conversational agent","authors":"Yu","year":"2015","venue":"SIG-DIAL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2015b] Zhou Yu, Alexandros Papangelis, and Alexander Rudnicky. Ticktock: A non-goal-oriented mul- timodal dialog system with engagement awareness. In Proceedings of the AAAI Spring Symposium, 2015.","title":"Ticktock: A non-goal-oriented multimodal dialog system with engagement awareness","authors":"Yu","year":"2015","venue":"Proceedings of the AAAI Spring Symposium","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2016a] Zhou Yu, Ziyu Xu, Alan Black, and Alex Rudnicky. Strategy and policy learning for non-task- oriented conversational systems. In SIGDIAL, 2016.","title":"Strategy and policy learning for non-taskoriented conversational systems","authors":"Yu","year":"2016","venue":"SIGDIAL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yu et al., 2016b] Zhou Yu, Ziyu Xu, Alan Black, and Alexander Rudnicky. Chatbot evaluation and database ex- pansion via crowdsourcing. In Proceedings of the chatbot workshop of LREC, 2016.","title":"Chatbot evaluation and database expansion via crowdsourcing","authors":"Yu","year":"2016","venue":"Proceedings of the chatbot workshop of LREC","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zue et al., 1994] Victor Zue, Stephanie Seneff, Joseph Po- lifroni, Michael Phillips, Christine Pao, David Goodine, David Goddeau, and James Glass. Pegasus: A spoken di- alogue interface for on-line air travel planning. Speech Communication, 15(3-4):331-340, 1994.","title":"Pegasus: A spoken dialogue interface for on-line air travel planning","authors":"Stephanie Victor Zue, Joseph Seneff, Michael Polifroni, Christine Phillips, David Pao, David Goodine, James Goddeau, Glass","year":"1994","venue":"Speech Communication","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}