{"type":"ScholarlyArticle","title":"T-CVAE: Transformer-Based Conditioned Variational Autoencoder for Story Completion","authors":[{"name":"Tianming Wang","slug":"tianming-wang","given":"Tianming","family":"Wang","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"},{"name":"Xiaojun Wan","slug":"xiaojun-wan","given":"Xiaojun","family":"Wan","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"}],"abstract":"Story completion is a very challenging task of generating the missing plot for an incomplete story, which requires not only understanding but also inference of the given contextual clues. In this paper, we present a novel conditional variational autoencoder based on Transformer for missing plot generation. Our model uses shared attention layers for encoder and decoder, which make the most of the contextual clues, and a latent variable for learning the distribution of coherent story plots. Through drawing samples from the learned distribution, diverse reasonable plots can be generated. Both automatic and manual evaluations show that our model generates better story plots than state-of-the-art models in terms of readability, diversity and coherence.","doi":"10.24963/ijcai.2019/727","url":"https://ijcai.org/proceedings/2019/727","pdf":"https://ijcai.org/proceedings/2019/0727.pdf","markdown":"https://ijcai.org/proceedings/2019/727.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/727","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":727,"pages":{"first":5233,"last":5239},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Generation"],"videos":[],"references":[{"position":1,"raw":"[Bamman et al., 2014] David Bamman, Brendan O'Connor, and Noah A Smith. Learning latent personas of film char- acters. In ACL, page 352, 2014.","title":"Learning latent personas of film characters","authors":"David Bamman, O' Brendan, Noah A Connor, Smith","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Black and Bower, 1980] John B Black and Gordon H Bower. Story understanding as problem-solving. Poetics, 9(1-3):223-250, 1980.","title":"Story understanding as problem-solving","authors":"B John, Gordon H Black, Bower","year":"1980","venue":"Poetics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao and Clark, 2017] Kris Cao and Stephen Clark. Latent variable dialogue models and their diversity. In EACL, Short Papers, volume 2, pages 182-187, 2017.","title":"Latent variable dialogue models and their diversity","authors":"Clark ; Cao, Kris Cao, Stephen Clark","year":"2017","venue":"EACL, Short Papers","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chambers and Jurafsky, 2008] Nathanael Chambers and Daniel Jurafsky. Unsupervised learning of narrative event chains. In ACL, volume 94305, pages 789-797, 2008.","title":"Unsupervised learning of narrative event chains","authors":"Jurafsky Chambers, Nathanael Chambers, Daniel Jurafsky","year":"2008","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chambers and Jurafsky, 2009] Nathanael Chambers and Dan Jurafsky. Unsupervised learning of narrative schemas and their participants. In ACL-IJCNLP 2009: Volume 2-Volume 2, pages 602-610, 2009.","title":"Unsupervised learning of narrative schemas and their participants","authors":"Jurafsky Chambers, Nathanael Chambers, Dan Jurafsky","year":"2009","venue":"ACL-IJCNLP 2009","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chaturvedi et al., 2017] Snigdha Chaturvedi, Haoruo Peng, and Dan Roth. Story comprehension for predicting what happens next. In EMNLP, pages 1603-1614, 2017.","title":"Story comprehension for predicting what happens next","authors":"Snigdha Chaturvedi, Haoruo Peng, Dan Roth","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2018] Jiaao Chen, Jianshu Chen, and Zhou Yu. Incorporating structured commonsense knowledge in story completion. arXiv preprint arXiv:1811.00625, 2018.","title":"Incorporating structured commonsense knowledge in story completion","authors":"Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fan et al., 2018] Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In ACL, volume 1, pages 889-898, 2018.","title":"Hierarchical neural story generation","authors":"Angela Fan, Mike Lewis, Yann Dauphin","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goyal et al., 2010] Amit Goyal, Ellen Riloff, and Hal Daumé III. Automatically producing plot unit represen- tations for narrative text. In EMNLP, pages 77-86, 2010.","title":"Automatically producing plot unit representations for narrative text","authors":"Amit Goyal, Ellen Riloff, Hal Daumé, Iii","year":"2010","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guan et al., 2018] Jian Guan, Yansen Wang, and Minlie Huang. Story ending generation with incremental en- coding and commonsense knowledge. arXiv preprint arXiv:1808.10113, 2018.","title":"Story ending generation with incremental encoding and commonsense knowledge","authors":"Jian Guan, Yansen Wang, Minlie Huang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2013] Boyang Li, Stephen Lee-Urban, George Johnston, and Mark Riedl. Story generation with crowd- sourced plot graphs. In AAAI, 2013.","title":"Story generation with crowdsourced plot graphs","authors":"Boyang Li, Stephen Lee-Urban, George Johnston, Mark Riedl","year":"2013","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2015] Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting ob- jective function for neural conversation models. arXiv preprint arXiv:1510.03055, 2015.","title":"A diversity-promoting objective function for neural conversation models","authors":"Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, Bill Dolan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2017] Jiwei Li, Will Monroe, Tianlin Shi, Sėbastien Jean, Alan Ritter, and Dan Jurafsky. Adversarial learning for neural dialogue generation. In EMNLP, pages 2157-2169, 2017.","title":"Adversarial learning for neural dialogue generation","authors":"Jiwei Li, Will Monroe, Tianlin Shi, Sėbastien Jean, Alan Ritter, Dan Jurafsky","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2018] Zhongyang Li, Xiao Ding, and Ting Liu. Generating reasonable and diversified story ending using sequence to sequence model with adversarial training. In COLING, pages 1033-1043, 2018.","title":"Generating reasonable and diversified story ending using sequence to sequence model with adversarial training","authors":"Zhongyang Li, Xiao Ding, Ting Liu","year":"2018","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Martin et al., 2018] Lara J Martin, Prithviraj Am- manabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O Riedl. Event represen- tations for automated story generation with deep neural nets. In AAAI, 2018.","title":"Event representations for automated story generation with deep neural nets","authors":"Prithviraj Lara J Martin, Xinyu Ammanabrolu, William Wang, Shruti Hancock, Brent Singh, Mark O Harrison, Riedl","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mostafazadeh et al., 2016a] Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. In NAACL-HLT, pages 839-849, 2016.","title":"A corpus and cloze evaluation for deeper understanding of commonsense stories","authors":"Mostafazadeh","year":"2016","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mostafazadeh et al., 2016b] Nasrin Mostafazadeh, Lucy Vanderwende, Wen-tau Yih, Pushmeet Kohli, and James Allen. Story cloze evaluator: Vector space representation evaluation by predicting what happens next. In Proceed- ings of the 1st Workshop on Evaluating Vector-Space Rep- resentations for NLP, pages 24-29, 2016.","title":"Story cloze evaluator: Vector space representation evaluation by predicting what happens next","authors":"Mostafazadeh","year":"2016","venue":"Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In ACL, pages 311-318, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rezende et al., 2014] Danilo Jimenez Rezende, Shakir Mo- hamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In ICML, pages 1278-1286, 2014.","title":"Stochastic backpropagation and approximate inference in deep generative models","authors":"Danilo Jimenez Rezende, Shakir Mohamed, Daan Wierstra","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schank and Abelson, 1977] RC Schank and R Abelson. Script, plans, goals and understanding: An inquiry into human knowledge structures. 1977.","title":"Script, plans, goals and understanding: An inquiry into human knowledge structures","authors":"Abelson ; Rc Schank, R Schank, Abelson","year":"1977","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Serban et al., 2017] Iulian Vlad Serban, Alessandro Sor- doni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, and Yoshua Bengio. A hierarchical latent vari- able encoder-decoder model for generating dialogues. In AAAI, pages 3295-3301, 2017.","title":"A hierarchical latent variable encoder-decoder model for generating dialogues","authors":"Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, Yoshua Bengio","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Winograd, 1972] Terry Winograd. Understanding natural language. Cognitive psychology, 3(1):1-191, 1972.","title":"Terry Winograd. Understanding natural language","authors":"Winograd","year":"1972","venue":"Cognitive psychology","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2018] Jingjing Xu, Xuancheng Ren, Yi Zhang, Qi Zeng, Xiaoyan Cai, and Xu Sun. A skeleton-based model for promoting coherence among sentences in narra- tive story generation. In EMNLP, pages 4306-4315, 2018.","title":"A skeleton-based model for promoting coherence among sentences in narrative story generation","authors":"Jingjing Xu, Xuancheng Ren, Yi Zhang, Qi Zeng, Xiaoyan Cai, Xu Sun","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yao et al., 2018] Lili Yao, Nanyun Peng, Weischedel Ralph, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan- and-write: Towards better automatic storytelling. arXiv preprint arXiv:1811.05701, 2018.","title":"Planand-write: Towards better automatic storytelling","authors":"Lili Yao, Nanyun Peng, Weischedel Ralph, Kevin Knight, Dongyan Zhao, Rui Yan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao et al., 2017] Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. Learning discourse-level diversity for neural di- alog models using conditional variational autoencoders. In ACL, volume 1, pages 654-664, 2017.","title":"Learning discourse-level diversity for neural dialog models using conditional variational autoencoders","authors":"Zhao","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}