{"type":"ScholarlyArticle","title":"Topic-to-Essay Generation with Neural Networks","authors":[{"name":"Xiaocheng Feng","slug":"xiaocheng-feng","given":"Xiaocheng","family":"Feng","affiliation":"Harbin Institute of Technology , China"},{"name":"Ming Liu","slug":"ming-liu","given":"Ming","family":"Liu","affiliation":"Harbin Institute of Technology , China"},{"name":"Jiahao Liu","slug":"jiahao-liu","given":"Jiahao","family":"Liu","affiliation":"Harbin Institute of Technology , China"},{"name":"Bing Qin","slug":"bing-qin","given":"Bing","family":"Qin","affiliation":"Harbin Institute of Technology , China"},{"name":"Yibo Sun","slug":"yibo-sun","given":"Yibo","family":"Sun","affiliation":"Harbin Institute of Technology , China"},{"name":"Ting Liu","slug":"ting-liu","given":"Ting","family":"Liu","affiliation":"Harbin Institute of Technology , China"}],"abstract":"We focus on essay generation, which is a challenging task that generates a paragraph-level text with multiple topics.Progress towards understanding different topics and expressing diversity in this task requires more powerful generators and richer training and evaluation resources. To address this,  we develop a multi-topic aware long short-term memory (MTA-LSTM) network.In this model, we maintain a novel multi-topic coverage vector, which learns the weight of each topic and is sequentially updated during the decoding process.Afterwards this vector is fed to an attention model to guide the generator.Moreover, we automatically construct two paragraph-level Chinese essay corpora, 305,000 essay paragraphs and 55,000 question-and-answer pairs.Empirical results show that our approach obtains much better BLEU score compared to various baselines.Furthermore, human judgment shows that MTA-LSTM has the ability to generate essays that are not only coherent but also closely related to the input topics.","doi":"10.24963/ijcai.2018/567","url":"https://ijcai.org/proceedings/2018/567","pdf":"https://ijcai.org/proceedings/2018/0567.pdf","markdown":"https://ijcai.org/proceedings/2018/567.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/567","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":567,"pages":{"first":4078,"last":4084},"keywords":["Natural Language Processing: Natural Language Generation","Machine Learning: Neural Networks","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Che et al., 2010] Wanxiang Che, Zhenghua Li, and Ting Liu. Ltp: A chinese language technology platform. In Proceedings of the 23rd International Conference on Com- putational Linguistics: Demonstrations, pages 13-16. As- sociation for Computational Linguistics, 2010.","title":"Ltp: A chinese language technology platform","authors":"Wanxiang Che, Zhenghua Li, Ting Liu","year":"2010","venue":"Proceedings of the 23rd International Conference on Computational Linguistics: Demonstrations","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the proper- ties of neural machine translation: Encoder-decoder ap- proaches. arXiv preprint arXiv:1409.1259, 2014.","title":"On the properties of neural machine translation: Encoder-decoder approaches","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2012] Jing He, Ming Zhou, and Long Jiang. Gen- erating chinese classical poems with statistical machine translation models. In AAAI, 2012.","title":"Generating chinese classical poems with statistical machine translation models","authors":"Jing He, Ming Zhou, Long Jiang","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Dan Jurafsky. Speech & language process- ing. Pearson Education India, 2000.","title":"Speech & language processing","authors":"Dan Jurafsky","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kiddon et al., 2016] Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi. Globally coherent text generation with neural checklist models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 329-339, 2016.","title":"Globally coherent text generation with neural checklist models","authors":"Kiddon","year":"2016","venue":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Manning et al., 1999] Christopher D Manning, Hinrich Schütze, et al. Foundations of statistical natural language processing, volume 999. MIT Press, 1999.","title":"Foundations of statistical natural language processing","authors":"Hinrich Christopher D Manning, Schütze","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[McKeown, 1992] Kathleen McKeown. Text generation. Cambridge University Press, 1992.","title":"Text generation","authors":"Kathleen Mckeown","year":"1992","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mihalcea and Tarau, 2004] Rada Mihalcea and Paul Tarau. Textrank: Bringing order into texts. Association for Com- putational Linguistics, 2004.","title":"Textrank: Bringing order into texts","authors":"Tarau Mihalcea, Rada Mihalcea, Paul Tarau","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for auto- matic evaluation of machine translation. In Computational linguistics, pages 311-318. Association for Computational Linguistics, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Computational linguistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Qin et al., 2015] Bing Qin, Duyu Tang, Xinwei Geng, Dan- dan Ning, Jiahao Liu, and Ting Liu. A planning based framework for essay generation. arXiv preprint arXiv:1512.05919, 2015.","title":"A planning based framework for essay generation","authors":"Qin","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Reiter et al., 2000] Ehud Reiter, Robert Dale, and Zhiwei Feng. Building natural language generation systems, vol- ume 33. MIT Press, 2000.","title":"Building natural language generation systems","authors":"Ehud Reiter, Robert Dale, Zhiwei Feng","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sauper and Barzilay, 2009] Christina Sauper and Regina Barzilay. Automatically generating wikipedia articles: A structure-aware approach. In ACL, pages 208-216. Asso- ciation for Computational Linguistics, 2009.","title":"Automatically generating wikipedia articles: A structure-aware approach","authors":"Barzilay Sauper, Christina Sauper, Regina Barzilay","year":"2009","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sutskever et al., 2011] Ilya Sutskever, James Martens, and Geoffrey E Hinton. Generating text with recurrent neural networks. In Proceedings of the 28th International Confer- ence on Machine Learning (ICML-11), pages 1017-1024, 2011.","title":"Generating text with recurrent neural networks","authors":"Sutskever","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning (ICML-11)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tu et al., 2016] Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. Modeling coverage for neu- ral machine translation. arXiv preprint arXiv:1601.04811, 2016.","title":"Modeling coverage for neural machine translation","authors":"Tu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2016a] Qixin Wang, Tianyi Luo, Dong Wang, and Chao Xing. Chinese song iambics generation with neural attention-based model. arXiv preprint arXiv:1604.06274, 2016.","title":"Chinese song iambics generation with neural attention-based model","authors":"Wang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/418"},{"position":18,"raw":"[Wang et al., 2016b] Zhe Wang, Wei He, Hua Wu, Haiyang Wu, Wei Li, Haifeng Wang, and Enhong Chen. Chinese poetry generation with planning based neural network. arXiv preprint arXiv:1610.09889, 2016.","title":"Chinese poetry generation with planning based neural network","authors":"Wang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wen et al., 2015] Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. Semantically conditioned lstm-based natural language generation for spoken dialogue systems. arXiv preprint arXiv:1508.01745, 2015.","title":"Semantically conditioned lstm-based natural language generation for spoken dialogue systems","authors":"Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, Steve Young","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xing et al., 2016] Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. Topic aware neural response generation. arXiv preprint arXiv:1606.08340, 2016.","title":"Topic aware neural response generation","authors":"Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yan and Wan, 2015] Su Yan and Xiaojun Wan. Deep de- pendency substructure-based learning for multidocument summarization. ACM Transactions on Information Sys- tems (TOIS), 34(1):3, 2015.","title":"Deep dependency substructure-based learning for multidocument summarization","authors":"Yan, Wan ; ] Su, Yan, Xiaojun Wan","year":"2015","venue":"ACM Transactions on Information Systems (TOIS)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yin et al., 2015] Jun Yin, Xin Jiang, Zhengdong Lu, Lifeng Shang, Hang Li, and Xiaoming Li. Neural generative question answering. arXiv preprint arXiv:1512.01337, 2015.","title":"Neural generative question answering","authors":"Jun Yin, Xin Jiang, Zhengdong Lu, Lifeng Shang, Hang Li, Xiaoming Li","year":"2015","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/422"},{"position":23,"raw":"[Zeiler, 2012] Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012.","title":"Adadelta: an adaptive learning rate method","authors":"D Matthew, Zeiler","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zeng et al., 2016] Wenyuan Zeng, Wenjie Luo, Sanja Fi- dler, and Raquel Urtasun. Efficient summarization with read-again and copy mechanism. arXiv preprint arXiv:1611.03382, 2016.","title":"Efficient summarization with read-again and copy mechanism","authors":"Wenyuan Zeng, Wenjie Luo, Sanja Fidler, Raquel Urtasun","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Lapata, 2014] Xingxing Zhang and Mirella Lap- ata. Chinese poetry generation with recurrent neural net- works. In EMNLP, pages 670-680, 2014.","title":"Xingxing Zhang and Mirella Lapata. Chinese poetry generation with recurrent neural networks","authors":"Lapata Zhang","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}