{"type":"ScholarlyArticle","title":"Chinese Poetry Generation with a Working Memory Model","authors":[{"name":"Xiaoyuan Yi","slug":"xiaoyuan-yi","given":"Xiaoyuan","family":"Yi","affiliation":"State Key Lab on Intelligent Technology and Systems , Beijing National Research Center for Information Science and Technology , Department of Computer Science and Technology , Tsinghua University , Beijing , China"},{"name":"Maosong Sun","slug":"maosong-sun","given":"Maosong","family":"Sun","affiliation":"State Key Lab on Intelligent Technology and Systems , Beijing National Research Center for Information Science and Technology , Department of Computer Science and Technology , Tsinghua University , Beijing , China"},{"name":"Ruoyu Li","slug":"ruoyu-li","given":"Ruoyu","family":"Li","affiliation":"6ESTATES PTE LTD , Singapore"},{"name":"Zonghan Yang","slug":"zonghan-yang","given":"Zonghan","family":"Yang","affiliation":"State Key Lab on Intelligent Technology and Systems , Beijing National Research Center for Information Science and Technology , Department of Computer Science and Technology , Tsinghua University , Beijing , China"}],"abstract":"As an exquisite and concise literary form, poetry is a gem of human culture. Automatic poetry generation is an essential step towards computer creativity. In recent years, several neural models have been designed for this task. However, among lines of a whole poem, the coherence in meaning and topics still remains a big challenge. In this paper, inspired by the theoretical concept in cognitive psychology, we propose a novel Working Memory model for poetry generation. Different from previous methods, our model explicitly maintains topics and informative limited history in a neural memory. During the generation process, our model reads the most relevant parts from memory slots to generate the current line. After each line is generated, it writes the most salient parts of the previous line into memory slots. By dynamic manipulation of the memory, our model keeps a coherent information flow and learns to express each topic flexibly and naturally. We experiment on three different genres of Chinese poetry: quatrain, iambic and chinoiserie lyric. Both automatic and human evaluation results show that our model outperforms current state-of-the-art methods.","doi":"10.24963/ijcai.2018/633","url":"https://ijcai.org/proceedings/2018/633","pdf":"https://ijcai.org/proceedings/2018/0633.pdf","markdown":"https://ijcai.org/proceedings/2018/633.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/633","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":633,"pages":{"first":4553,"last":4559},"keywords":["Natural Language Processing: Psycholinguistics","Natural Language Processing: Natural Language Generation","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceed- ings of the 2015 International Conference on Learning Representations, San Diego, CA, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"Proceedings of the 2015 International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceedings of the 2014 Confer- ence on Empirical Methods in Natural Language Process- ing, pages 1724-1734, Doha, Qatar, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fung, 2007] Anthony Y. H. Fung. The emerging (national) popular music culture in china. InterAsia Cultural Studies, 8(3):425-437, 2007.","title":"The emerging (national) popular music culture in china","authors":"Y H Anthony, Fung","year":"2007","venue":"InterAsia Cultural Studies","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ge, 2009] Zai Ge. cilinzhengyun. Shanghai Ancient Books Publishing House, 2009.","title":"cilinzhengyun","authors":"Zai Ge","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gervás, 2001] Pablo Gervás. An Expert System for the Composition of Formal Spanish Poetry, pages 181-188. Springer London, 2001.","title":"Pablo Gervás. An Expert System for the Composition of Formal Spanish Poetry","authors":"Gervás","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ghazvininejad et al., 2016] Marjan Ghazvininejad, Xing Shi, Yejin Choi, and Kevin Knight. Generating topical poetry. In Proceedings of the 2016 Conference on Em- pirical Methods in Natural Language Processing, pages 1183-1191, Texas, USA, 2016.","title":"Generating topical poetry","authors":"Marjan Ghazvininejad, Xing Shi, Yejin Choi, Kevin Knight","year":"2016","venue":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ghazvininejad et al., 2017] Marjan Ghazvininejad, Xing Shi, Jay Priyadarshi, and Kevin Knight. Hafez: an inter- active poetry generation system. In Proceedings of ACL 2017, System Demonstrations, pages 43-48. Association for Computational Linguistics, 2017.","title":"Hafez: an interactive poetry generation system","authors":"Marjan Ghazvininejad, Xing Shi, Jay Priyadarshi, Kevin Knight","year":"2017","venue":"Proceedings of ACL 2017, System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Graves et al., 2014] Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv preprint arXiv:1410.5401, 2014.","title":"Neural turing machines","authors":"Alex Graves, Greg Wayne, Ivo Danihelka","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gulcehre et al., 2017] Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho, and Yoshua Bengio. Dynamic neural tur- ing machine with soft and hard addressing schemes. arXiv preprint arXiv:1607.00036, 2017.","title":"Dynamic neural turing machine with soft and hard addressing schemes","authors":"Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2012] Jing He, Ming Zhou, and Long Jiang. Gen- erating chinese classical poems with statistical machine translation models. In Proceedings of the 26th AAAI Conference on Artificial Intelligence, pages 1650-1656, Toronto, Canada, 2012.","title":"Generating chinese classical poems with statistical machine translation models","authors":"Jing He, Ming Zhou, Long Jiang","year":"2012","venue":"Proceedings of the 26th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hopkins and Kiela, 2017] Jack Hopkins and Douwe Kiela. Automatically generating rhythmic verse with neural net- works. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 168- 178. Association for Computational Linguistics, 2017.","title":"Automatically generating rhythmic verse with neural networks","authors":"Kiela Hopkins, Jack Hopkins, Douwe Kiela","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Manurung, 2003] Hisar Maruli Manurung. An evolutionary algorithm approach to poetry generation. PhD thesis, Uni- versity of Edinburgh, 2003.","title":"An evolutionary algorithm approach to poetry generation","authors":"Hisar Maruli, Manurung","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[McCutchen, 2000] Deborah McCutchen. Knowledge, pro- cessing, and working memory: Implications for a theory of writing. Educational Psychologist, 35(1):13-23, 2000.","title":"Knowledge, processing, and working memory: Implications for a theory of writing","authors":"Deborah Mccutchen","year":"2000","venue":"Educational Psychologist","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mihalcea and Tarau, 2004] Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Lan- guage Processing, 2004.","title":"Textrank: Bringing order into text","authors":"Tarau Mihalcea, Rada Mihalcea, Paul Tarau","year":"2004","venue":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Priti and Miyake, 1999] Shah Priti and Akira Miyake. Mod- els of working memory: Mechanisms of active mainte- nance and executive control. Cambridge University Press, 1999.","title":"Models of working memory: Mechanisms of active maintenance and executive control","authors":"Shah Priti, Akira Miyake","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sanders et al., 2001] Ted Sanders, Joost Schilperoord, and Wilbert Spooren. Text representation: Linguistic and psy- cholinguistic aspects., pages 258-267.","title":"Text representation: Linguistic and psycholinguistic aspects","authors":"Sanders","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2016] Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. Mini- mum risk training for neural machine translation. In Pro- ceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1683-1692. Association for Computational Linguistics, 2016.","title":"Minimum risk training for neural machine translation","authors":"Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, Yang Liu","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tu et al., 2016] Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. Modeling coverage for neu- ral machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2016.","title":"Modeling coverage for neural machine translation","authors":"Tu","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2016a] Qixin Wang, Tianyi Luo, Dong Wang, and Chao Xing. Chinese song iambics generation with neural attention-based model. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 2943-2949, New York, USA, 2016.","title":"Chinese song iambics generation with neural attention-based model","authors":"Wang","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/418"},{"position":20,"raw":"[Wang et al., 2016b] Zhe Wang, Wei He, Hua Wu nad Haiyang Wu, Wei Li, Haifeng Wang, and Enhong Chen. Chinese poetry generation with planning based neu- ral network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguis- tics:Technical Papers, pages 1051-1060, Osaka, Japan, 2016.","title":"Chinese poetry generation with planning based neural network","authors":"Wang","year":"2016","venue":"Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics:Technical Papers","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Weston et al., 2015] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. In In International Conference on Learning Representations, San Diego, CA, 2015.","title":"Memory networks","authors":"Jason Weston, Sumit Chopra","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yan, 2016] Rui Yan. i,poet:automatic poetry composition through recurrent neural networks with iterative polishing schema. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 2238- 2244, New York, USA, 2016.","title":"poet:automatic poetry composition through recurrent neural networks with iterative polishing schema","authors":"Rui Yan","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and Lapata, 2014] Xingxing Zhang and Mirella Lap- ata. Chinese poetry generation with recurrent neural net- works. In Proceedings of the 2014 Conference on Em- pirical Methods in Natural Language Processing, pages 670-680, Doha, Qatar, 2014.","title":"Chinese poetry generation with recurrent neural networks","authors":"Lapata ; Zhang, Mirella Zhang, Lapata","year":"2014","venue":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2017] Jiyuan Zhang, Yang Feng, Dong Wang, Yang Wang, Andrew Abel, Shiyue Zhang, and Andi Zhang. Flexible and creative chinese poetry generation using neural memory. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1364-1373. Association for Computational Linguis- tics, 2017.","title":"Flexible and creative chinese poetry generation using neural memory","authors":"Jiyuan Zhang, Yang Feng, Dong Wang, Yang Wang, Andrew Abel, Shiyue Zhang, Andi Zhang","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}