{"type":"ScholarlyArticle","title":"Learning towards Abstractive Timeline Summarization","authors":[{"name":"Xiuying Chen","slug":"xiuying-chen","given":"Xiuying","family":"Chen","affiliation":"Center for Data Science , Peking University , Beijing , China; Institute of Computer Science and Technology , Peking University , Beijing , China"},{"name":"Zhangming Chan","slug":"zhangming-chan","given":"Zhangming","family":"Chan","affiliation":"Center for Data Science , Peking University , Beijing , China; Institute of Computer Science and Technology , Peking University , Beijing , China"},{"name":"Shen Gao","slug":"shen-gao","given":"Shen","family":"Gao","affiliation":"Institute of Computer Science and Technology , Peking University , Beijing , China"},{"name":"Meng-Hsuan Yu","slug":"meng-hsuan-yu","given":"Meng-Hsuan","family":"Yu","affiliation":"Institute of Computer Science and Technology , Peking University , Beijing , China"},{"name":"Dongyan Zhao","slug":"dongyan-zhao","given":"Dongyan","family":"Zhao","affiliation":"Center for Data Science , Peking University , Beijing , China; Institute of Computer Science and Technology , Peking University , Beijing , China"},{"name":"Rui Yan","slug":"rui-yan","given":"Rui","family":"Yan","affiliation":"Center for Data Science , Peking University , Beijing , China; Institute of Computer Science and Technology , Peking University , Beijing , China"}],"abstract":"Timeline summarization targets at concisely summarizing the evolution trajectory along the timeline and existing timeline summarization approaches are all based on extractive methods.In this paper, we propose the task of abstractive timeline summarization, which tends to concisely paraphrase the information in the time-stamped events.Unlike traditional document summarization, timeline summarization needs to model the time series information of the input events and summarize important events in chronological order.To tackle this challenge, we propose a memory-based timeline summarization model (MTS).Concretely, we propose a time-event memory to establish a timeline, and use the time position of events on this timeline to guide generation process.Besides, in each decoding step, we incorporate event-level information into word-level attention to avoid confusion between events.Extensive experiments are conducted on a large-scale real-world dataset, and the results show that MTS achieves the state-of-the-art performance in terms of both automatic and human evaluations.","doi":"10.24963/ijcai.2019/686","url":"https://ijcai.org/proceedings/2019/686","pdf":"https://ijcai.org/proceedings/2019/0686.pdf","markdown":"https://ijcai.org/proceedings/2019/686.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/686","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":686,"pages":{"first":4939,"last":4945},"keywords":["Natural Language Processing: Natural Language Generation","Natural Language Processing: Natural Language Summarization"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: a system for large-scale machine learn- ing. In OSDI, volume 16, pages 265-283, 2016.","title":"Tensorflow: a system for large-scale machine learning","authors":"Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard","year":"2016","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Allan et al., 2001] James Allan, Rahul Gupta, and Vikas Khandelwal. Temporal summaries of new topics. In SI- GIR, pages 10-18. ACM, 2001.","title":"Temporal summaries of new topics","authors":"James Allan, Rahul Gupta, Vikas Khandelwal","year":"2001","venue":"SI-GIR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2018] Ziqiang Cao, Furu Wei, Wenjie Li, and Su- jian Li. Faithful to the original: Fact aware neural abstrac- tive summarization. In AAAI, 2018.","title":"Faithful to the original: Fact aware neural abstractive summarization","authors":"Ziqiang Cao, Furu Wei, Wenjie Li, Sujian Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2018] Xiuying Chen, Shen Gao, Chongyang Tao, Yan Song, Dongyan Zhao, and Rui Yan. Iterative document representation learning towards summarization with polishing. EMNLP, 2018.","title":"Iterative document representation learning towards summarization with polishing","authors":"Xiuying Chen, Shen Gao, Chongyang Tao, Yan Song, Dongyan Zhao, Rui Yan","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duchi et al., 2010] John C. Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. JMLR, 12:2121-2159, 2010.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"John C Duchi, Elad Hazan, Yoram Singer","year":"2010","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Grave et al., 2017] Edouard Grave, Armand Joulin, and Nicolas Usunier. Improving neural language models with a continuous cache. ICLR, 2017.","title":"Improving neural language models with a continuous cache","authors":"Grave","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gu et al., 2016] Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. Incorporating copying mechanism in sequence-to-sequence learning. CoRR, abs/1603.06393, 2016.","title":"Incorporating copying mechanism in sequence-to-sequence learning","authors":"Gu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hsu et al., 2018] Wan-Ting Hsu, Chieh-Kai Lin, Ming- Ying Lee, Kerui Min, Jing Tang, and Min Sun. A unified model for extractive and abstractive summarization using inconsistency loss. pages 132-141. ACL, 2018.","title":"A unified model for extractive and abstractive summarization using inconsistency loss","authors":"Wan-Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, Min Sun","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kaiser et al., 2017] Łukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. Learning to remember rare events. ICLR, 2017.","title":"Learning to remember rare events","authors":"Łukasz Kaiser, Ofir Nachum, Aurko Roy, Samy Bengio","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Landis and Koch, 1977] J Richard Landis and Gary G Koch. The measurement of observer agreement for cat- egorical data. biometrics, pages 159-174, 1977.","title":"The measurement of observer agreement for categorical data","authors":"Koch ; J Richard Landis, Gary G Landis, Koch","year":"1977","venue":"biometrics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Li, 2013] Jiwei Li and Sujian Li. Evolutionary hier- archical dirichlet process for timeline summarization. In ACL, volume 2, pages 556-560, 2013.","title":"Evolutionary hierarchical dirichlet process for timeline summarization","authors":"Li ; ] Jiwei Li, Sujian Li, Li","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2018] Jing Li, Aixin Sun, Jianglei Han, and Chen- liang Li. A survey on deep learning for named entity recognition. arXiv preprint arXiv:1812.09449, 2018.","title":"A survey on deep learning for named entity recognition","authors":"Jing Li, Aixin Sun, Jianglei Han, Chenliang Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out, 2004.","title":"Rouge: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text Summarization Branches Out","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mihalcea and Tarau, 2004] Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In EMNLP, 2004.","title":"Textrank: Bringing order into text","authors":"Tarau Mihalcea, Rada Mihalcea, Paul Tarau","year":"2004","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Miller et al., 2016] Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. Key-value memory networks for directly reading documents. In EMNLP, pages 1400-1409. ACL, 2016.","title":"Key-value memory networks for directly reading documents","authors":"Miller","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nallapati et al., 2017] Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. Summarunner: A recurrent neural net- work based sequence model for extractive summarization of documents. In AAAI, 2017.","title":"Summarunner: A recurrent neural network based sequence model for extractive summarization of documents","authors":"Ramesh Nallapati, Feifei Zhai, Bowen Zhou","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pritzel et al., 2017] Alexander Pritzel, Benigno Uria, Sri- ram Srinivasan, Adria Puigdomenech, Oriol Vinyals, Demis Hassabis, Daan Wierstra, and Charles Blundell. Neural episodic control. arXiv preprint arXiv:1703.01988, 2017.","title":"Neural episodic control","authors":"Alexander Pritzel, Benigno Uria, Sriram Srinivasan, Adria Puigdomenech, Oriol Vinyals, Demis Hassabis, Daan Wierstra, Charles Blundell","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ren et al., 2013] Zhaochun Ren, Shangsong Liang, Edgar Meij, and Maarten de Rijke. Personalized time-aware tweets summarization. In SIGIR, pages 513-522. ACM, 2013.","title":"Personalized time-aware tweets summarization","authors":"Ren","year":"2013","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schluter, 2017] Natalie Schluter. The limits of automatic summarisation according to rouge. In Proceedings of the 15th Conference of the European Chapter of the Associa- tion for Computational Linguistics: Volume 2, Short Pa- pers, pages 41-45. ACL, 2017.","title":"The limits of automatic summarisation according to rouge","authors":"Natalie Schluter","year":"2017","venue":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[See et al., 2017] Abigail See, Peter J. Liu, and Christo- pher D. Manning. Get to the point: Summarization with pointer-generator networks. pages 1073-1083. ACL, 2017.","title":"Get to the point: Summarization with pointer-generator networks","authors":"Abigail See, Peter J Liu, Christopher D Manning","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sun et al., 2018] Min Sun, Wan Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, and Jing Tang. A unified model for extractive and abstractive summarization using inconsistency loss. In ACL, 2018.","title":"A unified model for extractive and abstractive summarization using inconsistency loss","authors":"Sun","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Weston et al., 2015] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. ICLR, abs/1410.3916, 2015.","title":"Antoine Bordes","authors":"Jason Weston, Sumit Chopra","year":"2015","venue":"Memory networks","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yan et al., 2011a] Rui Yan, Liang Kong, Congrui Huang, Xiaojun Wan, Xiaoming Li, and Yan Zhang. Timeline gen- eration through evolutionary trans-temporal summariza- tion. In EMNLP, pages 433-443. ACL, 2011.","title":"Timeline generation through evolutionary trans-temporal summarization","authors":"Yan","year":"2011","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yan et al., 2011b] Rui Yan, Xiaojun Wan, Jahna Otter- bacher, Liang Kong, Xiaoming Li, and Yan Zhang. Evolu- tionary timeline summarization: a balanced optimization framework via iterative substitution. In SIGIR, pages 745- 754. ACM, 2011.","title":"Evolutionary timeline summarization: a balanced optimization framework via iterative substitution","authors":"Yan","year":"2011","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yan et al., 2012] Rui Yan, Xiaojun Wan, Mirella Lapata, Wayne Xin Zhao, Pu-Jen Cheng, and Xiaoming Li. Vi- sualizing timelines: Evolutionary summarization via iter- ative reinforcement between text and image streams. In CIKM, pages 275-284. ACM, 2012.","title":"Visualizing timelines: Evolutionary summarization via iterative reinforcement between text and image streams","authors":"Rui Yan, Xiaojun Wan, Mirella Lapata, Wayne Xin Zhao, Pu-Jen Cheng, Xiaoming Li","year":"2012","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao et al., 2013] Xin Wayne Zhao, Yanwei Guo, Rui Yan, Yulan He, and Xiaoming Li. Timeline generation with so- cial attention. In SIGIR, pages 1061-1064. ACM, 2013.","title":"Timeline generation with social attention","authors":"Xin Wayne Zhao, Yanwei Guo, Rui Yan, Yulan He, Xiaoming Li","year":"2013","venue":"SIGIR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}