{"type":"ScholarlyArticle","title":"From Neural Sentence Summarization to Headline Generation: A Coarse-to-Fine Approach","authors":[{"name":"Jiwei Tan","slug":"jiwei-tan","given":"Jiwei","family":"Tan","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"},{"name":"Xiaojun Wan","slug":"xiaojun-wan","given":"Xiaojun","family":"Wan","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"},{"name":"Jianguo Xiao","slug":"jianguo-xiao","given":"Jianguo","family":"Xiao","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"}],"abstract":"Headline generation is a task of abstractive text summarization, and previously suffers from the immaturity of natural language generation techniques. Recent success of neural sentence summarization models shows the capacity of generating informative, fluent headlines conditioned on selected recapitulative sentences. In this paper, we investigate the extension of sentence summarization models to the document headline generation task. The challenge is that extending the sentence summarization model to consider more document information will mostly confuse the model and hurt the performance. In this paper, we propose a coarse-to-fine approach, which first identifies the important sentences of a document using document summarization techniques, and then exploits a multi-sentence summarization model with hierarchical attention to leverage the important sentences for headline generation. Experimental results on a large real dataset demonstrate the proposed approach significantly improves the performance of neural sentence summarization models on the headline generation task.","doi":"10.24963/ijcai.2017/574","url":"https://ijcai.org/proceedings/2017/574","pdf":"https://ijcai.org/proceedings/2017/0574.pdf","markdown":"https://ijcai.org/proceedings/2017/574.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/574","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":574,"pages":{"first":4109,"last":4115},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Summarization","Natural Language Processing: Natural Language Generation"],"videos":[],"references":[{"position":1,"raw":"[Alfonseca et al., 2013] Enrique Alfonseca, Daniele Pighin, and Guillermo Garrido. HEADY: news headline abstrac- tion through event pattern clustering. In ACL 2013, pages 1243-1253, 2013.","title":"HEADY: news headline abstraction through event pattern clustering","authors":"Enrique Alfonseca, Daniele Pighin, Guillermo Garrido","year":"2013","venue":"ACL 2013","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Banko et al., 2000] Michele Banko, Vibhu O. Mittal, and Michael J. Witbrock. Headline generation based on sta- tistical translation. In ACL, pages 318-325, 2000.","title":"Headline generation based on statistical translation","authors":"Michele Banko, O Vibhu, Michael J Mittal, Witbrock","year":"2000","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2016] Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, and Hui Jiang. Distraction-based neural net- works for document summarization. In IJCAI-16, pages 2754-2760, 2016.","title":"Distraction-based neural networks for document summarization","authors":"Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang","year":"2016","venue":"IJCAI-16","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen, 2010] Chih-Ming Chen. Intelligent location-based mobile news service system with automatic news summa- rization. Expert Syst. Appl., 37(9):6651-6662, 2010.","title":"Intelligent location-based mobile news service system with automatic news summarization","authors":"Chih-Ming Chen","year":"2010","venue":"Expert Syst. Appl","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chopra et al., 2016] Sumit Chopra, Michael Auli, and Alexander M Rush. Abstractive sentence summarization with attentive recurrent neural networks. In NAACL, 2016.","title":"Abstractive sentence summarization with attentive recurrent neural networks","authors":"Sumit Chopra, Michael Auli, Alexander M Rush","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Colmenares et al., 2015] Carlos A. Colmenares, Marina L- itvak, Amin Mantrach, and Fabrizio Silvestri. HEADS: headline generation as sequence prediction using an ab- stract feature-rich space. In NAACL, 2015.","title":"HEADS: headline generation as sequence prediction using an abstract feature-rich space","authors":"Carlos A Colmenares, Marina Litvak, Amin Mantrach, Fabrizio Silvestri","year":"2015","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dorr et al., 2003] Bonnie Dorr, David Zajic, and Richard Schwartz. Hedge trimmer: A parse-and-trim approach to headline generation. In HLT-NAACL 03 on Text summa- rization workshop, pages 1-8, 2003.","title":"Hedge trimmer: A parse-and-trim approach to headline generation","authors":"Dorr","year":"2003","venue":"HLT-NAACL 03 on Text summarization workshop","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Erbs et al., 2013] Nicolai Erbs, Iryna Gurevych, and Torsten Zesch. Hierarchy identification for automatically generating table-of-contents. In RANLP 2013, pages 252- 260, 2013.","title":"Hierarchy identification for automatically generating table-of-contents","authors":"Nicolai Erbs, Iryna Gurevych, Torsten Zesch","year":"2013","venue":"RANLP 2013","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Erkan and Radev, 2004] Günes Erkan and Dragomir R. Radev. Lexrank: Graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. (JAIR), 22:457- 479, 2004.","title":"Lexrank: Graph-based lexical centrality as salience in text summarization","authors":"Günes Erkan, Dragomir R Radev","year":"2004","venue":"J. Artif. Intell. Res. (JAIR)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Graves, 2013] Alex Graves. Generating sequences with re- current neural networks. arXiv preprint arXiv:1308.0850, 2013.","title":"Generating sequences with recurrent neural networks","authors":"Alex Graves","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Haghighi and Vanderwende, 2009] Aria Haghighi and Lucy Vanderwende. Exploring content models for multi- document summarization. In NAACL, 2009.","title":"Aria Haghighi and Lucy Vanderwende. Exploring content models for multidocument summarization","authors":"Vanderwende Haghighi","year":"2009","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2015] Jiwei Li, Minh-Thang Luong, and Dan Ju- rafsky. A hierarchical neural autoencoder for paragraphs and documents. In ACL 2015, pages 1106-1115, 2015.","title":"A hierarchical neural autoencoder for paragraphs and documents","authors":"Jiwei Li, Minh-Thang Luong, Dan Jurafsky","year":"2015","venue":"ACL 2015","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out ACL-04 workshop, volume 8. Barcelona, Spain, 2004.","title":"Rouge: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text summarization branches out ACL-04 workshop","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lopyrev, 2015] Konstantin Lopyrev. Generating news head- lines with recurrent neural networks. arXiv preprint arX- iv:1512.01712, 2015.","title":"Generating news headlines with recurrent neural networks","authors":"Konstantin Lopyrev","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Luhn, 1958] Hans Peter Luhn. The automatic creation of literature abstracts. IBM Journal of Research and Devel- opment, 2(2):159-165, 1958.","title":"The automatic creation of literature abstracts","authors":"Hans Luhn, Peter Luhn","year":"1958","venue":"IBM Journal of Research and Development","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mihalcea and Tarau, 2004] Rada Mihalcea and Paul Tarau. Textrank: Bringing order into texts. In EMNLP, 2004.","title":"Textrank: Bringing order into texts","authors":"Tarau Mihalcea, Rada Mihalcea, Paul Tarau","year":"2004","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nallapati et al., 2016] Ramesh Nallapati, Bowen Zhou, Cícero Nogueira dos Santos, C ¸aglar Gülc ¸ehre, and Bing Xiang. Abstractive text summarization using sequence-to- sequence rnns and beyond. In CoNLL 2016, pages 280- 290, 2016.","title":"Abstractive text summarization using sequence-tosequence rnns and beyond","authors":"Ramesh Nallapati, Bowen Zhou, Cícero Nogueira Dos Santos, C ¸aglar Gülc ¸ehre, Bing Xiang","year":"2016","venue":"CoNLL 2016","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nenkova and Vanderwende, 2005] Ani Nenkova and Lucy Vanderwende. The impact of frequency on summariza- tion. Microsoft Research, Redmond, Washington, Tech. Rep. MSR-TR-2005-101, 2005.","title":"Ani Nenkova and Lucy Vanderwende. The impact of frequency on summarization","authors":"Vanderwende Nenkova","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Page et al., 1999] Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web, 1999.","title":"The pagerank citation ranking: Bringing order to the web","authors":"Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rumelhart et al., 1988] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representation- s by back-propagating errors. Cognitive modeling, 5:3, 1988.","title":"Learning representations by back-propagating errors","authors":"Geoffrey E David E Rumelhart, Ronald J Hinton, Williams","year":"1988","venue":"Cognitive modeling","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rush et al., 2015] Alexander M. Rush, Sumit Chopra, and Jason Weston. A neural attention model for abstractive sentence summarization. In EMNLP 2015, pages 379-389, 2015.","title":"A neural attention model for abstractive sentence summarization","authors":"Alexander M Rush, Sumit Chopra, Jason Weston","year":"2015","venue":"EMNLP 2015","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shen et al., 2016] Shiqi Shen, Zhiyuan Liu, Maosong Sun, et al. Neural headline generation with minimum risk train- ing. arXiv preprint arXiv:1604.01904, 2016.","title":"Neural headline generation with minimum risk training","authors":"Shiqi Shen, Zhiyuan Liu, Maosong Sun","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Soricut and Marcu, 2007] Radu Soricut and Daniel Marcu. Abstractive headline generation using widl-expressions. Inf. Process. Manage., 43(6):1536-1548, 2007.","title":"Abstractive headline generation using widl-expressions","authors":"Radu Soricut, Daniel Marcu","year":"2007","venue":"Inf. Process. Manage","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Steinberger and Jezek, 2004] Josef Steinberger and Karel Jezek. Text summarization and singular value decompo- sition. In ADVIS 2004, pages 245-254, 2004.","title":"Text summarization and singular value decomposition","authors":"Jezek Steinberger, Josef Steinberger, Karel Jezek","year":"2004","venue":"ADVIS 2004","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sun et al., 2015] Rui Sun, Yue Zhang, Meishan Zhang, and Dong-Hong Ji. Event-driven headline generation. In ACL 2015, pages 462-472, 2015.","title":"Event-driven headline generation","authors":"Rui Sun, Yue Zhang, Meishan Zhang, Dong-Hong Ji","year":"2015","venue":"ACL 2015","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Takase et al., 2016] Sho Takase, Jun Suzuki, Naoak- i Okazaki, Tsutomu Hirao, and Masaaki Nagata. Neural headline generation on abstract meaning representation. In EMNLP, pages 1054-1059, 2016.","title":"Neural headline generation on abstract meaning representation","authors":"Sho Takase, Jun Suzuki, Naoaki Okazaki, Tsutomu Hirao, Masaaki Nagata","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Woodsend et al., 2010] Kristian Woodsend, Yansong Feng, and Mirella Lapata. Title generation with quasi- synchronous grammar. In EMNLP, pages 513-523, 2010.","title":"Title generation with quasisynchronous grammar","authors":"Kristian Woodsend, Yansong Feng, Mirella Lapata","year":"2010","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2010] Songhua Xu, Shaohui Yang, and Francis Chi-Moon Lau. Keyword extraction and headline gener- ation using novel word features. In AAAI, 2010.","title":"Keyword extraction and headline generation using novel word features","authors":"Songhua Xu, Shaohui Yang, Francis Chi-Moon Lau","year":"2010","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yu et al., 2016] Lang-Chi Yu, Hung-yi Lee, and Lin-Shan Lee. Abstractive headline generation for spoken content by attentive recurrent neural networks with ASR error modeling. In SLT, pages 151-157, 2016.","title":"Abstractive headline generation for spoken content by attentive recurrent neural networks with ASR error modeling","authors":"Lang-Chi Yu, Hung-Yi Lee, Lin-Shan Lee","year":"2016","venue":"SLT","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}