{"type":"ScholarlyArticle","title":"Stage-wise Stylistic Headline Generation: Style Generation and Summarized Content Insertion","authors":[{"name":"Jiaao Zhan","slug":"jiaao-zhan","given":"Jiaao","family":"Zhan","affiliation":"School of Computer Science and Technology , Beijing Institute of Technology , Beijing , China"},{"name":"Yang Gao","slug":"yang-gao","given":"Yang","family":"Gao","affiliation":"School of Computer Science and Technology , Beijing Institute of Technology , Beijing , China; Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications , Beijing , China"},{"name":"Yu Bai","slug":"yu-bai","given":"Yu","family":"Bai","affiliation":"School of Computer Science and Technology , Beijing Institute of Technology , Beijing , China"},{"name":"Qianhui Liu","slug":"qianhui-liu","given":"Qianhui","family":"Liu","affiliation":"School of Computer Science and Technology , Beijing Institute of Technology , Beijing , China"}],"abstract":"A quality headline with a high click-rate should not only summarize the content of an article, but also reflect a style that attracts users. Such demand has drawn rising attention to the task of stylistic headline generation (SHG). An intuitive method is to first generate plain headlines leveraged by document-headline parallel data then transfer them to a target style. However, this inevitably suffers from error propagation. Therefore, to unify the two sub-tasks and explicitly decompose style-relevant attributes and summarize content, we propose an end-to-end stage-wise SHG model containing the style generation component and the content insertion component, where the former generates stylistic-relevant intermediate outputs and the latter receives these outputs then inserts the summarized content. The intermediate outputs are observable, making the style generation easy to control. Our system is comprehensively evaluated by both quantitative and qualitative metrics, and it achieves state-of-the-art results in SHG over three different stylistic datasets.","doi":"10.24963/ijcai.2022/623","url":"https://ijcai.org/proceedings/2022/623","pdf":"https://ijcai.org/proceedings/2022/0623.pdf","markdown":"https://ijcai.org/proceedings/2022/623.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/623","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":623,"pages":{"first":4489,"last":4495},"keywords":["Natural Language Processing: Summarization","Natural Language Processing: Language Generation","Natural Language Processing: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986994","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985137","length_sec":658,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chopra et al., 2016] Sumit Chopra, Michael Auli, and Alexander M. Rush. Abstractive sentence summarization with attentive recurrent neural networks. In Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, pages 93-98, 2016.","title":"Abstractive sentence summarization with attentive recurrent neural networks","authors":"Sumit Chopra, Michael Auli, Alexander M Rush","year":"2016","venue":"Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dai et al., 2019] Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. Style transformer: Unpaired text style transfer without disentangled latent representation. In ACL, 2019.","title":"Style transformer: Unpaired text style transfer without disentangled latent representation","authors":"Ning Dai, Jianze Liang, Xipeng Qiu, Xuanjing Huang","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hu et al., 2017] Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. Toward controlled generation of text. In ICML, 2017.","title":"Toward controlled generation of text","authors":"Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, Eric P Xing","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Jin et al., 2020] Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Peter Szolovits. Hooks in the headline: Learning to generate headlines with controlled styles. In ACL, 2020.","title":"Hooks in the headline: Learning to generate headlines with controlled styles","authors":"Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, Peter Szolovits","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[John et al., 2019] Vineet John, Lili Mou, Hareesh Bahu- leyan, and Olga Vechtomova. Disentangled representation learning for non-parallel text style transfer. In ACL, 2019.","title":"Disentangled representation learning for non-parallel text style transfer","authors":"Vineet John, Lili Mou, Hareesh Bahuleyan, Olga Vechtomova","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Joulin et al., 2017] Armand Joulin, Edouard Grave, and Pi- otr Bojanowski Tomas Mikolov. Bag of tricks for efficient text classification. EACL 2017, page 427, 2017.","title":"Bag of tricks for efficient text classification","authors":"Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomas Mikolov","year":"2017","venue":"EACL 2017","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kiritchenko and Mohammad, 2017] Svetlana Kiritchenko and Saif M. Mohammad. Best-worst scaling more reli- able than rating scales: A case study on sentiment intensity annotation. ArXiv, abs/1712.01765, 2017.","title":"Best-worst scaling more reliable than rating scales: A case study on sentiment intensity annotation","authors":"Svetlana Kiritchenko, Saif M Mohammad","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Krishna et al., 2020] Kalpesh Krishna, John Wieting, and Mohit Iyyer. Reformulating unsupervised style transfer as paraphrase generation. ArXiv, abs/2010.05700, 2020.","title":"Reformulating unsupervised style transfer as paraphrase generation","authors":"Kalpesh Krishna, John Wieting, Mohit Iyyer","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2016] Jiwei Li, Michel Galley, Chris Brockett, Jian- feng Gao, and William B. Dolan. A diversity-promoting ob- jective function for neural conversation models. In NAACL, 2016.","title":"A diversity-promoting objective function for neural conversation models","authors":"Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, William B Dolan","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2018] Juncen Li, Robin Jia, He He, and Percy Liang. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In NAACL, 2018.","title":"Delete, retrieve, generate: a simple approach to sentiment and style transfer","authors":"Li","year":"2018","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2020a] Xiao Li, Guanyi Chen, Chenghua Lin, and Ruizhe Li. Dgst: a dual-generator network for text style transfer. arXiv e-prints, pages arXiv-2010, 2020.","title":"Dgst: a dual-generator network for text style transfer","authors":"Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2020b] Xiao Li, Guanyi Chen, Chenghua Lin, and Ruizhe Li. A dual-generator network for text style transfer applications. ArXiv, abs/2010.14557, 2020.","title":"A dual-generator network for text style transfer applications","authors":"Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.","title":"Rouge: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text summarization branches out","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu and Lapata, 2019] Yang Liu and Mirella Lapata. Text summarization with pretrained encoders. ArXiv, abs/1908.08345, 2019.","title":"Text summarization with pretrained encoders","authors":"Yang Liu, Mirella Lapata","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mai et al., 2020] Florian Mai, Nikolaos Pappas, Ivan Mon- tero, Noah A. Smith, James Henderson Idiap Research Institute, Epfl, University of Washington, and Allen Insti- tute for Artificial Intelligence. Plug and play autoencoders for conditional text generation. In EMNLP, 2020.","title":"EMNLP","authors":"Mai","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: a method for auto- matic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguistics, pages 311-318, 2002.","title":"BLEU: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Radford et al., 2018] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. Technical report, OpenAI, 2018.","title":"Improving language understanding by generative pre-training","authors":"Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shang et al., 2019] Mingyue Shang, Piji Li, Zhenxin Fu, Li- dong Bing, Dongyan Zhao, Shuming Shi, and Rui Yan. Semi-supervised text style transfer: Cross projection in latent space. In EMNLP, 2019.","title":"Semi-supervised text style transfer: Cross projection in latent space","authors":"Mingyue Shang, Piji Li, Zhenxin Fu, Lidong Bing, Dongyan Zhao, Shuming Shi, Rui Yan","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shen et al., 2017] Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi S Jaakkola. Style transfer from non-parallel text by cross-alignment. In NIPS, 2017.","title":"Style transfer from non-parallel text by cross-alignment","authors":"Shen","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shen et al., 2020] Tianxiao Shen, Jonas Mueller, Regina Barzilay, and Tommi Jaakkola. Educating text autoen- coders: Latent representation guidance via denoising. In In- ternational Conference on Machine Learning, pages 8719- 8729. PMLR, 2020.","title":"Educating text autoencoders: Latent representation guidance via denoising","authors":"Shen","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2019] Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. Mass: Masked sequence to sequence pre-training for language generation. In International Con- ference on Machine Learning, pages 5926-5936. PMLR, 2019.","title":"Mass: Masked sequence to sequence pre-training for language generation","authors":"Song","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sudhakar et al., 2019] Akhilesh Sudhakar, Bhargav Upad- hyay, and Arjun Maheswaran. \"transforming\" delete, re- trieve, generate approach for controlled text style transfer. In EMNLP, 2019.","title":"transforming\" delete, retrieve, generate approach for controlled text style transfer","authors":"Akhilesh Sudhakar, Bhargav Upadhyay, Arjun Maheswaran","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu et al., 2018] Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, and Wenjie Li. Unpaired sentiment-to-sentiment translation: A cycled re- inforcement learning approach. In ACL (1), 2018.","title":"Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach","authors":"Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, Wenjie Li","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2018] Junbo Jake Zhao, Yoon Kim, Kelly Zhang, Alexander M Rush, and Yann LeCun. Adversarially regu- larized autoencoders. In ICML, 2018.","title":"Adversarially regularized autoencoders","authors":"Jake Junbo, Yoon Zhao, Kelly Kim, Alexander M Zhang, Yann Rush, Lecun","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2015] Yukun Zhu, Ryan Kiros, Richard S Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and read- ing books. In ICCV, 2015.","title":"Aligning books and movies: Towards story-like visual explanations by watching movies and reading books","authors":"Yukun Zhu, Ryan Kiros, Richard S Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, Sanja Fidler","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu et al., 2018] Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. arXiv preprint arXiv:1802.01886, 2018.","title":"Texygen: A benchmarking platform for text generation models","authors":"Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, Yong Yu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}