{"type":"ScholarlyArticle","title":"Generating a Structured Summary of Numerous Academic Papers: Dataset and Method","authors":[{"name":"Shuaiqi LIU","slug":"shuaiqi-liu","given":"Shuaiqi","family":"LIU","affiliation":"Department of Computing , The Hong Kong Polytechnic University"},{"name":"Jiannong Cao","slug":"jiannong-cao","given":"Jiannong","family":"Cao","affiliation":"Department of Computing , The Hong Kong Polytechnic University"},{"name":"Ruosong Yang","slug":"ruosong-yang","given":"Ruosong","family":"Yang","affiliation":"Department of Computing , The Hong Kong Polytechnic University"},{"name":"Zhiyuan Wen","slug":"zhiyuan-wen","given":"Zhiyuan","family":"Wen","affiliation":"Department of Computing , The Hong Kong Polytechnic University"}],"abstract":"Writing a survey paper on one research topic usually needs to cover the salient content from numerous related papers, which can be modeled as a multi-document summarization (MDS) task. Existing MDS datasets usually focus on producing the structureless summary covering a few input documents. Meanwhile, previous structured summary generation works focus on summarizing a single document into a multi-section summary. These existing datasets and methods cannot meet the requirements of summarizing numerous academic papers into a structured summary. To deal with the scarcity of available data, we propose BigSurvey, the first large-scale dataset for generating comprehensive summaries of numerous academic papers on each topic. We collect target summaries from more than seven thousand survey papers and utilize their 430 thousand reference papers’ abstracts as input documents. To organize the diverse content from dozens of input documents and ensure the efficiency of processing long text sequences, we propose a summarization method named category-based alignment and sparse transformer (CAST). The experimental results show that our CAST method outperforms various advanced summarization methods.","doi":"10.24963/ijcai.2022/591","url":"https://ijcai.org/proceedings/2022/591","pdf":"https://ijcai.org/proceedings/2022/0591.pdf","markdown":"https://ijcai.org/proceedings/2022/591.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/591","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":591,"pages":{"first":4259,"last":4265},"keywords":["Natural Language Processing: Summarization","Natural Language Processing: Language Generation","Natural Language Processing: Resources and Evaluation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986593","length_sec":80,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984736","length_sec":344,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ammar et al., 2018] Waleed Ammar, Dirk Groeneveld, et al. Construction of the literature graph in semantic scholar. In NAACL-HLT, 2018.","title":"Construction of the literature graph in semantic scholar","authors":"Waleed Ammar, Dirk Groeneveld","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aviv-Reuven and Rosenfeld, 2021] Shir Aviv-Reuven and Ariel Rosenfeld. Publication patterns' changes due to the covid-19 pandemic: A longitudinal and short-term sciento- metric analysis. Scientometrics, pages 1-24, 2021.","title":"Publication patterns' changes due to the covid-19 pandemic: A longitudinal and short-term scientometric analysis","authors":"Aviv-Reuven, Shir Rosenfeld, Ariel Aviv-Reuven, Rosenfeld","year":"2021","venue":"Scientometrics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beltagy et al., 2020] Iz Beltagy, Matthew E Peters, and Ar- man Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020.","title":"Iz Beltagy, Matthew E Peters, and Arman Cohan","authors":"Beltagy","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bird et al., 2009] Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python. O'Reilly Media, Inc., 2009.","title":"Natural language processing with Python","authors":"Steven Bird, Ewan Klein, Edward Loper","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choromanski et al., 2020] Krzysztof Choromanski, Valerii Likhosherstov, et al. Rethinking attention with performers. arXiv preprint arXiv:2009.14794, 2020.","title":"Rethinking attention with performers","authors":"Krzysztof Choromanski, Valerii Likhosherstov","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cohan et al., 2019] Arman Cohan, Iz Beltagy, King, et al. Pretrained language models for sequential sentence classifi- cation. In EMNLP-IJCNLP, pages 3693-3699, 2019.","title":"Pretrained language models for sequential sentence classification","authors":"Arman Cohan, Iz Beltagy, King","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Devlin and others, 2019] Jacob Devlin et al. Bert: Pre- training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171-4186, 2019.","title":"Pretraining of deep bidirectional transformers for language understanding","authors":"Jacob Devlin","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Erkan and Radev, 2004] Günes Erkan and Dragomir R Radev. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence research, pages 457-479, 2004.","title":"Lexrank: Graph-based lexical centrality as salience in text summarization","authors":"Radev ; ] Günes Erkan, Erkan, Dragomir R Radev","year":"2004","venue":"Journal of artificial intelligence research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fabbri et al., 2019] Alexander Richard Fabbri, Irene Li, Tianwei She, et al. Multi-news: A large-scale multi- document summarization dataset and abstractive hierar- chical model. In ACL, pages 1074-1084, 2019.","title":"Multi-news: A large-scale multidocument summarization dataset and abstractive hierarchical model","authors":"Alexander Richard Fabbri, Irene Li, Tianwei She","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fleiss, 1971] Joseph L Fleiss. Measuring nominal scale agreement among many raters. Psychological bulletin, page 378, 1971.","title":"Measuring nominal scale agreement among many raters","authors":"L Joseph, Fleiss","year":"1971","venue":"Psychological bulletin","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gehrmann et al., 2018] Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. Bottom-up abstractive summarization. In EMNLP, pages 4098-4109, 2018.","title":"Bottom-up abstractive summarization","authors":"Sebastian Gehrmann, Yuntian Deng, Alexander Rush","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gidiotis and Tsoumakas, 2019] Alexios Gidiotis and Grigo- rios Tsoumakas. Structured summarization of academic publications. In PKDD/ECML, pages 636-645, 2019.","title":"Alexios Gidiotis and Grigorios Tsoumakas. Structured summarization of academic publications","authors":"Tsoumakas Gidiotis","year":"2019","venue":"PKDD/ECML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Grusky et al., 2018] Max Grusky, Mor Naaman, and Yoav Artzi. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In NAACL-HLT, pages 708-719, 2018.","title":"Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies","authors":"Grusky","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hartley, 2004] James Hartley. Current findings from re- search on structured abstracts. Journal of the Medical Library Association, page 368, 2004.","title":"Current findings from research on structured abstracts","authors":"James Hartley","year":"2004","venue":"Journal of the Medical Library Association","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hartley, 2014] James Hartley. Current findings from re- search on structured abstracts: An update. Journal of the Medical Library Association, pages 146-148, 2014.","title":"Current findings from research on structured abstracts: An update","authors":"James Hartley, Hartley","year":"2014","venue":"Journal of the Medical Library Association","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lewis et al., 2020] Mike Lewis, Yinhan Liu, Naman Goyal, et al. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and compre- hension. In ACL, pages 7871-7880, 2020.","title":"Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.","title":"Rouge: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text summarization branches out","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2018] Peter J Liu, Mohammad Saleh, Etienne Pot, et al. Generating wikipedia by summarizing long sequences. In ICLR, 2018.","title":"Generating wikipedia by summarizing long sequences","authors":"J Peter, Mohammad Liu, Etienne Saleh, Pot","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, et al. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2021] Shuaiqi Liu, Jiannong Cao, et al. Highlight- transformer: Leveraging key phrase aware attention to im- prove abstractive multi-document summarization. In Find- ings of the ACL-IJCNLP, pages 5021-5027, 2021.","title":"Highlighttransformer: Leveraging key phrase aware attention to improve abstractive multi-document summarization","authors":"Liu","year":"2021","venue":"Findings of the ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2022] Shuaiqi Liu, Jiannong Cao, et al. Key phrase aware transformer for abstractive summarization. Information Processing Management, 59(3):102913, 2022.","title":"Key phrase aware transformer for abstractive summarization","authors":"Liu","year":"2022","venue":"Information Processing Management","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lu et al., 2020] Yao Lu, Yue Dong, and Laurent Charlin. Multi-XScience: A large-scale dataset for extreme multi- document summarization of scientific articles. In EMNLP, pages 8068-8074, 2020.","title":"Multi-XScience: A large-scale dataset for extreme multidocument summarization of scientific articles","authors":"Yao Lu, Yue Dong, Laurent Charlin","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Meng et al., 2021] Rui Meng, Khushboo Thaker, Lei Zhang, et al. Bringing structure into summaries: a faceted sum- marization dataset for long scientific documents. In ACL- IJCNLP, pages 1080-1089, 2021.","title":"Bringing structure into summaries: a faceted summarization dataset for long scientific documents","authors":"Rui Meng, Khushboo Thaker, Lei Zhang","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Mihalcea and Tarau, 2004] Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In EMNLP, pages 404- 411, 2004.","title":"Textrank: Bringing order into text","authors":"Tarau Mihalcea, Rada Mihalcea, Paul Tarau","year":"2004","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ott and others, 2019] Myle Ott et al. fairseq: A fast, extensi- ble toolkit for sequence modeling. In NAACL-HLT, 2019.","title":"fairseq: A fast, extensible toolkit for sequence modeling","authors":"Myle Ott","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[See et al., 2017] Abigail See, Peter J. Liu, and Christopher D. Manning. Get to the point: Summarization with pointer- generator networks. In ACL, pages 1073-1083, 2017.","title":"Get to the point: Summarization with pointergenerator networks","authors":"Abigail See, Peter J Liu, Christopher D Manning","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sinha et al., 2015] Arnab Sinha, Zhihong Shen, Yang Song, et al. An overview of microsoft academic service (mas) and applications. In WWW, pages 243-246, 2015.","title":"An overview of microsoft academic service (mas) and applications","authors":"Arnab Sinha, Zhihong Shen, Yang Song","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vaswani and others, 2017] Ashish Vaswani et al. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wolf et al., 2020] Thomas Wolf, Julien Chaumond, Lysan- dre Debut, et al. Transformers: State-of-the-art natural language processing. In EMNLP, pages 38-45, 2020.","title":"Transformers: State-of-the-art natural language processing","authors":"Thomas Wolf, Julien Chaumond, Lysandre Debut","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zaheer et al., 2020] Manzil Zaheer, Guru Guruganesh, Ku- mar Avinava Dubey, et al. Big bird: Transformers for longer sequences. In NeurIPS, 2020.","title":"Big bird: Transformers for longer sequences","authors":"Manzil Zaheer, Guru Guruganesh, Avinava Kumar, Dubey","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang and others, 2020] Jingqing Zhang et al. Pegasus: Pre- training with extracted gap-sentences for abstractive sum- marization. In ICML, pages 11328-11339, 2020.","title":"Pegasus: Pretraining with extracted gap-sentences for abstractive summarization","authors":"Others ; Jingqing Zhang, Zhang","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}