{"type":"ScholarlyArticle","title":"Beyond Pure Text: Summarizing Financial Reports Based on Both Textual and Tabular Data","authors":[{"name":"Ziao Wang","slug":"ziao-wang","given":"Ziao","family":"Wang","affiliation":"School of Computer Science and Technology , Harbin Institute of Technology , Shenzhen , China"},{"name":"Zelin Jiang","slug":"zelin-jiang","given":"Zelin","family":"Jiang","affiliation":""},{"name":"Xiaofeng Zhang","slug":"xiaofeng-zhang","given":"Xiaofeng","family":"Zhang","affiliation":"School of Computer Science and Technology , Harbin Institute of Technology , Shenzhen , China"},{"name":"Jaehyeon Soon","slug":"jaehyeon-soon","given":"Jaehyeon","family":"Soon","affiliation":"School of Computer Science and Technology , Harbin Institute of Technology , Shenzhen , China"},{"name":"Jialu Zhang","slug":"jialu-zhang","given":"Jialu","family":"Zhang","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Wang Xiaoyao","slug":"wang-xiaoyao","given":"Wang","family":"Xiaoyao","affiliation":"School of Computer Science and Technology , Harbin Institute of Technology , Shenzhen , China"},{"name":"Hongwei Du","slug":"hongwei-du","given":"Hongwei","family":"Du","affiliation":"School of Computer Science and Technology , Harbin Institute of Technology , Shenzhen , China"}],"abstract":"Abstractive text summarization is to generate concise summaries that well preserve both salient information and the overall semantic meanings of the given documents. However, real-world documents, e.g., financial reports, generally contain rich data such as charts and tabular data which invalidates most existing text summarization approaches. This paper is thus motivated to propose this novel approach to simultaneously summarize both textual and tabular data. Particularly, we first manually construct a “table+text → summary” dataset. Then, the tabular data is respectively embedded in a row-wise and column-wise manner, and the textual data is encoded at the sentence-level via an employed pre-trained model. We propose a salient detector gate respectively performed between each pair of row/column and sentence embeddings. The highly correlated content is considered as salient information that must be summarized. Extensive experiments have been performed on our constructed dataset and the promising results demonstrate the effectiveness of the proposed approach w.r.t. a number of both automatic and human evaluation criteria.","doi":"10.24963/ijcai.2023/581","url":"https://ijcai.org/proceedings/2023/581","pdf":"https://ijcai.org/proceedings/2023/0581.pdf","markdown":"https://ijcai.org/proceedings/2023/581.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/581","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Natural Language Processing","track":"Main Track","number":581,"pages":{"first":5233,"last":5241},"keywords":["Natural Language Processing: NLP: Applications","Natural Language Processing: NLP: Language generation","Natural Language Processing: NLP: Summarization"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyung, Hyun Cho, Yoshua Bengio","year":"2015","venue":"3rd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen and Zhuge, 2018] Jingqiang Chen and Hai Zhuge. Abstractive text-image summarization using multi-modal attentional hierarchical RNN. In Proc. of EMNLP, pages 4046-4056, Brussels, Belgium, October-November 2018.","title":"Abstractive text-image summarization using multi-modal attentional hierarchical RNN","authors":"Chen, Zhuge ;, Jingqiang Chen, Hai Zhuge","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"ACL.","title":null,"authors":"Acl","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2022] Miao Chen, Xinjiang Lu, Tong Xu, Yanyan Li, Zhou Jingbo, Dejing Dou, and Hui Xiong. Towards table-to-text generation with pretrained language model: A table structure understanding and text deliber- ating approach. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022.","title":"Towards table-to-text generation with pretrained language model: A table structure understanding and text deliberating approach","authors":"Xinjiang Miao Chen, Tong Lu, Yanyan Xu, Zhou Li, Dejing Jingbo, Hui Dou, Xiong","year":"2022","venue":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chu and Liu, 2019] Eric Chu and Peter Liu. MeanSum: A neural model for unsupervised multi-document abstrac- tive summarization. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th Interna- tional Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1223- 1232. PMLR, 09-15 Jun 2019.","title":"MeanSum: A neural model for unsupervised multi-document abstractive summarization","authors":"Chu, Liu ; ] Eric Chu, Peter Liu","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dong et al., 2021] Luobing Dong, Meghana N Satpute, Weili Wu, and Ding-Zhu Du. Two-phase multidocument summarization through content-attention-based subtopic detection. IEEE Transactions on Computational Social Systems, 8(6):1379-1392, 2021.","title":"Two-phase multidocument summarization through content-attention-based subtopic detection","authors":"Dong","year":"2021","venue":"IEEE Transactions on Computational Social Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Erkan and Radev, 2004] Günes Erkan and Dragomir R Radev. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence re- search, 22:457-479, 2004.","title":"Lexrank: Graph-based lexical centrality as salience in text summarization","authors":"Radev ; ] Günes Erkan, Erkan, Dragomir R Radev","year":"2004","venue":"Journal of artificial intelligence research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fu et al., 2020] Xiyan Fu, Jun Wang, Jinghan Zhang, Jin- mao Wei, and Zhenglu Yang. Document summarization with vhtm: Variational hierarchical topic-aware mecha- nism. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7740-7747, Apr. 2020.","title":"Document summarization with vhtm: Variational hierarchical topic-aware mechanism","authors":"Fu","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Galanis et al., 2012] Dimitrios Galanis, Gerasimos Lam- pouras, and Ion Androutsopoulos. Extractive multi- document summarization with integer linear programming and support vector regression. In Proceedings of COLING 2012, pages 911-926, 2012.","title":"Extractive multidocument summarization with integer linear programming and support vector regression","authors":"Dimitrios Galanis, Gerasimos Lampouras, Ion Androutsopoulos","year":"2012","venue":"Proceedings of COLING 2012","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gong et al., 2019] Heng Gong, Xiaocheng Feng, Bing Qin, and Ting Liu. Table-to-text generation with effective hi- erarchical encoder on three dimensions (row, column and time). In Proceedings of the 2019 Conference on Em- pirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3143-3152, Hong Kong, China, November 2019. ACL.","title":"Table-to-text generation with effective hierarchical encoder on three dimensions (row, column and time)","authors":"Heng Gong, Xiaocheng Feng, Bing Qin, Ting Liu","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gui et al., 2018] Min Gui, Zhengkun Zhang, Zhenglu Yang, Yanhui Gu, and Guandong Xu. An effective joint frame- work for document summarization. In Companion Pro- ceedings of the The Web Conference 2018, pages 121-122, 2018.","title":"An effective joint framework for document summarization","authors":"Gui","year":"2018","venue":"Companion Proceedings of the The Web Conference 2018","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gui et al., 2019] Min Gui, Junfeng Tian, Rui Wang, and Zhenglu Yang. Attention optimization for abstractive doc- ument summarization. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Process- ing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLP), pages 1222- 1228, Hong Kong, China, November 2019. ACL.","title":"Attention optimization for abstractive document summarization","authors":"Gui","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2021] Yi-Chong Huang, Xia-Chong Feng, Xiao-Cheng Feng, and Bing Qin. The factual inconsis- tency problem in abstractive text summarization: A sur- vey. CoRR, 2021.","title":"The factual inconsistency problem in abstractive text summarization: A survey","authors":"Yi-Chong Huang, Xia-Chong Feng, Xiao-Cheng Feng, Bing Qin","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jangra et al., 2020a] Anubhav Jangra, Raghav Jain, Vaib- hav Mavi, Sriparna Saha, and Pushpak Bhattacharyya. Semantic extractor-paraphraser based abstractive summa- rization. In Proceedings of the 17th International Con- ference on Natural Language Processing (ICON), pages 191-199, Indian Institute of Technology Patna, Patna, In- dia, December 2020. NLP Association of India (NLPAI).","title":"Semantic extractor-paraphraser based abstractive summarization","authors":"Jangra","year":"2020","venue":"Proceedings of the 17th International Conference on Natural Language Processing (ICON)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jangra et al., 2020b] Anubhav Jangra, Adam Jatowt, Mo- hammad Hasanuzzaman, and Sriparna Saha. Text-image- video summary generation using joint integer linear pro- gramming. In Joemon M. Jose, Emine Yilmaz, João Magalhães, Pablo Castells, Nicola Ferro, Mário J. Silva, and Flávio Martins, editors, Advances in Information Re- trieval, pages 190-198, Cham, 2020. Springer Interna- tional Publishing.","title":"Text-imagevideo summary generation using joint integer linear programming","authors":"Jangra","year":"2020","venue":"Advances in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Klein et al., 2014] Benjamin Klein, Guy Lev, Gil Sadeh, and Lior Wolf. Fisher vectors derived from hybrid gaussian-laplacian mixture models for image annotation. arXiv preprint arXiv:1411.7399, 2014.","title":"Fisher vectors derived from hybrid gaussian-laplacian mixture models for image annotation","authors":"Benjamin Klein, Guy Lev, Gil Sadeh, Lior Wolf","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Krizhevsky et al., 2017] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Commun. ACM, 60(6):84-90, may 2017.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2017","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kupiec et al., 1995] Julian Kupiec, Jan Pedersen, and Francine Chen. A trainable document summarizer. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, pages 68-73, 1995.","title":"A trainable document summarizer","authors":"Julian Kupiec, Jan Pedersen, Francine Chen","year":"1995","venue":"Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lewis et al., 2020] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics, 2020.","title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin, 2004] Chin-Yew Lin. ROUGE: A package for auto- matic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, 2004.","title":"ROUGE: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text Summarization Branches Out","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Litvak et al., 2010] Marina Litvak, Mark Last, and Mena- hem Friedman. A new approach to improving multilingual summarization using a genetic algorithm. In Proceedings of the 48th annual meeting of the association for compu- tational linguistics, pages 927-936, 2010.","title":"A new approach to improving multilingual summarization using a genetic algorithm","authors":"Litvak","year":"2010","venue":"Proceedings of the 48th annual meeting of the association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu and Lapata, 2019] Yang Liu and Mirella Lapata. Text summarization with pretrained encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), pages 3730-3740, Hong Kong, China, Novem- ber 2019. ACL. Mihalcea and Tarau, 2004] Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In Proceedings of the 2004 conference on empirical methods in natural lan- guage processing, pages 404-411, 2004.","title":"Text summarization with pretrained encoders","authors":"Lapata ; Liu, Yang Liu, Mirella Lapata, Rada Mihalcea, Paul Tarau","year":"2004","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Moosavi et al., 2021] Nafise Sadat Moosavi, Andreas Rücklé, Dan Roth, and Iryna Gurevych. Scigen: a dataset for reasoning-aware text generation from scientific tables. In Thirty-fifth Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track (Round 2), 2021.","title":"Scigen: a dataset for reasoning-aware text generation from scientific tables","authors":"Moosavi","year":"2021","venue":"Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nallapati et al., 2016] Ramesh Nallapati, Bowen Zhou, Ci- cero dos Santos, C ¸aglar Gulc ¸ehre, and Bing Xiang. Ab- stractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learn- ing, pages 280-290, 2016.","title":"Abstractive text summarization using sequence-to-sequence rnns and beyond","authors":"Ramesh Nallapati, Bowen Zhou, C Cicero Dos Santos, Bing ¸aglar Gulc ¸ehre, Xiang","year":"2016","venue":"Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Nayeem et al., 2018] Mir Tafseer Nayeem, Tanvir Ahmed Fuad, and Yllias Chali. Abstractive unsupervised multi- document summarization using paraphrastic sentence fu- sion. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1191-1204, Santa Fe, New Mexico, USA, August 2018. ACL.","title":"Abstractive unsupervised multidocument summarization using paraphrastic sentence fusion","authors":"Nayeem","year":"2018","venue":"Proceedings of the 27th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-140:67, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rothe et al., 2020] Sascha Rothe, Shashi Narayan, and Ali- aksei Severyn. Leveraging pre-trained checkpoints for se- quence generation tasks. TACL, 8:264-280, 2020.","title":"Leveraging pre-trained checkpoints for sequence generation tasks","authors":"Sascha Rothe, Shashi Narayan, Aliaksei Severyn","year":"2020","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Saini et al., 2018] Naveen Saini, Sriparna Saha, Anubhav Jangra, and Pushpak Bhattacharyya. Extractive single doc- ument summarization using multi-objective optimization: Exploring self-organized differential evolution, grey wolf optimizer and water cycle algorithm. Knowledge-Based Systems, 164, 11 2018.","title":"Extractive single document summarization using multi-objective optimization: Exploring self-organized differential evolution, grey wolf optimizer and water cycle algorithm","authors":"Naveen Saini, Sriparna Saha, Anubhav Jangra, Pushpak Bhattacharyya","year":"2018","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[See et al., 2017] Abigail See, Peter J Liu, and Christopher D Manning. Get to the point: Summarization with pointer- generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083, 2017.","title":"Get to the point: Summarization with pointergenerator networks","authors":"See","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tampe et al., 2022] Ignacio Tampe, Marcelo Mendoza, and Evangelos Milios. Neural abstractive unsupervised sum- marization of online news discussions. In Kohei Arai, ed- itor, Intelligent Systems and Applications, pages 822-841, Cham, 2022. Springer International Publishing.","title":"Neural abstractive unsupervised summarization of online news discussions","authors":"Tampe","year":"2022","venue":"itor, Intelligent Systems and Applications","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2021] Hongyan Xu, Hongtao Liu, Pengfei Jiao, and Wenjun Wang. Transformer reasoning network for personalized review summarization. In SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.","title":"Transformer reasoning network for personalized review summarization","authors":"Hongyan Xu, Hongtao Liu, Pengfei Jiao, Wenjun Wang","year":"2021","venue":"SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yin et al., 2020] Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. TaBERT: Pretraining for joint understanding of textual and tabular data. In Proc. of ACL, pages 8413-8426, Online, July 2020. ACL.","title":"TaBERT: Pretraining for joint understanding of textual and tabular data","authors":"Pengcheng Yin, Graham Neubig, Wen-Tau Yih, Sebastian Riedel","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2020a] Jingqing Zhang, Yao Zhao, Moham- mad Saleh, and Peter Liu. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, vol- ume 119 of Proceedings of Machine Learning Research, pages 11328-11339. PMLR, 13-18 Jul 2020.","title":"PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization","authors":"Zhang","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang* et al., 2020b] Tianyi Zhang*, Varsha Kishore*, Fe- lix Wu*, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In ICLR, 2020.","title":"Bertscore: Evaluating text generation with bert","authors":"* Zhang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}