{"type":"ScholarlyArticle","title":"Enhancing Text Generation via Multi-Level Knowledge Aware Reasoning","authors":[{"name":"Feiteng Mu","slug":"feiteng-mu","given":"Feiteng","family":"Mu","affiliation":"The Department of Computing , The Hong Kong Polytechnic University , Hong Kong"},{"name":"Wenjie Li","slug":"wenjie-li","given":"Wenjie","family":"Li","affiliation":"The Department of Computing , The Hong Kong Polytechnic University , Hong Kong"}],"abstract":"How to generate high-quality textual content is a non-trivial task. Existing methods generally generate text by grounding on word-level knowledge. However, word-level knowledge cannot express multi-word text units, hence existing methods may generate low-quality and unreasonable text. \r\nIn this paper, we leverage event-level knowledge to enhance text generation. \r\nHowever, event knowledge is very sparse. To solve this problem, we split a coarse-grained event into fine-grained word components to obtain the word-level knowledge among event components. The word-level knowledge models the interaction among event components, which makes it possible to reduce the sparsity of events. Based on the event-level and the word-level knowledge, we devise a multi-level knowledge aware reasoning framework. Specifically, we first utilize event knowledge to make event-based content planning, i.e., select reasonable event sketches conditioned by the input text. Then, we combine the selected event sketches with the word-level knowledge for text generation. We validate our method on two widely used datasets, experimental results demonstrate the effectiveness of our framework to text generation.","doi":"10.24963/ijcai.2022/598","url":"https://ijcai.org/proceedings/2022/598","pdf":"https://ijcai.org/proceedings/2022/0598.pdf","markdown":"https://ijcai.org/proceedings/2022/598.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/598","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":598,"pages":{"first":4310,"last":4316},"keywords":["Natural Language Processing: Language Generation","Knowledge Representation and Reasoning: Common-Sense Reasoning","Natural Language Processing: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986503","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984646","length_sec":341,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Banerjee and Lavie, 2005] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. Intrin- sic and Extrinsic Evaluation Measures for Machine Trans- lation and/or Summarization, page 65, 2005.","title":"Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization","authors":"Lavie Banerjee, Satanjeev Banerjee, Alon Lavie","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Goldfarb-Tarrant et al., 2020] Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. Content planning for neural story generation with aris- totelian rescoring. arXiv:2009.09870, 2020.","title":"Content planning for neural story generation with aristotelian rescoring","authors":"Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, Nanyun Peng","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Holmes-Higgin, 1994] Paul Holmes-Higgin. Text gener- ation-using discourse strategies and focus constraints to generate natural language text by kathleen r. mcke- own, cambridge university press, 1992, pp 246,£ 13.95, isbn 0-521-43802-0. The Knowledge Engineering Review, 9(4):421-422, 1994.","title":"Text generation-using discourse strategies and focus constraints to generate natural language text by kathleen r. mckeown","authors":"Paul Holmes-Higgin, Holmes-Higgin","year":"1992","venue":"The Knowledge Engineering Review","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hwang et al., 2020] Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. Comet-atomic 2020: On sym- bolic and neural commonsense knowledge graphs. arXiv preprint arXiv:2010.05953, 2020.","title":"Comet-atomic 2020: On symbolic and neural commonsense knowledge graphs","authors":"Jena D Hwang, Chandra Bhagavatula, Le Ronan, Jeff Bras, Keisuke Da, Antoine Sakaguchi, Yejin Bosselut, Choi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ji et al., 2020] Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, and Minlie Huang. Language gener- ation with multi-hop reasoning on commonsense knowl- edge graph. arXiv preprint arXiv:2009.11692, 2020.","title":"Language generation with multi-hop reasoning on commonsense knowledge graph","authors":"Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Lewis et al., 2019] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: De- noising sequence-to-sequence pre-training for natural lan- guage generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019.","title":"Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, Luke Zettlemoyer","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Li et al., 2015] Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting ob- jective function for neural conversation models. arXiv preprint arXiv:1510.03055, 2015.","title":"A diversity-promoting objective function for neural conversation models","authors":"Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, Bill Dolan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Mostafazadeh et al., 2016] Nasrin Mostafazadeh, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. pages 839-849, 2016.","title":"A corpus and cloze evaluation for deeper understanding of commonsense stories","authors":"Nasrin Mostafazadeh, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, James Allen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Compu- tational Linguistics, pages 311-318, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Puduppully et al., 2019] Ratish Puduppully, Li Dong, and Mirella Lapata. Data-to-text generation with content se- lection and planning. In Proceedings of the AAAI con- ference on artificial intelligence, volume 33, pages 6908- 6915, 2019.","title":"Data-to-text generation with content selection and planning","authors":"Ratish Puduppully, Li Dong, Mirella Lapata","year":"2019","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Radford","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Radinsky et al., 2012] Kira Radinsky, Sagie Davidovich, and Shaul Markovitch. Learning causality for news events prediction. In Proceedings of the 21st international con- ference on World Wide Web, pages 909-918, 2012.","title":"Learning causality for news events prediction","authors":"Kira Radinsky, Sagie Davidovich, Shaul Markovitch","year":"2012","venue":"Proceedings of the 21st international conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Raffel et al., 2019] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sap et al., 2019] Maarten Sap, Ronan Le Bras, Emily All- away, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. Atomic: An atlas of machine commonsense for if-then rea- soning. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 33, pages 3027-3035, 2019.","title":"Atomic: An atlas of machine commonsense for if-then reasoning","authors":"Sap","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Schlichtkrull et al., 2018] Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convo- lutional networks. In European semantic web conference, pages 593-607. Springer, 2018.","title":"Modeling relational data with graph convolutional networks","authors":"Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den, Ivan Berg, Max Titov, Welling","year":"2018","venue":"European semantic web conference","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Xu et al., 2020] Jun Xu, Zeyang Lei, Haifeng Wang, Zheng-Yu Niu, Hua Wu, and Wanxiang Che. Enhancing dialog coherence with event graph grounded content plan- ning. In IJCAI, pages 3941-3947, 2020.","title":"Enhancing dialog coherence with event graph grounded content planning","authors":"Jun Xu, Zeyang Lei, Haifeng Wang, Zheng-Yu Niu, Hua Wu, Wanxiang Che","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/545"},{"position":17,"raw":"[Yao et al., 2019] Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and- write: Towards better automatic storytelling. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 33, pages 7378-7385, 2019.","title":"Plan-andwrite: Towards better automatic storytelling","authors":"Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, Rui Yan","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhang et al., 2019a] Houyu Zhang, Zhenghao Liu, Chenyan Xiong, and Zhiyuan Liu. Grounded con- versation generation as guided traverses in commonsense knowledge graphs. arXiv:1911.02707, 2019.","title":"Grounded conversation generation as guided traverses in commonsense knowledge graphs","authors":"Zhang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang et al., 2019b] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019.","title":"Bertscore: Evaluating text generation with bert","authors":"Zhang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhou et al., 2018] Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. Com- monsense knowledge aware conversation generation with graph attention. In IJCAI, pages 4623-4629, 2018.","title":"Commonsense knowledge aware conversation generation with graph attention","authors":"Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, Xiaoyan Zhu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/643"},{"position":21,"raw":"[Zhou et al., 2020] Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam, Seyeon Lee, Bill Yuchen Lin, and Xiang Ren. Pre-training text-to-text transform- ers for concept-centric common sense. arXiv preprint arXiv:2011.07956, 2020.","title":"Pre-training text-to-text transformers for concept-centric common sense","authors":"Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam, Seyeon Lee, Bill Yuchen Lin, Xiang Ren","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}