{"type":"ScholarlyArticle","title":"Generating Reasonable Legal Text through the Combination of Language Modeling and Question Answering","authors":[{"name":"Weijing Huang","slug":"weijing-huang","given":"Weijing","family":"Huang","affiliation":""},{"name":"Xianfeng Liao","slug":"xianfeng-liao","given":"Xianfeng","family":"Liao","affiliation":"University of Science and Technology of China"},{"name":"Zhiqiang Xie","slug":"zhiqiang-xie","given":"Zhiqiang","family":"Xie","affiliation":"University of Science and Technology of China"},{"name":"Jiang Qian","slug":"jiang-qian","given":"Jiang","family":"Qian","affiliation":""},{"name":"Bojin Zhuang","slug":"bojin-zhuang","given":"Bojin","family":"Zhuang","affiliation":""},{"name":"Shaojun Wang","slug":"shaojun-wang","given":"Shaojun","family":"Wang","affiliation":""},{"name":"Jing Xiao","slug":"jing-xiao","given":"Jing","family":"Xiao","affiliation":"Ping An Insurance (Group ) Company of China"}],"abstract":"Due to the improvement of Language Modeling, the emerging NLP assistant tools aiming for text generation greatly reduce the human workload on writing documents. However, the generation of legal text faces greater challenges than ordinary texts because of its high requirement for keeping logic reasonable, which can not be guaranteed by  Language Modeling right now. To generate reasonable legal documents, we propose a novel method CoLMQA, which (1) combines Language Modeling and Question Answering, (2) generates text with slots by Language Modeling, and (3) fills the slots by our proposed Question Answering method named Transformer-based Key-Value Memory Networks. In CoLMQA, the slots represent the text part that needs to be highly constrained by logic, such as the name of the law and the number of the law article. And the Question Answering fills the slots in context with the help of Legal Knowledge Base to keep logic reasonable. The experiment verifies the quality of legal documents generated by CoLMQA, surpassing the documents generated by pure Language Modeling.","doi":"10.24963/ijcai.2020/510","url":"https://ijcai.org/proceedings/2020/510","pdf":"https://ijcai.org/proceedings/2020/0510.pdf","markdown":"https://ijcai.org/proceedings/2020/510.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/510","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":510,"pages":{"first":3687,"last":3693},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Question Answering","Natural Language Processing: Natural Language Generation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25296/file_25296.m3u8","length_sec":292,"provider":"hls"}],"references":[{"position":1,"raw":"[Alschner and Skougarevskiy, 2017] Wolfgang Alschner and Dmitriy Skougarevskiy. Towards an automated production of legal texts using recurrent neural networks. In ICAIL, pages 229-232. ACM, 2017.","title":"Wolfgang Alschner and Dmitriy Skougarevskiy. Towards an automated production of legal texts using recurrent neural networks","authors":"Skougarevskiy Alschner","year":"2017","venue":"ICAIL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andor et al., 2019] Daniel Andor, Luheng He, Kenton Lee, and Emily Pitler. Giving BERT a calculator: Finding op- erations and arguments with reading comprehension. In EMNLP, 2019.","title":"Giving BERT a calculator: Finding operations and arguments with reading comprehension","authors":"Daniel Andor, Luheng He, Kenton Lee, Emily Pitler","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bellegarda, 2004] Jerome R Bellegarda. Statistical lan- guage model adaptation: review and perspectives. Speech communication, 42(1):93-108, 2004.","title":"Statistical language model adaptation: review and perspectives","authors":"Jerome R Bellegarda","year":"2004","venue":"Speech communication","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Berant et al., 2013] Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on Freebase from question-answer pairs. In EMNLP, 2013.","title":"Semantic parsing on Freebase from question-answer pairs","authors":"Jonathan Berant, Andrew Chou, Roy Frostig, Percy Liang","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bordes et al., 2015] Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075, 2015.","title":"Large-scale simple question answering with memory networks","authors":"Antoine Bordes, Nicolas Usunier, Sumit Chopra, Jason Weston","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bouraoui et al., 2020] Zied Bouraoui, Jose Camacho- Collados, and Steven Schockaert. Inducing relational knowledge from bert. AAAI, 2020.","title":"Inducing relational knowledge from bert","authors":"Zied Bouraoui, Jose Camacho-Collados, Steven Schockaert","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Branting, 1998] L Karl Branting. Techniques for automated drafting of judicial documents. IJLIT, 6(2):214-229, 1998.","title":"Techniques for automated drafting of judicial documents","authors":"Branting Karl","year":"1998","venue":"IJLIT","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2019] Huajie Chen, Deng Cai, Wei Dai, Zehui Dai, and Yadong Ding. Charge-based prison term predic- tion with deep gating network. EMNLP, 2019.","title":"Charge-based prison term prediction with deep gating network","authors":"Huajie Chen, Deng Cai, Wei Dai, Zehui Dai, Yadong Ding","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Das et al., 2017] Rajarshi Das, Manzil Zaheer, Siva Reddy, and Andrew McCallum. Question answering on knowl- edge bases and text using universal schema and memory networks. In ACL, 2017.","title":"Question answering on knowledge bases and text using universal schema and memory networks","authors":"Rajarshi Das, Manzil Zaheer, Siva Reddy, Andrew Mccallum","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In ACL, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dinan et al., 2019] Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of wikipedia: Knowledge-powered conversational agents. In ICLR, 2019.","title":"Wizard of wikipedia: Knowledge-powered conversational agents","authors":"Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, Jason Weston","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gostojić and Marković, 2019] Stevan Gostojić and Marko Marković. Legal document management: An integrated approach. In Sinteza 2019, pages 374-380, 2019.","title":"Legal document management: An integrated approach","authors":"Marković Gostojić, Stevan Gostojić, Marko Marković","year":"2019","venue":"Sinteza 2019","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guan et al., 2020] Jian Guan, Fei Huang, Zhihao Zhao, Xi- aoyan Zhu, and Minglie Huang. A knowledge-enhanced pretraining model for commonsense story generation. TACL, 2020.","title":"A knowledge-enhanced pretraining model for commonsense story generation","authors":"Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, Minglie Huang","year":"2020","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2018] Zikun Hu, Xiang Li, Cunchao Tu, Zhiyuan Liu, and Maosong Sun. Few-shot charge prediction with discriminative legal attributes. In COLING, 2018.","title":"Few-shot charge prediction with discriminative legal attributes","authors":"Zikun Hu, Xiang Li, Cunchao Tu, Zhiyuan Liu, Maosong Sun","year":"2018","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kannan et al., 2016] Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Mik- los, Greg Corrado, Laszlo Lukacs, Marina Ganea, Peter Young, et al. Smart reply: Automated response suggestion for email. In KDD, pages 955-964, 2016.","title":"Smart reply: Automated response suggestion for email","authors":"Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Miklos, Greg Corrado, Laszlo Lukacs, Marina Ganea, Peter Young","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Logan et al., 2019] Robert Logan, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. Barack's wife hillary: Using knowledge graphs for fact-aware language modeling. In ACL, 2019.","title":"Barack's wife hillary: Using knowledge graphs for fact-aware language modeling","authors":"Robert Logan, Nelson F Liu, Matthew E Peters, Matt Gardner, Sameer Singh","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Luo et al., 2017] Bingfeng Luo, Yansong Feng, Jianbo Xu, Xiang Zhang, and Dongyan Zhao. Learning to predict charges for criminal cases with legal basis. arXiv preprint arXiv:1707.09168, 2017.","title":"Learning to predict charges for criminal cases with legal basis","authors":"Luo","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mao et al., 2019] Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian McAuley, and Garrison W Cottrell. Im- proving neural story generation by targeted common sense grounding. EMNLP, 2019.","title":"Improving neural story generation by targeted common sense grounding","authors":"Mao","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mikolov et al., 2010] Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černockỳ, and Sanjeev Khudanpur. Re- current neural network based language model. In INTER- SPEECH, 2010.","title":"Recurrent neural network based language model","authors":"Mikolov","year":"2010","venue":"INTER-SPEECH","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Miller et al., 2016] Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. Key-value memory networks for directly reading documents. In EMNLP, 2016.","title":"Key-value memory networks for directly reading documents","authors":"Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein, Antoine Karimi, Jason Bordes, Weston","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Petroni et al., 2019] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? EMNLP, 2019.","title":"Language models as knowledge bases? EMNLP","authors":"Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, Sebastian Riedel","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Radford et al., 2018] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. 2018.","title":"Improving language understanding by generative pre-training","authors":"Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI Blog","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In EMNLP, 2016.","title":"SQuAD: 100,000+ questions for machine comprehension of text","authors":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sukhbaatar et al., 2015] Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-to-end memory networks. In NIPS, pages 2440-2448, 2015.","title":"End-to-end memory networks","authors":"Sukhbaatar","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Weston et al., 2015] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. ICLR, 2015.","title":"Memory networks","authors":"Jason Weston, Sumit Chopra","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xu et al., 2019] Kun Xu, Yuxuan Lai, Yansong Feng, and Zhiguo Wang. Enhancing key-value memory neural net- works for knowledge based question answering. In ACL, pages 2937-2947, 2019.","title":"Enhancing key-value memory neural networks for knowledge based question answering","authors":"Kun Xu, Yuxuan Lai, Yansong Feng, Zhiguo Wang","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ye et al., 2018] Hai Ye, Xin Jiang, Zhunchen Luo, and Wenhan Chao. Interpretable charge predictions for crimi- nal cases: Learning to generate court views from fact de- scriptions. In NAACL, pages 1854-1864, 2018.","title":"Interpretable charge predictions for criminal cases: Learning to generate court views from fact descriptions","authors":"Hai Ye, Xin Jiang, Zhunchen Luo, Wenhan Chao","year":"2018","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ye et al., 2020] Rong Ye, Wenxian Shi, Hao Zhou, and Lei Li. Variational template machine for data-to-text genera- tion. In ICLR, 2020.","title":"Variational template machine for data-to-text generation","authors":"Rong Ye, Wenxian Shi, Hao Zhou, Lei Li","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhong et al., 2018] Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Chaojun Xiao, Zhiyuan Liu, and Maosong Sun. Legal judgment prediction via topological learning. In EMNLP, pages 3540-3549, 2018.","title":"Legal judgment prediction via topological learning","authors":"Zhong","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhong et al., 2020] Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. Jec- qa: A legal-domain question answering dataset. AAAI, 2020.","title":"Jecqa: A legal-domain question answering dataset","authors":"Zhong","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}