{"type":"ScholarlyArticle","title":"Abstract Rule Learning for Paraphrase Generation","authors":[{"name":"Xianggen Liu","slug":"xianggen-liu","given":"Xianggen","family":"Liu","affiliation":"College of Computer Science , Sichuan University"},{"name":"Wenqiang Lei","slug":"wenqiang-lei","given":"Wenqiang","family":"Lei","affiliation":"College of Computer Science , Sichuan University"},{"name":"Jiancheng Lv","slug":"jiancheng-lv","given":"Jiancheng","family":"Lv","affiliation":"College of Computer Science , Sichuan University"},{"name":"Jizhe Zhou","slug":"jizhe-zhou","given":"Jizhe","family":"Zhou","affiliation":"College of Computer Science , Sichuan University"}],"abstract":"In early years, paraphrase generation typically adopts rule-based methods, which are interpretable and able to make global transformations to the original sentence. But they struggle to produce fluent paraphrases. Recently, deep neural networks have shown impressive performances in generating paraphrases. However, the current neural models are black boxes and are prone to make local modifications to the inputs. In this work, we combine these two approaches into RULER, a novel approach that performs abstract rule learning for paraphrasing. The key idea is to explicitly learn generalizable rules that could enhance the paraphrase generation process of neural networks. In RULER, we first propose a rule generalizability metric to guide the model to generate rules underlying the paraphrasing. Then, we leverage neural networks to generate paraphrases by refining the sentences transformed by the learned rules. Extensive experimental results demonstrate the superiority of RULER over previous state-of-the-art methods in terms of paraphrase quality, generalization ability and interpretability.","doi":"10.24963/ijcai.2022/593","url":"https://ijcai.org/proceedings/2022/593","pdf":"https://ijcai.org/proceedings/2022/0593.pdf","markdown":"https://ijcai.org/proceedings/2022/593.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/593","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":593,"pages":{"first":4273,"last":4279},"keywords":["Knowledge Representation and Reasoning: Learning and reasoning","Natural Language Processing: Summarization","Natural Language Processing: Language Generation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986500","length_sec":76,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984643","length_sec":299,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Barzilay and Lee, 2003] Regina Barzilay and Lillian Lee. Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. In ACL, pages 16-23, 2003.","title":"Learning to paraphrase: An unsupervised approach using multiple-sequence alignment","authors":"Lee ; Barzilay, Regina Barzilay, Lillian Lee","year":"2003","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dolan et al., 2004] William Dolan, Chris Quirk, Chris Brockett, and Bill Dolan. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In COLING, pages 350-356, 2004.","title":"Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources","authors":"William Dolan, Chris Quirk, Chris Brockett, Bill Dolan","year":"2004","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fader et al., 2013] Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. Paraphrase-driven learning for open ques- tion answering. In ACL, pages 1608-1618, 2013.","title":"Paraphrase-driven learning for open question answering","authors":"Anthony Fader, Luke Zettlemoyer, Oren Etzioni","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fu et al., 2019] Yao Fu, Yansong Feng, and John P Cun- ningham. Paraphrase generation with latent bag of words. In NeurIPS, pages 13645-13656, 2019.","title":"Paraphrase generation with latent bag of words","authors":"Yao Fu, Yansong Feng, John P Cunningham","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gao et al., 2020] Silin Gao, Yichi Zhang, Zhijian Ou, and Zhou Yu. Paraphrase augmented task-oriented dialog gen- eration. In ACL, pages 639-649, 2020.","title":"Paraphrase augmented task-oriented dialog generation","authors":"Silin Gao, Yichi Zhang, Zhijian Ou, Zhou Yu","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goyal and Durrett, 2020] Tanya Goyal and Greg Durrett. Neural syntactic preordering for controlled paraphrase generation. In ACL, pages 238-252, 2020.","title":"Neural syntactic preordering for controlled paraphrase generation","authors":"Tanya Goyal, Greg Durrett","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gupta et al., 2018] Ankush Gupta, Arvind Agarwal, Prawaan Singh, and Piyush Rai. A deep generative framework for paraphrase generation. In AAAI, pages 5149-5156, 2018.","title":"A deep generative framework for paraphrase generation","authors":"Ankush Gupta, Arvind Agarwal, Prawaan Singh, Piyush Rai","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hosking and Lapata, 2021] Tom Hosking and Mirella Lap- ata. Factorising meaning and form for intent-preserving paraphrasing. In ACL-IJCNLP, pages 1405-1418, 2021.","title":"Tom Hosking and Mirella Lapata. Factorising meaning and form for intent-preserving paraphrasing","authors":"Lapata Hosking","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2016] Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric Xing. Harnessing deep neural net- works with logic rules. In ACL, pages 2410-2420, 2016.","title":"Harnessing deep neural networks with logic rules","authors":"Hu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Knight and Marcu, 2000] Kevin Knight and Daniel Marcu. Statistics-based summarization step one: Sentence com- pression. In AAAI, pages 703-710, 2000.","title":"Statistics-based summarization step one: Sentence compression","authors":"Marcu Knight, Kevin Knight, Daniel Marcu","year":"2000","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kumar et al., 2019] Ashutosh Kumar, Satwik Bhattamishra, Manik Bhandari, and Partha Talukdar. Submodular optimization-based diverse paraphrasing and its effective- ness in data augmentation. In NAACL, pages 3609-3619, 2019.","title":"Submodular optimization-based diverse paraphrasing and its effectiveness in data augmentation","authors":"Ashutosh Kumar, Satwik Bhattamishra, Manik Bhandari, Partha Talukdar","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2019] Zichao Li, Xin Jiang, Lifeng Shang, and Qun Liu. Decomposable neural paraphrase generation. In ACL, pages 3403--3414, 2019.","title":"Decomposable neural paraphrase generation","authors":"Zichao Li, Xin Jiang, Lifeng Shang, Qun Liu","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin and Pantel, 2001] Dekang Lin and Patrick Pantel. Dis- covery of inference rules for question-answering. Natural Language Engineering, 7(4):343-360, 2001.","title":"Discovery of inference rules for question-answering","authors":"Lin, Pantel ;, Dekang Lin, Patrick Pantel","year":"2001","venue":"Natural Language Engineering","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin and Wan, 2021] Zhe Lin and Xiaojun Wan. Pushing paraphrase away from original sentence: A multi-round paraphrase generation approach. In ACL-IJCNLP, pages 1548-1557, 2021.","title":"Pushing paraphrase away from original sentence: A multi-round paraphrase generation approach","authors":"Zhe Lin, Xiaojun Wan","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2018] Xianggen Liu, Lili Mou, Haotian Cui, Zhengdong Lu, and Sen Song. Jumper: Learning when to make classification decision in reading. In IJCAI, pages 4237-4243, 2018.","title":"Jumper: Learning when to make classification decision in reading","authors":"Liu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/589"},{"position":16,"raw":"[Liu et al., 2020] Xianggen Liu, Lili Mou, Fandong Meng, Hao Zhou, Jie Zhou, and Sen Song. Unsupervised para- phrasing by simulated annealing. In ACL, pages 302-312, 2020.","title":"Unsupervised paraphrasing by simulated annealing","authors":"Liu","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2021] Xianggen Liu, Pengyong Li, Fandong Meng, Hao Zhou, Huasong Zhong, Jie Zhou, Lili Mou, and Sen Song. Simulated annealing for optimization of graphs and sequences. Neurocomputing, 465:310-324, 2021.","title":"Simulated annealing for optimization of graphs and sequences","authors":"Liu","year":"2021","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Marcus et al., 1999] Gary F Marcus, Sugumaran Vijayan, S Bandi Rao, and Peter M Vishton. Rule learning by seven-month-old infants. Science, 283(5398):77-80, 1999.","title":"Rule learning by seven-month-old infants","authors":"Sugumaran Gary F Marcus, S Vijayan, Peter M Bandi Rao, Vishton","year":"1999","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mckeown, 1983] Kathleen R Mckeown. Paraphrasing ques- tions using given and new information. Computational Linguistics, 9(1):1-10, 1983.","title":"Paraphrasing questions using given and new information","authors":"Kathleen R Mckeown","year":"1983","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Miao et al., 2019] Ning Miao, Hao Zhou, Lili Mou, Rui Yan, and Lei Li. Constrained sentence generation by Metropolis-Hastings sampling. In AAAI, pages 6834- 6842, 2019.","title":"Constrained sentence generation by Metropolis-Hastings sampling","authors":"Ning Miao, Hao Zhou, Lili Mou, Rui Yan, Lei Li","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Prakash et al., 2016] Aaditya Prakash, Sadid A Hasan, Kathy Lee, Vivek Datla, Ashequl Qadir, Joey Liu, and Oladimeji Farri. Neural paraphrase generation with stacked residual LSTM networks. In COLING, pages 2923-2934, 2016.","title":"Neural paraphrase generation with stacked residual LSTM networks","authors":"Aaditya Prakash, A Sadid, Kathy Hasan, Vivek Lee, Ashequl Datla, Joey Qadir, Oladimeji Liu, Farri","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Quirk et al., 2004] Chris Quirk, Chris Brockett, and William B Dolan. Monolingual machine translation for paraphrase generation. In EMNLP, pages 142-149, 2004.","title":"Monolingual machine translation for paraphrase generation","authors":"Chris Quirk, Chris Brockett, William B Dolan","year":"2004","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Van den Oord et al., 2017] Aaron Van den Oord, Oriol Vinyals, et al. Neural discrete representation learning. In NeurIPS, pages 6306-6315, 2017.","title":"Neural discrete representation learning","authors":"Oord Van Den","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2019] Su Wang, Rahul Gupta, Nancy Chang, and Jason Baldridge. A task in a suit and a tie: Paraphrase generation with semantic augmentation. In AAAI, pages 7176-7183, 2019.","title":"A task in a suit and a tie: Paraphrase generation with semantic augmentation","authors":"Su Wang, Rahul Gupta, Nancy Chang, Jason Baldridge","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wieting and Gimpel, 2017] John Wieting and Kevin Gim- pel. ParaNMT-50M: Pushing the limits of paraphrastic sentence embeddings with millions of machine transla- tions. In ACL, pages 451-462, 2017.","title":"ParaNMT-50M: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations","authors":"Gimpel Wieting, John Wieting, Kevin Gimpel","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Williams, 1992] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8(3):229-256, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Williams Ronald","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao et al., 2009] Shiqi Zhao, Xiang Lan, Ting Liu, and Sheng Li. Application-driven statistical paraphrase gen- eration. In ACL, pages 834-842, 2009.","title":"Application-driven statistical paraphrase generation","authors":"Shiqi Zhao, Xiang Lan, Ting Liu, Sheng Li","year":"2009","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}