{"type":"ScholarlyArticle","title":"CUP: Curriculum Learning based Prompt Tuning for Implicit Event Argument Extraction","authors":[{"name":"Jiaju Lin","slug":"jiaju-lin","given":"Jiaju","family":"Lin","affiliation":"School of Computer Science and Technology , East China Normal University"},{"name":"Qin Chen","slug":"qin-chen","given":"Qin","family":"Chen","affiliation":"School of Computer Science and Technology , East China Normal University"},{"name":"Jie Zhou","slug":"jie-zhou","given":"Jie","family":"Zhou","affiliation":"School of Computer Science , Fudan University"},{"name":"Jian Jin","slug":"jian-jin","given":"Jian","family":"Jin","affiliation":"School of Computer Science and Technology , East China Normal University"},{"name":"Liang He","slug":"liang-he","given":"Liang","family":"He","affiliation":"School of Computer Science and Technology , East China Normal University"}],"abstract":"Implicit event argument extraction (EAE) aims to identify arguments that could scatter over the document. Most previous work focuses on learning the direct relations between arguments and the given trigger, while the implicit relations with long-range dependency are not well studied. Moreover, recent neural network based approaches rely on a large amount of labeled data for training, which is unavailable due to the high labelling cost. In this paper, we propose a Curriculum learning based Prompt tuning (CUP) approach, which resolves implicit EAE by four learning stages. The stages are defined according to the relations with the trigger node in a semantic graph, which well captures the long-range dependency between arguments and the trigger. In addition, we integrate a prompt-based encoder-decoder model to elicit related knowledge from pre-trained language models (PLMs) in each stage, where the prompt templates are adapted with the learning progress to enhance the reasoning for arguments. Experimental results on two well-known benchmark datasets show the great advantages of our proposed approach. In particular, we outperform the state-of-the-art models in both fully-supervised and low-data scenarios.","doi":"10.24963/ijcai.2022/589","url":"https://ijcai.org/proceedings/2022/589","pdf":"https://ijcai.org/proceedings/2022/0589.pdf","markdown":"https://ijcai.org/proceedings/2022/589.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/589","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":589,"pages":{"first":4245,"last":4251},"keywords":["Natural Language Processing: Language Models","Natural Language Processing: Information Extraction"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986736","length_sec":85,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984879","length_sec":353,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Banarescu et al., 2013] Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. Abstract meaning representation for sembanking. In ACL Work- shop, pages 178-186, 2013.","title":"Abstract meaning representation for sembanking","authors":"Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, Nathan Schneider","year":"2013","venue":"ACL Workshop","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2009] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML, ICML '09, page 41-48, New York, NY, USA, 2009. Association for Computing Machinery.","title":"Curriculum learning","authors":"Yoshua Bengio, Jérôme Louradour, Ronan Collobert, Jason Weston","year":"2009","venue":"ICML, ICML '09","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bevilacqua et al., 2021] Michele Bevilacqua, Rexhina Blloshmi, and Roberto Navigli. One spring to rule them both: Symmetric amr semantic parsing and generation without a complex pipeline. In AAAI, 2021.","title":"One spring to rule them both: Symmetric amr semantic parsing and generation without a complex pipeline","authors":"Michele Bevilacqua, Rexhina Blloshmi, Roberto Navigli","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2015] Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. Event extraction via dynamic multi-pooling convolutional neural networks. In IJCNLP, pages 167-176, Bei- jing, China, July 2015.","title":"Event extraction via dynamic multi-pooling convolutional neural networks","authors":"Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, Jun Zhao","year":"2015","venue":"IJCNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Yunmo Chen, Tongfei Chen, and Benjamin Van Durme. Joint modeling of arguments for event understand- ing. In Proceedings of the First Workshop on Computational Ap- proaches to Discourse, pages 96-101, Online, November 2020. Association for Computational Linguistics.","title":"Joint modeling of arguments for event understanding","authors":"Chen","year":"2020","venue":"Proceedings of the First Workshop on Computational Approaches to Discourse","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Du and Cardie, 2020a] Xinya Du and Claire Cardie. Document- level event role filler extraction using multi-granularity contextu- alized encoding. In ACL, pages 8010-8020, 2020.","title":"Xinya Du and Claire Cardie. Documentlevel event role filler extraction using multi-granularity contextualized encoding","authors":"Cardie Du","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du and Cardie, 2020b] Xinya Du and Claire Cardie. Event extrac- tion by answering (almost) natural questions. In EMNLP, pages 671-683, 2020.","title":"Xinya Du and Claire Cardie. Event extraction by answering (almost) natural questions","authors":"Cardie Du","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Du et al., 2020] Xinya Du, Alexander Rush, and Claire Cardie. Document-level event-based extraction using generative template-filling transformers. arXiv e-prints, pages arXiv-2008, 2020.","title":"Xinya Du","authors":"Du","year":"2020","venue":"Document-level event-based extraction using generative template-filling transformers","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ebner et al., 2020] Seth Ebner, Patrick Xia, Ryan Culkin, Kyle Rawlins, and Benjamin Van Durme. Multi-sentence argument linking. In ACL, pages 8057-8077, Online, July 2020.","title":"Multi-sentence argument linking","authors":"Seth Ebner, Patrick Xia, Ryan Culkin, Kyle Rawlins, Benjamin Van Durme","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lee et al., 2018] Kenton Lee, Luheng He, and L. Zettlemoyer. Higher-order coreference resolution with coarse-to-fine infer- ence. In NAACL-HLT, 2018.","title":"Higher-order coreference resolution with coarse-to-fine inference","authors":"Kenton Lee, Luheng He, L Zettlemoyer","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Con- stant. The power of scale for parameter-efficient prompt tuning. In EMNLP, pages 3045-3059, Online and Punta Cana, Domini- can Republic, November 2021.","title":"The power of scale for parameter-efficient prompt tuning","authors":"Brian Lester, Rami Al-Rfou, Noah Constant","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lewis et al., 2020] Mike Lewis, Yinhan Liu, Naman Goyal, Mar- jan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence- to-sequence pre-training for natural language generation, trans- lation, and comprehension. In ACL, pages 7871-7880, Online, July 2020.","title":"BART: Denoising sequenceto-sequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In ACL-IJCNLP, pages 4582-4597, Online, August 2021.","title":"Prefix-tuning: Optimizing continuous prompts for generation","authors":"Li, Liang ; Xiang, Lisa Li, Percy Liang","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2019] Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. Entity-relation extraction as multi-turn question answering. In ACL, pages 1340- 1350, Florence, Italy, July 2019.","title":"Entity-relation extraction as multi-turn question answering","authors":"Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, Jiwei Li","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2020] Fayuan Li, Weihua Peng, Yuguang Chen, Quan Wang, Lu Pan, Yajuan Lyu, and Yong Zhu. Event extraction as multi-turn question answering. In EMNLP: Findings, pages 829-838, 2020.","title":"Event extraction as multi-turn question answering","authors":"Fayuan Li, Weihua Peng, Yuguang Chen, Quan Wang, Lu Pan, Yajuan Lyu, Yong Zhu","year":"2020","venue":"EMNLP: Findings","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2021] Sha Li, Heng Ji, and Jiawei Han. Document-level event argument extraction by conditional generation. In NAACL, pages 894-908, 2021.","title":"Document-level event argument extraction by conditional generation","authors":"Li","year":"2021","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2018] Xiao Liu, Zhunchen Luo, and Heyan Huang. Jointly multiple events extraction via attention-based graph in- formation aggregation. In EMNLP, pages 1247-1256, 2018.","title":"Jointly multiple events extraction via attention-based graph information aggregation","authors":"Xiao Liu, Zhunchen Luo, Heyan Huang","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2020] Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and Xi- aojiang Liu. Event extraction as machine reading comprehension. In EMNLP, pages 1641-1651, Online, November 2020.","title":"Event extraction as machine reading comprehension","authors":"Jian Liu, Yubo Chen, Kang Liu, Wei Bi, Xiaojiang Liu","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2021a] Jian Liu, Yufeng Chen, and Jinan Xu. Machine reading comprehension as data augmentation: A case study on implicit event argument extraction. In EMNLP, pages 2716- 2725, Online and Punta Cana, Dominican Republic, November 2021.","title":"Machine reading comprehension as data augmentation: A case study on implicit event argument extraction","authors":"Liu","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2021b] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. CoRR, abs/2107.13586, 2021.","title":"Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing","authors":"Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mostafazadeh Davani et al., 2019] Aida Mostafazadeh Davani, Leigh Yeh, Mohammad Atari, Brendan Kennedy, Gwenyth Por- tillo Wightman, Elaine Gonzalez, Natalie Delong, Rhea Bhatia, Arineh Mirinjian, Xiang Ren, and Morteza Dehghani. Reporting the unreported: Event extraction for analyzing the local repre- sentation of hate crimes. In EMNLP-IJCNLP, pages 5753-5757, 2019.","title":"Reporting the unreported: Event extraction for analyzing the local representation of hate crimes","authors":"Mostafazadeh Davani","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Petroni et al., 2019] Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexan- der Miller. Language models as knowledge bases? In EMNLP- IJCNLP, pages 2463-2473, Hong Kong, China, November 2019.","title":"Language models as knowledge bases?","authors":"Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wei et al., 2021] Kaiwen Wei, Xian Sun, Zequn Zhang, Jingyuan Zhang, Guo Zhi, and Li Jin. Trigger is not sufficient: Exploiting frame-aware knowledge for implicit event argument extraction. In ACL-IJCNLP, pages 4672-4682, Online, August 2021.","title":"Trigger is not sufficient: Exploiting frame-aware knowledge for implicit event argument extraction","authors":"Kaiwen Wei, Xian Sun, Zequn Zhang, Jingyuan Zhang, Guo Zhi, Li Jin","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xiangyu et al., 2021] Xi Xiangyu, Wei Ye, Shikun Zhang, Quanxiu Wang, Huixing Jiang, and Wei Wu. Capturing event argument interaction via a bi-directional entity-level recurrent de- coder. In IJCNLP, pages 210-219, Online, August 2021.","title":"Capturing event argument interaction via a bi-directional entity-level recurrent decoder","authors":"Xi Xiangyu, Wei Ye, Shikun Zhang, Quanxiu Wang, Huixing Jiang, Wei Wu","year":"2021","venue":"IJCNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhan and Jiang, 2019] Liying Zhan and Xuping Jiang. Survey on event extraction technology in information extraction research area. In ITNEC, pages 2121-2126. IEEE, 2019.","title":"Survey on event extraction technology in information extraction research area","authors":"Liying Zhan, Xuping Jiang","year":"2019","venue":"ITNEC","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2020] Zhisong Zhang, Xiang Kong, Zhengzhong Liu, Xuezhe Ma, and Eduard Hovy. A two-step approach for implicit event argument detection. In ACL, pages 7479-7485, Online, July 2020.","title":"A two-step approach for implicit event argument detection","authors":"Zhang","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2021] Jiawei Zhou, Tahira Naseem, Ramón Fernan- dez Astudillo, and Radu Florian. AMR parsing with action- pointer transformer. In NAACL, pages 5585-5598, 2021.","title":"AMR parsing with actionpointer transformer","authors":"Jiawei Zhou, Tahira Naseem, Ramón Fernandez Astudillo, Radu Florian","year":"2021","venue":"NAACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}