{"type":"ScholarlyArticle","title":"PPAT: Progressive Graph Pairwise Attention Network for Event Causality Identification","authors":[{"name":"Zhenyu Liu","slug":"zhenyu-liu","given":"Zhenyu","family":"Liu","affiliation":"Harbin Institute of Technology , Shenzhen"},{"name":"Baotian Hu","slug":"baotian-hu","given":"Baotian","family":"Hu","affiliation":"Harbin Institute of Technology , Shenzhen"},{"name":"Zhenran Xu","slug":"zhenran-xu","given":"Zhenran","family":"Xu","affiliation":"Harbin Institute of Technology , Shenzhen"},{"name":"Min Zhang","slug":"min-zhang","given":"Min","family":"Zhang","affiliation":"Harbin Institute of Technology , Shenzhen"}],"abstract":"Event Causality Identification (ECI) aims to identify the causality between a pair of event mentions in a document, which is composed of sentence-level ECI (SECI) and document-level ECI (DECI). Previous work applies various reasoning models to identify the implicit event causality. However, they indiscriminately reason all event causality in the same way, ignoring that most inter-sentence event causality depends on intra-sentence event causality to infer.  In this paper, we propose a progressive graph pairwise attention network (PPAT) to consider the above dependence. PPAT applies a progressive reasoning strategy, as it first predicts the intra-sentence event causality, and then infers the more implicit inter-sentence event causality based on the SECI result. We construct a sentence boundary event relational graph, and PPAT leverages a simple pairwise attention mechanism, which attends to different reasoning chains on the graph. In addition, we propose a causality-guided training strategy for assisting PPAT in learning causality-related representations on every layer. Extensive experiments show that our model achieves state-of-the-art performance on three benchmark datasets (5.5%, 2.2% and 4.5% F1 gains on EventStoryLine, MAVEN-ERE and Causal-TimeBank). Code is available at https://github.com/HITsz-TMG/PPAT.","doi":"10.24963/ijcai.2023/572","url":"https://ijcai.org/proceedings/2023/572","pdf":"https://ijcai.org/proceedings/2023/0572.pdf","markdown":"https://ijcai.org/proceedings/2023/572.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/572","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Natural Language Processing","track":"Main Track","number":572,"pages":{"first":5150,"last":5158},"keywords":["Natural Language Processing: NLP: Information extraction","Natural Language Processing: NLP: Applications"],"videos":[],"references":[{"position":1,"raw":"[Beltagy et al., 2020] Iz Beltagy, Matthew E. Peters, and Ar- man Cohan. Longformer: The long-document trans- former. CoRR, abs/2004.05150, 2020.","title":"Longformer: The long-document transformer","authors":"Iz Beltagy, Matthew E Peters, Arman Cohan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berant et al., 2014] Jonathan Berant, Vivek Srikumar, Pei- Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang, Peter Clark, and Christopher D. Manning. Mod- eling biological processes for reading comprehension. In Proceedings of the 2014 Conference on Empirical Meth- ods in Natural Language Processing (EMNLP), pages 1499-1510, Doha, Qatar, October 2014. Association for Computational Linguistics.","title":"Modeling biological processes for reading comprehension","authors":"Jonathan Berant, Vivek Srikumar, Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang, Peter Clark, Christopher D Manning","year":"2014","venue":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2021] Pengfei Cao, Xinyu Zuo, Yubo Chen, Kang Liu, Jun Zhao, Yuguang Chen, and Weihua Peng. Knowledge-enriched event causality identification via la- tent structure induction networks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4862-4872, Online, August 2021. Association for Computational Linguistics.","title":"Knowledge-enriched event causality identification via latent structure induction networks","authors":"Pengfei Cao, Xinyu Zuo, Yubo Chen, Kang Liu, Jun Zhao, Yuguang Chen, Weihua Peng","year":"2021","venue":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Caselli and Vossen, 2017] Tommaso Caselli and Piek Vossen. The event StoryLine corpus: A new benchmark for causal and temporal relation extraction. In Proceedings of the Events and Stories in the News Workshop, pages 77-86, Vancouver, Canada, August 2017. Association for Computational Linguistics.","title":"The event StoryLine corpus: A new benchmark for causal and temporal relation extraction","authors":"Tommaso Caselli, Piek Vossen","year":"2017","venue":"Proceedings of the Events and Stories in the News Workshop","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022] Meiqi Chen, Yixin Cao, Kunquan Deng, Mukai Li, Kun Wang, Jing Shao, and Yan Zhang. ERGO: Event relational graph transformer for document-level event causality identification. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2118-2128, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguis- tics.","title":"ERGO: Event relational graph transformer for document-level event causality identification","authors":"Chen","year":"2022","venue":"Proceedings of the 29th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapo- lis, Minnesota, June 2019. Association for Computational Linguistics.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Do et al., 2011] Quang Do, Yee Seng Chan, and Dan Roth. Minimally supervised event causality identification. In Proceedings of the 2011 Conference on Empirical Meth- ods in Natural Language Processing, pages 294-303, Ed- inburgh, Scotland, UK., July 2011. Association for Com- putational Linguistics.","title":"Minimally supervised event causality identification","authors":"Quang Do, Yee Seng Chan, Dan Roth","year":"2011","venue":"Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2019] Lei Gao, Prafulla Kumar Choubey, and Ruihong Huang. Modeling document-level causal struc- tures for event causal relation identification. In Proceed- ings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Papers), pages 1808-1817, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.","title":"Modeling document-level causal structures for event causal relation identification","authors":"Lei Gao, Prafulla Kumar Choubey, Ruihong Huang","year":"2019","venue":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hashimoto, 2019] Chikara Hashimoto. Weakly supervised multilingual causality extraction from wikipedia. In Pro- ceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Interna- tional Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2988-2999, 2019.","title":"Weakly supervised multilingual causality extraction from wikipedia","authors":"Chikara Hashimoto","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hidey and McKeown, 2016] Christopher Hidey and Kathy McKeown. Identifying causal relations using parallel Wikipedia articles. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1424-1433, Berlin, Ger- many, August 2016. Association for Computational Lin- guistics.","title":"Identifying causal relations using parallel Wikipedia articles","authors":"Mckeown Hidey, Christopher Hidey, Kathy Mckeown","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2017] Zhichao Hu, Elahe Rahimtoroghi, and Mar- ilyn Walker. Inference of fine-grained event causality from blogs and films. In Proceedings of the Events and Stories in the News Workshop, pages 52-58, Vancouver, Canada, August 2017. Association for Computational Linguistics.","title":"Inference of fine-grained event causality from blogs and films","authors":"Zhichao Hu, Elahe Rahimtoroghi, Marilyn Walker","year":"2017","venue":"Proceedings of the Events and Stories in the News Workshop","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kadowaki et al., 2019] Kazuma Kadowaki, Ryu Iida, Ken- taro Torisawa, Jong-Hoon Oh, and Julien Kloetzer. Event causality recognition exploiting multiple annotators' judg- ments and background knowledge. In Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP-IJCNLP), pages 5816-5822, Hong Kong, China, November 2019. Association for Computational Linguistics.","title":"Event causality recognition exploiting multiple annotators' judgments and background knowledge","authors":"Kazuma Kadowaki, Ryu Iida, Kentaro Torisawa, Jong-Hoon Oh, Julien Kloetzer","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Rep- resentations, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2021] Sha Li, Heng Ji, and Jiawei Han. Document- level event argument extraction by conditional generation. In Proceedings of the 2021 Conference of the North Ameri- can Chapter of the Association for Computational Linguis- tics: Human Language Technologies, pages 894-908, On- line, June 2021. Association for Computational Linguis- tics.","title":"Documentlevel event argument extraction by conditional generation","authors":"Sha Li, Ji Heng, Jiawei Han","year":"2021","venue":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense ob- ject detection. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2999-3007, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2020] Jian Liu, Yubo Chen, and Jun Zhao. Knowledge enhanced event causality identification with mention masking generalizations. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 3608-3614. International Joint Conferences on Ar- tificial Intelligence Organization, 7 2020. Main track.","title":"Knowledge enhanced event causality identification with mention masking generalizations","authors":"Jian Liu, Yubo Chen, Jun Zhao","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20","doi":null,"url":null,"internal_url":"/proceedings/2020/499"},{"position":17,"raw":"[Loshchilov and Hutter, 2019] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Inter- national Conference on Learning Representations, 2019.","title":"Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization","authors":"Hutter Loshchilov","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mirza, 2014] Paramita Mirza. Extracting temporal and causal relations between events. In Proceedings of the ACL 2014 Student Research Workshop, pages 10-17, Bal- timore, Maryland, USA, June 2014. Association for Com- putational Linguistics.","title":"Extracting temporal and causal relations between events","authors":"Paramita Mirza, Mirza","year":"2014","venue":"Proceedings of the ACL 2014 Student Research Workshop","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ning et al., 2018] Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. Joint reasoning for temporal and causal re- lations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2278-2288, Melbourne, Australia, July 2018. Association for Computational Linguistics.","title":"Joint reasoning for temporal and causal relations","authors":"Qiang Ning, Zhili Feng, Hao Wu, Dan Roth","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Oh et al., 2016] Jong-Hoon Oh, Kentaro Torisawa, Chikara Hashimoto, Ryu Iida, Masahiro Tanaka, and Julien Kloet- zer. A semi-supervised learning approach to why-question answering. Proceedings of the AAAI Conference on Artifi- cial Intelligence, 30(1), Mar. 2016.","title":"A semi-supervised learning approach to why-question answering","authors":"Jong-Hoon Oh, Kentaro Torisawa, Chikara Hashimoto, Ryu Iida, Masahiro Tanaka, Julien Kloetzer","year":"2016","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Phu and Nguyen, 2021] Minh Tran Phu and Thien Huu Nguyen. Graph convolutional networks for event causality identification with rich document-level structures. In Pro- ceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3480-3490, 2021.","title":"Graph convolutional networks for event causality identification with rich document-level structures","authors":"Nguyen Phu, Minh, Tran Phu, Thien Huu Nguyen","year":"2021","venue":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Riaz and Girju, 2014a] Mehwish Riaz and Roxana Girju. In-depth exploitation of noun and verb semantics to iden- tify causation in verb-noun pairs. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 161-170, Philadelphia, PA, U.S.A., June 2014. Association for Computational Linguistics.","title":"In-depth exploitation of noun and verb semantics to identify causation in verb-noun pairs","authors":"Girju Riaz, Mehwish Riaz, Roxana Girju","year":"2014","venue":"Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Riaz and Girju, 2014b] Mehwish Riaz and Roxana Girju. Recognizing causality in verb-noun pairs via noun and verb semantics. In Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL), pages 48-57, Gothenburg, Sweden, April 2014. Association for Computational Linguistics.","title":"Recognizing causality in verb-noun pairs via noun and verb semantics","authors":"Girju Riaz, Mehwish Riaz, Roxana Girju","year":"2014","venue":"Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, vol- ume 30.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł Ukasz Kaiser, Illia Polosukhin","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2022] Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, and Jie Zhou. MAVEN- ERE: A unified large-scale dataset for event coreference, temporal, causal, and subevent relation extraction. CoRR, abs/2211.07342, 2022.","title":"MAVEN-ERE: A unified large-scale dataset for event coreference, temporal, causal, and subevent relation extraction","authors":"Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yao et al., 2019] Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, and Maosong Sun. DocRED: A large-scale document-level relation extraction dataset. In Proceedings of the 57th Annual Meeting of the Association for Compu- tational Linguistics, pages 764-777, Florence, Italy, July 2019. Association for Computational Linguistics.","title":"DocRED: A large-scale document-level relation extraction dataset","authors":"Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhong and Chen, 2021] Zexuan Zhong and Danqi Chen. A frustratingly easy approach for entity and relation extrac- tion. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 50- 61, Online, June 2021. Association for Computational Lin- guistics.","title":"A frustratingly easy approach for entity and relation extraction","authors":"Chen ; Zhong, Zexuan Zhong, Danqi Chen","year":"2021","venue":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zuo et al., 2020] Xinyu Zuo, Yubo Chen, Kang Liu, and Jun Zhao. Knowdis: Knowledge enhanced data augmentation for event causality detection via distant supervision. In Proceedings of the 28th International Conference on Com- putational Linguistics, pages 1544-1550, 2020.","title":"Knowdis: Knowledge enhanced data augmentation for event causality detection via distant supervision","authors":"Xinyu Zuo, Yubo Chen, Kang Liu, Jun Zhao","year":"2020","venue":"Proceedings of the 28th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zuo et al., 2021a] Xinyu Zuo, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Weihua Peng, and Yuguang Chen. Improving event causality identification via self- supervised representation learning on external causal statement. In Findings of the Association for Computa- tional Linguistics: ACL-IJCNLP 2021, pages 2162-2172, 2021.","title":"Improving event causality identification via selfsupervised representation learning on external causal statement","authors":"Zuo","year":"2021","venue":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zuo et al., 2021b] Xinyu Zuo, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Weihua Peng, and Yuguang Chen. LearnDA: Learnable knowledge-guided data augmenta- tion for event causality identification. In Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages 3558-3571, Online, August 2021. Associ- ation for Computational Linguistics.","title":"LearnDA: Learnable knowledge-guided data augmentation for event causality identification","authors":"Zuo","year":"2021","venue":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}