{"type":"ScholarlyArticle","title":"CausalNET: Unveiling Causal Structures on Event Sequences by Topology-Informed Causal Attention","authors":[{"name":"Hua Zhu","slug":"hua-zhu","given":"Hua","family":"Zhu","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab , Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology , Wuhan , China"},{"name":"Hong Huang","slug":"hong-huang","given":"Hong","family":"Huang","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab , Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology , Wuhan , China"},{"name":"Kehan Yin","slug":"kehan-yin","given":"Kehan","family":"Yin","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab , Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology , Wuhan , China"},{"name":"Zejun Fan","slug":"zejun-fan","given":"Zejun","family":"Fan","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab , Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology , Wuhan , China"},{"name":"Hai Jin","slug":"hai-jin","given":"Hai","family":"Jin","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab , Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology , Wuhan , China; CIFAR AI Chair , Canada"},{"name":"Bang Liu","slug":"bang-liu","given":"Bang","family":"Liu","affiliation":"DIRO , Université de Montréal & Mila & Canada"}],"abstract":"Causal discovery on event sequences holds a pivotal significance across domains such as healthcare, finance, and industrial systems. The crux of this endeavor lies in unraveling causal structures among event types, typically portrayed as directed acyclic graphs (DAGs). Nonetheless, prevailing methodologies often grapple with untenable assumptions and intricate optimization hurdles. To address these challenges, we present a novel model named CausalNET. At the heart of CausalNET is a special prediction module based on the Transformer architecture, which prognosticates forthcoming events by leveraging historical occurrences, with its predictive prowess amplified by a trainable causal graph engineered to fathom causal relationships among event types. Further, to augment the predictive paradigm, we devise a causal decay matrix to encapsulate the reciprocal influence of events upon each other within the topological network. During training, we alternatively refine the prediction module and fine-tune the causal graph. Comprehensive evaluation on a spectrum of real-world and synthetic datasets underscores the superior performance and scalability of CausalNET, which marks a promising step forward in the realm of causal discovery. Code and Appendix are available at https://github.com/CGCL-codes/CausalNET.","doi":"10.24963/ijcai.2024/790","url":"https://ijcai.org/proceedings/2024/790","pdf":"https://ijcai.org/proceedings/2024/0790.pdf","markdown":"https://ijcai.org/proceedings/2024/790.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/790","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Uncertainty in AI","track":"Main Track","number":790,"pages":{"first":7144,"last":7152},"keywords":["Uncertainty in AI: UAI: Causality, structural causal models and causal inference","Machine Learning: ML: Causality"],"videos":[],"references":[{"position":1,"raw":"[Achab et al., 2017] Massil Achab, Emmanuel Bacry, Stéphane Gaıffas, Iacopo Mastromatteo, and Jean- Franc ¸ois Muzy. Uncovering causality from multivariate hawkes integrated cumulants. In Proceedings of the 34th International Conference on Machine Learning, pages 1-10. PMLR, 2017.","title":"Uncovering causality from multivariate hawkes integrated cumulants","authors":"Achab","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bhattacharjya et al., 2018] Debarun Bhattacharjya, Dhar- mashankar Subramanian, and Tian Gao. Proximal graphi- cal event models. In Proceedings of the 32nd Annual Con- ference on Neural Information Processing Systems, pages 8136-8145, 2018.","title":"Proximal graphical event models","authors":"Bhattacharjya","year":"2018","venue":"Proceedings of the 32nd Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhattacharjya et al., 2022] Debarun Bhattacharjya, Karthikeyan Shanmugam, Tian Gao, and Dharmashankar Subramanian. Process independence testing in proximal graphical event models. In Proceedings of the 3rd Conference on Causal Learning and Reasoning, pages 144-161. PMLR, 2022.","title":"Process independence testing in proximal graphical event models","authors":"Bhattacharjya","year":"2022","venue":"Proceedings of the 3rd Conference on Causal Learning and Reasoning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2022] Ruichu Cai, Siyu Wu, Jie Qiao, Zhifeng Hao, Keli Zhang, and Xi Zhang. Thps: Topological hawkes processes for learning causal structure on event sequences. IEEE Transactions on Neural Networks and Learning Systems, 2022.","title":"Thps: Topological hawkes processes for learning causal structure on event sequences","authors":"Ruichu Cai, Siyu Wu, Jie Qiao, Zhifeng Hao, Keli Zhang, Xi Zhang","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cheng et al., 2022] Yuxiao Cheng, Runzhao Yang, Tingx- iong Xiao, Zongren Li, Jinli Suo, Kunlun He, and Qiong- hai Dai. Cuts: Neural causal discovery from irregular time-series data. In Proceedings of the 10th International Conference on Learning Representations, 2022.","title":"Cuts: Neural causal discovery from irregular time-series data","authors":"Cheng","year":"2022","venue":"Proceedings of the 10th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chickering, 1996] David Maxwell Chickering. Learning bayesian networks is np-complete. Learning from data: Artificial intelligence and statistics V, pages 121-130, 1996.","title":"Learning bayesian networks is np-complete","authors":"David Maxwell, Chickering","year":"1996","venue":"Learning from data: Artificial intelligence and statistics V","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deleu et al., 2022] Tristan Deleu, António Góis, Chris Emezue, Mansi Rankawat, Simon Lacoste-Julien, Stefan Bauer, and Yoshua Bengio. Bayesian structure learning with generative flow networks. In Uncertainty in Artificial Intelligence, pages 518-528. PMLR, 2022.","title":"Bayesian structure learning with generative flow networks","authors":"Tristan Deleu, António Góis, Chris Emezue, Mansi Rankawat, Simon Lacoste-Julien, Stefan Bauer, Yoshua Bengio","year":"2022","venue":"Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gong et al., 2023] Chang Gong, Di Yao, Chuzhe Zhang, Wenbin Li, Jingping Bi, Lun Du, and Jin Wang. Causal discovery from temporal data. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5803-5804, 2023.","title":"Causal discovery from temporal data","authors":"Chang Gong, Di Yao, Chuzhe Zhang, Wenbin Li, Jingping Bi, Lun Du, Jin Wang","year":"2023","venue":"Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hawkes, 1971] Alan G. Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.","title":"Spectra of some self-exciting and mutually exciting point processes","authors":"Alan G Hawkes","year":"1971","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2021] Yue He, Peng Cui, Zheyan Shen, Renzhe Xu, Furui Liu, and Yong Jiang. Daring: Differentiable causal discovery with residual independence. In Proceed- ings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 596-605, 2021.","title":"Daring: Differentiable causal discovery with residual independence","authors":"Yue He, Peng Cui, Zheyan Shen, Renzhe Xu, Furui Liu, Yong Jiang","year":"2021","venue":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Idé et al., 2021] Tsuyoshi Idé, Georgios Kollias, Dzung Phan, and Naoki Abe. Cardinality-regularized hawkes- granger model. In Proceedings of the Thirty-fifth Annual Conference on Neural Information Processing Systems, pages 2682-2694, 2021.","title":"Cardinality-regularized hawkesgranger model","authors":"Tsuyoshi Idé, Georgios Kollias, Dzung Phan, Naoki Abe","year":"2021","venue":"Proceedings of the Thirty-fifth Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jang et al., 2016] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In Proceedings of the 4th International Conference on Learn- ing Representations, 2016.","title":"Categorical reparameterization with gumbel-softmax","authors":"Eric Jang, Shixiang Gu, Ben Poole","year":"2016","venue":"Proceedings of the 4th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2022] Wenqian Li, Yinchuan Li, Shengyu Zhu, Yunfeng Shao, Jianye Hao, and Yan Pang. Gflowcausal: Generative flow networks for causal discovery. arXiv preprint arXiv:2210.08185, 2022.","title":"Gflowcausal: Generative flow networks for causal discovery","authors":"Li","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2024] Yuequn Liu, Ruichu Cai, Wei Chen, Jie Qiao, Yuguang Yan, Zijian Li, Keli Zhang, and Zhifeng Hao. Tnpar: Topological neural poisson auto-regressive model for learning granger causal structure from event se- quences. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, pages 20491-20499, 2024.","title":"Tnpar: Topological neural poisson auto-regressive model for learning granger causal structure from event sequences","authors":"Liu","year":"2024","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Luo et al., 2015] Dixin Luo, Hongteng Xu, Yi Zhen, Xia Ning, Hongyuan Zha, Xiaokang Yang, and Wenjun Zhang. Multi-task multi-dimensional hawkes processes for mod- eling event sequences. In Proceedings of the 24th Interna- tional Conference on Artificial Intelligence, pages 3685- 3691, 2015.","title":"Multi-task multi-dimensional hawkes processes for modeling event sequences","authors":"Luo","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/518"},{"position":16,"raw":"[Ng et al., 2020] Ignavier Ng, AmirEmad Ghassami, and Kun Zhang. On the role of sparsity and dag constraints for learning linear dags. In Proceedings of the Thirty-fourth Annual Conference on Neural Information Processing Sys- tems, pages 17943-17954, 2020.","title":"On the role of sparsity and dag constraints for learning linear dags","authors":"Ng","year":"2020","venue":"Proceedings of the Thirty-fourth Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ng et al., 2022] Ignavier Ng, Shengyu Zhu, Zhuangyan Fang, Haoyang Li, Zhitang Chen, and Jun Wang. Masked gradient-based causal structure learning. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pages 424-432. SIAM, 2022.","title":"Masked gradient-based causal structure learning","authors":"Ng","year":"2022","venue":"Proceedings of the 2022 SIAM International Conference on Data Mining (SDM)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qiao et al., 2023] Jie Qiao, Ruichu Cai, Siyu Wu, Yu Xiang, Keli Zhang, and Zhifeng Hao. Structural hawkes processes for learning causal structure from discrete-time event se- quences. In Proceedings of the 32nd International Joint Conference on Artificial Intelligence, pages 5702-5710, 2023.","title":"Structural hawkes processes for learning causal structure from discrete-time event sequences","authors":"Jie Qiao, Ruichu Cai, Siyu Wu, Yu Xiang, Keli Zhang, Zhifeng Hao","year":"2023","venue":"Proceedings of the 32nd International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2023/633"},{"position":19,"raw":"[Rasmussen, 2018] Jakob Gulddahl Rasmussen. Lecture notes: Temporal point processes and the conditional in- tensity function. arXiv preprint arXiv:1806.00221, 2018.","title":"Jakob Gulddahl Rasmussen. Lecture notes: Temporal point processes and the conditional intensity function","authors":"Rasmussen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rinne, 2008] Horst Rinne. The Weibull distribution: a handbook. CRC press, 2008.","title":"The Weibull distribution: a handbook","authors":"Horst Rinne","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Runge et al., 2019] Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. Detect- ing and quantifying causal associations in large nonlinear time series datasets. Science Advances, 5(11):eaau4996, 2019.","title":"Detecting and quantifying causal associations in large nonlinear time series datasets","authors":"Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, Dino Sejdinovic","year":"2019","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shimizu et al., 2006] Shohei Shimizu, Patrik O. Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.","title":"A linear non-gaussian acyclic model for causal discovery","authors":"Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, Michael Jordan","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shimizu et al., 2011] Shohei Shimizu, Takanori Inazumi, Yasuhiro Sogawa, Aapo Hyvarinen, Yoshinobu Kawahara, Takashi Washio, Patrik O. Hoyer, Kenneth Bollen, and Pa- trik Hoyer. Directlingam: A direct method for learning a linear non-gaussian structural equation model. Journal of Machine Learning Research, 12(Apr):1225-1248, 2011.","title":"Directlingam: A direct method for learning a linear non-gaussian structural equation model","authors":"Shohei Shimizu, Takanori Inazumi, Yasuhiro Sogawa, Aapo Hyvarinen, Yoshinobu Kawahara, Takashi Washio, Patrik O Hoyer, Kenneth Bollen, Patrik Hoyer","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Spirtes and Glymour, 1991] Peter Spirtes and Clark Gly- mour. An algorithm for fast recovery of sparse causal graphs. Social Science Computer Review, 9(1):62-72, 1991.","title":"Peter Spirtes and Clark Glymour. An algorithm for fast recovery of sparse causal graphs","authors":"Glymour Spirtes","year":"1991","venue":"Social Science Computer Review","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems, pages 5998- 6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proceedings of the 31st Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In Proceedings of the Sixth International Conference on Learning Repre- sentations, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"Proceedings of the Sixth International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2016a] Hongteng Xu, Mehrdad Farajtabar, and Hongyuan Zha. Learning granger causality for hawkes processes. In Proceedings of the 33rd International Con- ference on Machine Learning, pages 1717-1726. PMLR, 2016.","title":"Learning granger causality for hawkes processes","authors":"Xu","year":"2016","venue":"Proceedings of the 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2016b] Hongteng Xu, Mehrdad Farajtabar, and Hongyuan Zha. Learning granger causality for hawkes processes. In Proceedings of the 33rd International Con- ference on Machine Learning, pages 1717-1726. PMLR, 2016.","title":"Learning granger causality for hawkes processes","authors":"Xu","year":"2016","venue":"Proceedings of the 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2020a] Qiang Zhang, Aldo Lipani, Omer Kir- nap, and Emine Yilmaz. Self-attentive hawkes process. In Proceedings of the 37th International Conference on Ma- chine Learning, pages 11183-11193. PMLR, 2020.","title":"Self-attentive hawkes process","authors":"Zhang","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2020b] Wei Zhang, Thomas Panum, Somesh Jha, Prasad Chalasani, and David Page. Cause: Learning granger causality from event sequences using attribution methods. In Proceedings of the 37th International Confer- ence on Machine Learning, pages 11235-11245. PMLR, 2020.","title":"Cause: Learning granger causality from event sequences using attribution methods","authors":"Zhang","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2021] Keli Zhang, Shengyu Zhu, Marcus Ka- lander, Ignavier Ng, Junjian Ye, Zhitang Chen, and Lujia Pan. gcastle: A python toolbox for causal discovery. arXiv preprint arXiv:2111.15155, 2021.","title":"gcastle: A python toolbox for causal discovery","authors":"Keli Zhang, Shengyu Zhu, Marcus Kalander, Ignavier Ng, Junjian Ye, Zhitang Chen, Lujia Pan","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zheng et al., 2018] Xun Zheng, Bryon Aragam, Pradeep K. Ravikumar, and Eric P. Xing. Dags with no tears: Contin- uous optimization for structure learning. In Proceedings of the 32nd Annual Conference on Neural Information Pro- cessing Systems, pages 9472-9483, 2018.","title":"Dags with no tears: Continuous optimization for structure learning","authors":"Xun Zheng, Bryon Aragam, K Pradeep, Eric P Ravikumar, Xing","year":"2018","venue":"Proceedings of the 32nd Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhou et al., 2013] Ke Zhou, Hongyuan Zha, and Le Song. Learning social infectivity in sparse low-rank networks us- ing multi-dimensional hawkes processes. In Proceedings of the 16th International Conference on Artificial Intelli- gence and Statistics, pages 641-649. PMLR, 2013.","title":"Learning social infectivity in sparse low-rank networks using multi-dimensional hawkes processes","authors":"Ke Zhou, Hongyuan Zha, Le Song","year":"2013","venue":"Proceedings of the 16th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhu et al., 2019] Shengyu Zhu, Ignavier Ng, and Zhitang Chen. Causal discovery with reinforcement learning. In Proceedings of the Seventh International Conference on Learning Representations, 2019.","title":"Causal discovery with reinforcement learning","authors":"Zhu","year":"2019","venue":"Proceedings of the Seventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zuo et al., 2020] Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer hawkes pro- cess. In Proceedings of the 37th International Conference on Machine Learning, pages 11692-11702. PMLR, 2020.","title":"Transformer hawkes process","authors":"Zuo","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}