{"type":"ScholarlyArticle","title":"Not Only Pairwise Relationships: Fine-Grained Relational Modeling for Multivariate Time Series Forecasting","authors":[{"name":"Jinming Wu","slug":"jinming-wu","given":"Jinming","family":"Wu","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications {wjm 18 , qiqi8266 ,"},{"name":"Qi Qi","slug":"qi-qi","given":"Qi","family":"Qi","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications {wjm 18 , qiqi8266 ,"},{"name":"Jingyu Wang","slug":"jingyu-wang","given":"Jingyu","family":"Wang","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications {wjm 18 , qiqi8266 ,"},{"name":"Haifeng Sun","slug":"haifeng-sun","given":"Haifeng","family":"Sun","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications {wjm 18 , qiqi8266 ,"},{"name":"Zhikang Wu","slug":"zhikang-wu","given":"Zhikang","family":"Wu","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications {wjm 18 , qiqi8266 ,"},{"name":"Zirui Zhuang","slug":"zirui-zhuang","given":"Zirui","family":"Zhuang","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications {wjm 18 , qiqi8266 ,"},{"name":"Jianxin Liao","slug":"jianxin-liao","given":"Jianxin","family":"Liao","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications {wjm 18 , qiqi8266 ,"}],"abstract":"Recent graph-based methods achieve significant success in multivariate time series modeling and forecasting due to their ability to handle relationships among time series variables. However, only pairwise relationships are considered in most existing works. They ignore beyond-pairwise relationships and their potential categories in practical scenarios, which leads to incomprehensive relationship learning for multivariate time series forecasting. In this paper, we present ReMo, a Relational Modeling-based method, to promote fine-grained relational learning among multivariate time series data. Firstly, by treating time series variables and complex relationships as nodes and hyperedges, we extract multi-view hypergraphs from data to capture beyond-pairwise relationships. Secondly, a novel hypergraph message passing strategy is designed to characterize both nodes and hyperedges by inferring the potential categories of relationships and further distinguishing their impacts on time series variables. By integrating these two modules into the time series forecasting framework, ReMo effectively improves the performance of multivariate time series forecasting. The experimental results on seven commonly used datasets from different domains demonstrate the superiority of our model.","doi":"10.24963/ijcai.2023/491","url":"https://ijcai.org/proceedings/2023/491","pdf":"https://ijcai.org/proceedings/2023/0491.pdf","markdown":"https://ijcai.org/proceedings/2023/491.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/491","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":491,"pages":{"first":4416,"last":4423},"keywords":["Machine Learning: ML: Time series and data streams","Data Mining: DM: Mining graphs","Data Mining: DM: Mining spatial and/or temporal data"],"videos":[],"references":[{"position":1,"raw":"[Bai et al., 2020] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. In NeurIPS, 2020.","title":"Adaptive graph convolutional recurrent network for traffic forecasting","authors":"Lei Bai, Lina Yao, Can Li, Xianzhi Wang, Can Wang","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bai et al., 2021] Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110:107637, 2021.","title":"Hypergraph convolution and hypergraph attention","authors":"Song Bai, Feihu Zhang, Philip Hs Torr","year":"2021","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Binkowski et al., 2018] Mikolaj Binkowski, Gautier Marti, and Philippe Donnat. Autoregressive convolutional neural networks for asynchronous time series. In ICML, 2018.","title":"Autoregressive convolutional neural networks for asynchronous time series","authors":"Mikolaj Binkowski, Gautier Marti, Philippe Donnat","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2020] Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bix- iong Xu, Jing Bai, Jie Tong, and Qi Zhang. Spectral temporal graph neural network for multivariate time-series forecasting. In NeurIPS, 2020.","title":"Spectral temporal graph neural network for multivariate time-series forecasting","authors":"Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, Qi Zhang","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2021] Yuzhou Chen, Ignacio Segovia, and Yu- lia R Gel. Z-gcnets: time zigzags at graph convolutional networks for time series forecasting. In ICML, 2021.","title":"Z-gcnets: time zigzags at graph convolutional networks for time series forecasting","authors":"Yuzhou Chen, Ignacio Segovia, Yulia R Gel","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feng et al., 2019] Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural net- works. In AAAI, 2019.","title":"Hypergraph neural networks","authors":"Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, Yue Gao","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jang et al., 2017] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In ICLR, 2017.","title":"Categorical reparameterization with gumbel-softmax","authors":"Eric Jang, Shixiang Gu, Ben Poole","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jin et al., 2018] Bo Jin, Haoyu Yang, Leilei Sun, Chuanren Liu, Yue Qu, and Jianing Tong. A treatment engine by predicting next-period prescriptions. In KDD, 2018.","title":"A treatment engine by predicting next-period prescriptions","authors":"Bo Jin, Haoyu Yang, Leilei Sun, Chuanren Liu, Yue Qu, Jianing Tong","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kipf et al., 2018] Thomas N. Kipf, Ethan Fetaya, Kuan- Chieh Wang, Max Welling, and Richard S. Zemel. Neu- ral relational inference for interacting systems. In ICML, 2018.","title":"Neural relational inference for interacting systems","authors":"Thomas N Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, Richard S Zemel","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lai et al., 2018] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In SIGIR, 2018.","title":"Modeling long-and short-term temporal patterns with deep neural networks","authors":"Lai","year":"2018","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2018] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. In ICLR, 2018.","title":"Diffusion convolutional recurrent neural network: Data-driven traffic forecasting","authors":"Yaguang Li, Rose Yu, Cyrus Shahabi, Yan Liu","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lim and Zohren, 2021] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a survey. Phi- los. Trans. Royal Soc. A, 379:20200209, 2021.","title":"Time-series forecasting with deep learning: a survey","authors":"Bryan Lim, Stefan Zohren","year":"2021","venue":"Philos. Trans. Royal Soc. A","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Maddison et al., 2017] Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A con- tinuous relaxation of discrete random variables. In ICLR, 2017.","title":"The concrete distribution: A continuous relaxation of discrete random variables","authors":"Chris J Maddison, Andriy Mnih, Yee Whye Teh","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mudelsee, 2019] Manfred Mudelsee. Trend analysis of cli- mate time series: A review of methods. Earth-science re- views, 190:310-322, 2019.","title":"Trend analysis of climate time series: A review of methods","authors":"Manfred Mudelsee, Mudelsee","year":"2019","venue":"Earth-science reviews","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Roberts et al., 2013] Stephen Roberts, Michael Osborne, Mark Ebden, Steven Reece, Neale Gibson, and Suzanne Aigrain. Gaussian processes for time-series modelling. Philos. Trans. Royal Soc. A, 371:20110550, 2013.","title":"Gaussian processes for time-series modelling","authors":"Stephen Roberts, Michael Osborne, Mark Ebden, Steven Reece, Neale Gibson, Suzanne Aigrain","year":"2013","venue":"Philos. Trans. Royal Soc. A","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shih et al., 2019] Shun-Yao Shih, Fan-Keng Sun, and Hung-yi Lee. Temporal pattern attention for multivari- ate time series forecasting. Machine Learning, 108:1421- 1441, 2019.","title":"Temporal pattern attention for multivariate time series forecasting","authors":"Shun-Yao Shih, Fan-Keng Sun, Hung-Yi Lee","year":"2019","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2022] Fan-Yun Sun, Isaac Kauvar, Ruohan Zhang, Jiachen Li, Mykel Kochenderfer, Jiajun Wu, and Nick Haber. Interaction modeling with multiplex atten- tion. arXiv preprint arXiv:2208.10660, 2022.","title":"Interaction modeling with multiplex attention","authors":"Fan-Yun Sun, Isaac Kauvar, Ruohan Zhang, Jiachen Li, Mykel Kochenderfer, Jiajun Wu, Nick Haber","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wu et al., 2019] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In IJCAI, 2019.","title":"Graph wavenet for deep spatial-temporal graph modeling","authors":"Wu","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/264"},{"position":19,"raw":"[Wu et al., 2020] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Con- necting the dots: Multivariate time series forecasting with graph neural networks. In KDD, 2020.","title":"Connecting the dots: Multivariate time series forecasting with graph neural networks","authors":"Wu","year":"2020","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2022] Chenxin Xu, Maosen Li, Zhenyang Ni, Ya Zhang, and Siheng Chen. Groupnet: Multiscale hy- pergraph neural networks for trajectory prediction with re- lational reasoning. In CVPR, 2022.","title":"Groupnet: Multiscale hypergraph neural networks for trajectory prediction with relational reasoning","authors":"Xu","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yadati et al., 2019] Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha P. Talukdar. Hypergcn: A new method for training graph convolutional networks on hypergraphs. In NeurIPS, 2019.","title":"Hypergcn: A new method for training graph convolutional networks on hypergraphs","authors":"Yadati","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ye et al., 2022] Junchen Ye, Zihan Liu, Bowen Du, Leilei Sun, Weimiao Li, Yanjie Fu, and Hui Xiong. Learning the evolutionary and multi-scale graph structure for multivari- ate time series forecasting. In KDD, 2022.","title":"Learning the evolutionary and multi-scale graph structure for multivariate time series forecasting","authors":"Ye","year":"2022","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2018] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In IJCAI, 2018.","title":"Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting","authors":"Yu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/505"},{"position":24,"raw":"[Yu et al., 2022] Hongyuan Yu, Ting Li, Weichen Yu, Jian- guo Li, Yan Huang, Liang Wang, and Alex X. Liu. Regu- larized graph structure learning with semantic knowledge for multi-variates time-series forecasting. In IJCAI, 2022.","title":"Regularized graph structure learning with semantic knowledge for multi-variates time-series forecasting","authors":"Hongyuan Yu, Ting Li, Weichen Yu, Jianguo Li, Yan Huang, Liang Wang, Alex X Liu","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/328"},{"position":25,"raw":"[Zheng et al., 2020] Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. Gman: A graph multi-attention network for traffic prediction. In AAAI, 2020.","title":"Gman: A graph multi-attention network for traffic prediction","authors":"Zheng","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2006] Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clus- tering, classification, and embedding. In NeurIPS, 2006.","title":"Learning with hypergraphs: Clustering, classification, and embedding","authors":"Zhou","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}