{"type":"ScholarlyArticle","title":"Regularized Graph Structure Learning with Semantic Knowledge for Multi-variates Time-Series Forecasting","authors":[{"name":"Hongyuan Yu","slug":"hongyuan-yu","given":"Hongyuan","family":"Yu","affiliation":"AI School , University of Chinese Academy of Sciences"},{"name":"Ting Li","slug":"ting-li","given":"Ting","family":"Li","affiliation":"AI School , University of Chinese Academy of Sciences"},{"name":"Weichen Yu","slug":"weichen-yu","given":"Weichen","family":"Yu","affiliation":"AI School , University of Chinese Academy of Sciences"},{"name":"Jianguo Li","slug":"jianguo-li","given":"Jianguo","family":"Li","affiliation":"AI School , University of Chinese Academy of Sciences"},{"name":"Yan Huang","slug":"yan-huang","given":"Yan","family":"Huang","affiliation":"CRIPAC , NLPR , Institute of Automation , Chinese Academy of Sciences , China"},{"name":"Liang Wang","slug":"liang-wang","given":"Liang","family":"Wang","affiliation":"CRIPAC , NLPR , Institute of Automation , Chinese Academy of Sciences , China"},{"name":"Alex Liu","slug":"alex-liu","given":"Alex","family":"Liu","affiliation":"CRIPAC , NLPR , Institute of Automation , Chinese Academy of Sciences , China"}],"abstract":"Multivariate time-series forecasting is a critical task for many applications, and graph time-series network is widely studied due to its capability to capture the spatial-temporal correlation simultaneously. However, most existing works focus more on learning with the explicit prior graph structure, while ignoring potential information from the implicit graph structure, yielding incomplete structure modeling. Some recent works attempts to learn the intrinsic or implicit graph structure directly, while lacking a way to combine explicit prior structure with implicit structure together. In this paper, we propose Regularized Graph Structure Learning (RGSL) model to incorporate both explicit prior structure and implicit structure together, and learn the forecasting deep networks along with the graph structure. RGSL consists of two innovative modules. First, we derive an implicit dense similarity matrix through node embedding, and learn the sparse graph structure using the Regularized Graph Generation (RGG) based on the Gumbel Softmax trick. Second, we propose a Laplacian Matrix Mixed-up Module (LM3) to fuse the explicit graph and implicit graph together. We conduct experiments on three real-word datasets. Results show that the proposed RGSL model outperforms existing graph forecasting algorithms with a notable margin, while learning meaningful graph structure simultaneously. Our code and models are made publicly available at https://github.com/alipay/RGSL.git.","doi":"10.24963/ijcai.2022/328","url":"https://ijcai.org/proceedings/2022/328","pdf":"https://ijcai.org/proceedings/2022/0328.pdf","markdown":"https://ijcai.org/proceedings/2022/328.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/328","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":328,"pages":{"first":2362,"last":2368},"keywords":["Machine Learning: Time-series; Data Streams","Data Mining: Mining Spatial and/or Temporal Data","Data Mining: Mining Graphs"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986820","length_sec":56,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984963","length_sec":203,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bai et al., 2020] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. NeurIPS, 33, 2020.","title":"Adaptive graph convolutional recurrent network for traffic forecasting","authors":"Lei Bai, Lina Yao, Can Li, Xianzhi Wang, Can Wang","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2020] Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, et al. Spectral temporal graph neural network for multivariate time-series forecasting. NeurIPS, 33, 2020.","title":"Spectral temporal graph neural network for multivariate time-series forecasting","authors":"Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019] Cen Chen, Kenli Li, Sin G Teo, Xiaofeng Zou, Kang Wang, Jie Wang, and Zeng Zeng. Gated residual recurrent graph neural networks for traffic prediction. In AAAI, volume 33, pages 485-492, 2019.","title":"Gated residual recurrent graph neural networks for traffic prediction","authors":"Cen Chen, Kenli Li, G Sin, Xiaofeng Teo, Kang Zou, Jie Wang, Zeng Wang, Zeng","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Xu Chen, Yuanxing Zhang, Lun Du, Zheng Fang, Yi Ren, Kaigui Bian, and Kunqing Xie. Tssrgcn: Tem- poral spectral spatial retrieval graph convolutional network for traffic flow forecasting. In ICDM. IEEE, 2020.","title":"Tssrgcn: Temporal spectral spatial retrieval graph convolutional network for traffic flow forecasting","authors":"Yuanxing Xu Chen, Lun Zhang, Zheng Du, Yi Fang, Kaigui Ren, Kunqing Bian, Xie","year":"2020","venue":"ICDM. IEEE","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2021a] Feilong Chen, Xiuyi Chen, Fandong Meng, Peng Li, and Jie Zhou. Gog: Relation-aware graph-over-graph network for visual dialog. arXiv preprint arXiv:2109.08475, 2021.","title":"Gog: Relation-aware graph-over-graph network for visual dialog","authors":"Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2021b] Yuzhou Chen, Ignacio Segovia-Dominguez, and Yulia R Gel. Z-gcnets: Time zigzags at graph convolutional networks for time series forecasting. ICML, 2021.","title":"Z-gcnets: Time zigzags at graph convolutional networks for time series forecasting","authors":"Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2021] Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W Tsang. St-norm: Spatial and temporal nor- malization for multi-variate time series forecasting. In 27th ACM SIGKDD, pages 269-278, 2021.","title":"St-norm: Spatial and temporal normalization for multi-variate time series forecasting","authors":"Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, Ivor W Tsang","year":"2021","venue":"27th ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Drucker et al., 1997] Harris Drucker, Chris JC Burges, Linda Kaufman, Alex Smola, Vladimir Vapnik, et al. Support vector regression machines. NIPS, 9:155-161, 1997.","title":"Support vector regression machines","authors":"Harris Drucker, J C Chris, Linda Burges, Alex Kaufman, Vladimir Smola, Vapnik","year":"1997","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo et al., 2019] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In AAAI, vol- ume 33, pages 922-929, 2019.","title":"Attention based spatial-temporal graph convolutional networks for traffic flow forecasting","authors":"Guo","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2020] Kan Guo, Yongli Hu, Zhen Qian, Hao Liu, Ke Zhang, Yanfeng Sun, Junbin Gao, and Baocai Yin. Optimized graph convolution recurrent neural network for traffic prediction. IEEE T-ITS, 2020.","title":"Optimized graph convolution recurrent neural network for traffic prediction","authors":"Kan Guo, Yongli Hu, Zhen Qian, Hao Liu, Ke Zhang, Yanfeng Sun, Junbin Gao, Baocai Yin","year":"2020","venue":"IEEE T-ITS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2019] Siteng Huang, Donglin Wang, Xuehan Wu, and Ao Tang. Dsanet: Dual self-attention network for multivari- ate time series forecasting. In CDKP, pages 2129-2132, 2019.","title":"Dsanet: Dual self-attention network for multivariate time series forecasting","authors":"Huang","year":"2019","venue":"CDKP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jin et al., 2020] Guangyin Jin, Zhexu Xi, Hengyu Sha, Yanghe Feng, and Jincai Huang. Deep multi-view spatiotemporal vir- tual graph neural network for significant citywide ride-hailing demand prediction. arXiv preprint arXiv:2007.15189, 2020.","title":"Deep multi-view spatiotemporal virtual graph neural network for significant citywide ride-hailing demand prediction","authors":"Jin","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee et al., 2019] Doyup Lee, Suehun Jung, Yeongjae Cheon, Dongil Kim, and Seungil You. Demand forecasting from spa- tiotemporal data with graph networks and temporal-guided em- bedding. arXiv preprint arXiv:1905.10709, 2019.","title":"Demand forecasting from spatiotemporal data with graph networks and temporal-guided embedding","authors":"Lee","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Moura, 2020] Yang Li and José MF Moura. Forecaster: A graph transformer for forecasting spatial and time-dependent data. In ECAI, 2020.","title":"Forecaster: A graph transformer for forecasting spatial and time-dependent data","authors":"Yang Li, José Mf Moura","year":"2020","venue":"ECAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2018] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In ICLR, 2018.","title":"Diffusion convolutional recurrent neural network: Data-driven traffic forecasting","authors":"Yaguang Li, Rose Yu, Cyrus Shahabi, Yan Liu","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lu et al., 2021] Ping Lu, Wenjia Bai, Daniel Rueckert, and J.Alison Noble. Dynamic spatio-temporal graph convolutional networks for cardiac motion analysis. In ISBI, pages 122-125, 2021.","title":"Dynamic spatio-temporal graph convolutional networks for cardiac motion analysis","authors":"Ping Lu, Wenjia Bai, Daniel Rueckert, J, Alison Noble","year":"2021","venue":"ISBI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lv et al., 2020] Mingqi Lv, Zhaoxiong Hong, Ling Chen, Tieming Chen, Tiantian Zhu, and Shouling Ji. Temporal multi-graph con- volutional network for traffic flow prediction. IEEE T-ITS, 2020.","title":"Temporal multi-graph convolutional network for traffic flow prediction","authors":"Lv","year":"2020","venue":"IEEE T-ITS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pan et al., 2020] Zheyi Pan, Wentao Zhang, Yuxuan Liang, Weinan Zhang, Yong Yu, Junbo Zhang, and Yu Zheng. Spatio- temporal meta learning for urban traffic prediction. TKDE, 2020.","title":"Spatiotemporal meta learning for urban traffic prediction","authors":"Pan","year":"2020","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shang et al., 2021] Chao Shang, Jie Chen, and Jinbo Bi. Discrete graph structure learning for forecasting multiple time series. In ICLR, 2021.","title":"Discrete graph structure learning for forecasting multiple time series","authors":"Chao Shang, Jie Chen, Jinbo Bi","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Song et al., 2020a] Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. Spatial-temporal sychronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In AAAI, 2020.","title":"Spatial-temporal sychronous graph convolutional networks: A new framework for spatial-temporal network data forecasting","authors":"Song","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2020b] Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In AAAI, volume 34, 2020.","title":"Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting","authors":"Song","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ting, 2021] Ta Jiun Ting. Machine Learning Models for Traffic Flow Prediction. PhD thesis, 2021.","title":"Machine Learning Models for Traffic Flow Prediction","authors":"Ta Jiun, Ting","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wei et al., 2019] Long Wei, Zhengxu Yu, Zhongming Jin, Liang Xie, Jianqiang Huang, Deng Cai, Xiaofei He, and Xian-Sheng Hua. Dual graph for traffic forecasting. IEEE Access, 2019.","title":"Dual graph for traffic forecasting","authors":"Long Wei, Zhengxu Yu, Zhongming Jin, Liang Xie, Jianqiang Huang, Deng Cai, Xiaofei He, Xian-Sheng Hua","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Williams and Hoel, 2003] Billy M Williams and Lester A Hoel. Modeling and forecasting vehicular traffic flow as a seasonal arima process: Theoretical basis and empirical results. Journal of transportation engineering, 129(6):664-672, 2003.","title":"Modeling and forecasting vehicular traffic flow as a seasonal arima process: Theoretical basis and empirical results","authors":"Hoel ; ] Williams, M Billy, Lester A Williams, Hoel","year":"2003","venue":"Journal of transportation engineering","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wu et al., 2020] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A compre- hensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020.","title":"A comprehensive survey on graph neural networks","authors":"Wu","year":"2020","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yin et al., 2021] Xueyan Yin, Genze Wu, Jinze Wei, Yanming Shen, Heng Qi, and Baocai Yin. Deep learning on traffic pre- diction: Methods, analysis and future directions. IEEE T-ITS, 2021.","title":"Deep learning on traffic prediction: Methods, analysis and future directions","authors":"Xueyan Yin, Genze Wu, Jinze Wei, Yanming Shen, Heng Qi, Baocai Yin","year":"2021","venue":"IEEE T-ITS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yu et al., 2018] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio- temporal graph convolutional networks: a deep learning frame- work for traffic forecasting. In IJCAI, pages 3634-3640, 2018.","title":"Spatiotemporal graph convolutional networks: a deep learning framework for traffic forecasting","authors":"Yu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2020a] Duzhen Zhang, Xiuyi Chen, Shuang Xu, and Bo Xu. Knowledge aware emotion recognition in textual conver- sations via multi-task incremental transformer. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4429-4440, 2020.","title":"Knowledge aware emotion recognition in textual conversations via multi-task incremental transformer","authors":"Zhang","year":"2020","venue":"Proceedings of the 28th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2020b] Xiyue Zhang, Chao Huang, Yong Xu, and Lianghao Xia. Spatial-temporal convolutional graph attention networks for citywide traffic flow forecasting. In CIKM, pages 1853-1862, 2020.","title":"Spatial-temporal convolutional graph attention networks for citywide traffic flow forecasting","authors":"Zhang","year":"2020","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zivot and Wang, 2006] Eric Zivot and Jiahui Wang. Vector autore- gressive models for multivariate time series. Modeling Financial Time Series with S-Plus®, pages 385-429, 2006.","title":"Vector autoregressive models for multivariate time series","authors":"Wang ; Zivot, Eric Zivot, Jiahui Wang","year":"2006","venue":"Modeling Financial Time Series with S-Plus®","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}