{"type":"ScholarlyArticle","title":"Make Graph Neural Networks Great Again: A Generic Integration Paradigm of Topology-Free Patterns for Traffic Speed Prediction","authors":[{"name":"Yicheng Zhou","slug":"yicheng-zhou","given":"Yicheng","family":"Zhou","affiliation":"The State Key Laboratory of Internet of Things for Smart City , University of Macau , Macau; Department of Computer and Information Science , University of Macau , Macau"},{"name":"Pengfei Wang","slug":"pengfei-wang","given":"Pengfei","family":"Wang","affiliation":"Computer Network Information Center , Chinese Academy of Sciences , Beijing; University of Chinese Academy of Sciences , Chinese Academy of Sciences , Beijing"},{"name":"Hao Dong","slug":"hao-dong","given":"Hao","family":"Dong","affiliation":"Computer Network Information Center , Chinese Academy of Sciences , Beijing; University of Chinese Academy of Sciences , Chinese Academy of Sciences , Beijing"},{"name":"Denghui Zhang","slug":"denghui-zhang","given":"Denghui","family":"Zhang","affiliation":"School of Business , Stevens Institute of Technology , Hoboken"},{"name":"Dingqi Yang","slug":"dingqi-yang","given":"Dingqi","family":"Yang","affiliation":"The State Key Laboratory of Internet of Things for Smart City , University of Macau , Macau; Department of Computer and Information Science , University of Macau , Macau"},{"name":"Yanjie Fu","slug":"yanjie-fu","given":"Yanjie","family":"Fu","affiliation":"School of Computing and AI , Arizona State University , Tempe"},{"name":"Pengyang Wang","slug":"pengyang-wang","given":"Pengyang","family":"Wang","affiliation":"The State Key Laboratory of Internet of Things for Smart City , University of Macau , Macau; Department of Computer and Information Science , University of Macau , Macau"}],"abstract":"Urban traffic speed prediction aims to estimate the future traffic speed for improving urban transportation services. Enormous efforts have been made to exploit Graph Neural Networks (GNNs) for modeling spatial correlations and temporal dependencies of traffic speed evolving patterns, regularized by graph topology. While achieving promising results, current traffic speed prediction methods still suffer from ignoring topology-free patterns, which cannot be captured by GNNs. To tackle this challenge, we propose a generic model for enabling the current GNN-based methods to preserve topology-free patterns. Specifically, we first develop a Dual Cross-Scale Transformer (DCST) architecture, including a Spatial Transformer and a Temporal Transformer, to preserve the cross-scale topology-free patterns and associated dynamics, respectively. Then, to further integrate both topology-regularized/-free patterns, we propose a distillation-style learning framework, in which the existing GNN-based methods are considered as the teacher model, and the proposed DCST architecture is considered as the student model. The teacher model would inject the learned topology-regularized patterns into the student model for integrating topology-free patterns. The extensive experimental results demonstrated the effectiveness of our methods.","doi":"10.24963/ijcai.2024/288","url":"https://ijcai.org/proceedings/2024/288","pdf":"https://ijcai.org/proceedings/2024/0288.pdf","markdown":"https://ijcai.org/proceedings/2024/288.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/288","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":288,"pages":{"first":2607,"last":2615},"keywords":["Data Mining: DM: Mining spatial and/or temporal data"],"videos":[],"references":[{"position":1,"raw":"[Ahn et al., 2019] Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D Lawrence, and Zhenwen Dai. Varia- tional information distillation for knowledge transfer. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 9163-9171, 2019.","title":"Variational information distillation for knowledge transfer","authors":"Ahn","year":"2019","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bai et al., 2020] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in neural infor- mation processing systems, 33:17804-17815, 2020.","title":"Adaptive graph convolutional recurrent network for traffic forecasting","authors":"Lei Bai, Lina Yao, Can Li, Xianzhi Wang, Can Wang","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Benjamin et al., 2003] Michael A Benjamin, Robert A Rigby, and D Mikis Stasinopoulos. Generalized autore- gressive moving average models. Journal of the American Statistical association, pages 214-223, 2003.","title":"Generalized autoregressive moving average models","authors":"Robert A Michael A Benjamin, D Rigby, Stasinopoulos Mikis","year":"2003","venue":"Journal of the American Statistical association","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Box and Pierce, 1970] George EP Box and David A Pierce. Distribution of residual autocorrelations in autoregressive- integrated moving average time series models. Journal of the American statistical Association, pages 1509-1526, 1970.","title":"Distribution of residual autocorrelations in autoregressiveintegrated moving average time series models","authors":"E P George, David A Box, Pierce","year":"1970","venue":"Journal of the American statistical Association","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2023] Hao Dong, Zhiyuan Ning, Pengyang Wang, Ziyue Qiao, Pengfei Wang, Yuanchun Zhou, and Yanjie Fu. Adaptive path-memory network for temporal knowledge graph reasoning. arXiv preprint arXiv:2304.12604, 2023.","title":"Adaptive path-memory network for temporal knowledge graph reasoning","authors":"Hao Dong, Zhiyuan Ning, Pengyang Wang, Ziyue Qiao, Pengfei Wang, Yuanchun Zhou, Yanjie Fu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2023/232"},{"position":6,"raw":"[Dong et al., 2024] Hao Dong, Pengyang Wang, Meng Xiao, Zhiyuan Ning, Pengfei Wang, and Yuanchun Zhou. Tem- poral inductive path neural network for temporal knowl- edge graph reasoning. Artificial Intelligence, page 104085, 2024.","title":"Temporal inductive path neural network for temporal knowledge graph reasoning","authors":"Hao Dong, Pengyang Wang, Meng Xiao, Zhiyuan Ning, Pengfei Wang, Yuanchun Zhou","year":"2024","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guo et al., 2019] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 922-929, 2019.","title":"Attention based spatialtemporal graph convolutional networks for traffic flow forecasting","authors":"Guo","year":"2019","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2023] Xuanming Hu, Wei Fan, Kun Yi, Pengfei Wang, Yuanbo Xu, Yanjie Fu, and Pengyang Wang. Boost- ing urban prediction via addressing spatial-temporal dis- tribution shift. In 2023 IEEE International Conference on Data Mining (ICDM), pages 160-169. IEEE, 2023.","title":"Boosting urban prediction via addressing spatial-temporal distribution shift","authors":"Xuanming Hu, Wei Fan, Kun Yi, Pengfei Wang, Yuanbo Xu, Yanjie Fu, Pengyang Wang","year":"2023","venue":"IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2021] Renhe Jiang, Du Yin, Zhaonan Wang, Yizhuo Wang, Jiewen Deng, Hangchen Liu, Zekun Cai, Jinliang Deng, Xuan Song, and Ryosuke Shibasaki. Dl- traff: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings of the 30th ACM international conference on information & knowl- edge management, pages 4515-4525, 2021.","title":"Dltraff: Survey and benchmark of deep learning models for urban traffic prediction","authors":"Renhe Jiang, Du Yin, Zhaonan Wang, Yizhuo Wang, Jiewen Deng, Hangchen Liu, Zekun Cai, Jinliang Deng, Xuan Song, Ryosuke Shibasaki","year":"2021","venue":"Proceedings of the 30th ACM international conference on information & knowledge management","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lai et al., 2018] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & devel- opment in information retrieval, pages 95-104, 2018.","title":"Modeling long-and short-term temporal patterns with deep neural networks","authors":"Lai","year":"2018","venue":"The 41st international ACM SIGIR conference on research & development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2017] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926, 2017.","title":"Diffusion convolutional recurrent neural network: Data-driven traffic forecasting","authors":"Yaguang Li, Rose Yu, Cyrus Shahabi, Yan Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2021] Shizhan Liu, Hang Yu, Cong Liao, Jian- guo Li, Weiyao Lin, Alex X Liu, and Schahram Dust- dar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In Inter- national conference on learning representations, 2021.","title":"Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting","authors":"Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, Schahram Dustdar","year":"2021","venue":"International conference on learning representations","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2023] Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 4125-4129, 2023.","title":"Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting","authors":"Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, Xuan Song","year":"2023","venue":"Proceedings of the 32nd ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shao et al., 2022] Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecast- ing. In Proceedings of the 31st ACM International Con- ference on Information & Knowledge Management, pages 4454-4458, 2022.","title":"Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting","authors":"Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, Yongjun Xu","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2018] Pengyang Wang, Yanjie Fu, Jiawei Zhang, Pengfei Wang, Yu Zheng, and Charu Aggarwal. You are how you drive: Peer and temporal-aware repre- sentation learning for driving behavior analysis. In Pro- ceedings of the 24th ACM SIGKDD International Con- ference on Knowledge Discovery & Data Mining, pages 2457-2466, 2018.","title":"You are how you drive: Peer and temporal-aware representation learning for driving behavior analysis","authors":"Wang","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2019a] Pengyang Wang, Yanjie Fu, Hui Xiong, and Xiaolin Li. Adversarial substructured representation learning for mobile user profiling. In Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, pages 130-138, 2019.","title":"Adversarial substructured representation learning for mobile user profiling","authors":"Wang","year":"2019","venue":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2019b] Pengyang Wang, Xiaolin Li, Yu Zheng, Charu Aggarwal, and Yanjie Fu. Spatiotemporal represen- tation learning for driving behavior analysis: A joint per- spective of peer and temporal dependencies. IEEE Trans- actions on Knowledge and Data Engineering, 33(2):728- 741, 2019.","title":"Spatiotemporal representation learning for driving behavior analysis: A joint perspective of peer and temporal dependencies","authors":"Wang","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2020] Pengyang Wang, Kunpeng Liu, Lu Jiang, Xiaolin Li, and Yanjie Fu. Incremental mobile user profiling: Reinforcement learning with spatial knowledge graph for modeling event streams. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 853-861, 2020.","title":"Incremental mobile user profiling: Reinforcement learning with spatial knowledge graph for modeling event streams","authors":"Wang","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2023] Dongjie Wang, Pengyang Wang, Yanjie Fu, Kunpeng Liu, Hui Xiong, and Charles E Hughes. Re- inforced imitative graph learning for mobile user profiling. IEEE Transactions on Knowledge and Data Engineering, 2023.","title":"Reinforced imitative graph learning for mobile user profiling","authors":"Wang","year":"2023","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wei et al., 2018] Hua Wei, Guanjie Zheng, Huaxiu Yao, and Zhenhui Li. Intellilight: A reinforcement learning ap- proach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2496-2505, 2018.","title":"Intellilight: A reinforcement learning approach for intelligent traffic light control","authors":"Hua Wei, Guanjie Zheng, Huaxiu Yao, Zhenhui Li","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2019] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. arXiv preprint arXiv:1906.00121, 2019.","title":"Graph wavenet for deep spatial-temporal graph modeling","authors":"Wu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2019/264"},{"position":24,"raw":"[Wu et al., 2020] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Con- necting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discov- ery & data mining, pages 753-763, 2020.","title":"Connecting the dots: Multivariate time series forecasting with graph neural networks","authors":"Wu","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2017] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875, 2017.","title":"Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting","authors":"Yu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2018/505"},{"position":26,"raw":"[Zhang et al., 2020] Qi Zhang, Jianlong Chang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Spatio- temporal graph structure learning for traffic forecasting. In Proceedings of the AAAI conference on artificial intel- ligence, volume 34, pages 1177-1185, 2020.","title":"Spatiotemporal graph structure learning for traffic forecasting","authors":"Qi Zhang, Jianlong Chang, Gaofeng Meng, Shiming Xiang, Chunhong Pan","year":"2020","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2024] Zhaofan Zhang, Yanan Xiao, Lu Jiang, Dingqi Yang, Minghao Yin, and Pengyang Wang. Spatial- temporal interplay in human mobility: A hierarchical rein- forcement learning approach with hypergraph representa- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 9396-9404, 2024.","title":"Spatialtemporal interplay in human mobility: A hierarchical reinforcement learning approach with hypergraph representation","authors":"Zhang","year":"2024","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao et al., 2019] Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. T-gcn: A temporal graph convolutional network for traffic prediction. IEEE transactions on intelligent transportation systems, 21(9):3848-3858, 2019.","title":"T-gcn: A temporal graph convolutional network for traffic prediction","authors":"Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, Haifeng Li","year":"2019","venue":"IEEE transactions on intelligent transportation systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zheng et al., 2020] Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. Gman: A graph multi- attention network for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1234-1241, 2020.","title":"Gman: A graph multiattention network for traffic prediction","authors":"Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, Jianzhong Qi","year":"2020","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}