{"type":"ScholarlyArticle","title":"Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecasting","authors":[{"name":"Haotian Gao","slug":"haotian-gao","given":"Haotian","family":"Gao","affiliation":"The University of Tokyo; Southern University of Science and Technology"},{"name":"Renhe Jiang","slug":"renhe-jiang","given":"Renhe","family":"Jiang","affiliation":"The University of Tokyo"},{"name":"Zheng Dong","slug":"zheng-dong","given":"Zheng","family":"Dong","affiliation":"Southern University of Science and Technology"},{"name":"Jinliang Deng","slug":"jinliang-deng","given":"Jinliang","family":"Deng","affiliation":"University of Technology Sydney"},{"name":"Yuxin Ma","slug":"yuxin-ma","given":"Yuxin","family":"Ma","affiliation":"Southern University of Science and Technology"},{"name":"Xuan Song","slug":"xuan-song","given":"Xuan","family":"Song","affiliation":"Southern University of Science and Technology"}],"abstract":"Spatiotemporal forecasting techniques are significant for various domains such as transportation, energy, and weather. Accurate prediction of spatiotemporal series remains challenging due to the complex spatiotemporal heterogeneity. In particular, current end-to-end models are limited by input length and thus often fall into spatiotemporal mirage, i.e., similar input time series followed by dissimilar future values and vice versa. To address these problems, we propose a novel self-supervised pre-training framework Spatial-Temporal-Decoupled Masked Pre-training (STD-MAE) that employs two decoupled masked autoencoders to reconstruct spatiotemporal series along the spatial and temporal dimensions. Rich-context representations learned through such reconstruction could be seamlessly integrated by downstream predictors with arbitrary architectures to augment their performances. A series of quantitative and qualitative evaluations on four widely used benchmarks (PEMS03, PEMS04, PEMS07, and PEMS08) are conducted to validate the state-of-the-art performance of STD-MAE. Codes are available at https://github.com/Jimmy-7664/STD-MAE.","doi":"10.24963/ijcai.2024/442","url":"https://ijcai.org/proceedings/2024/442","pdf":"https://ijcai.org/proceedings/2024/0442.pdf","markdown":"https://ijcai.org/proceedings/2024/442.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/442","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":442,"pages":{"first":3998,"last":4006},"keywords":["Knowledge Representation and Reasoning: KRR: Qualitative, geometric, spatial, and temporal reasoning","Machine Learning: ML: Time series and data streams","Data Mining: DM: Mining spatial and/or temporal data"],"videos":[],"references":[{"position":1,"raw":"[Bai et al., 2020] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in Neural Infor- mation Processing Systems, 33:17804-17815, 2020.","title":"Adaptive graph convolutional recurrent network for traffic forecasting","authors":"Lei Bai, Lina Yao, Can Li, Xianzhi Wang, Can Wang","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bao et al., 2021] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In International Conference on Learning Representations, 2021.","title":"Beit: Bert pre-training of image transformers","authors":"Hangbo Bao, Li Dong, Songhao Piao, Furu Wei","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2020] Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bix- iong Xu, Jing Bai, Jie Tong, et al. Spectral temporal graph neural network for multivariate time-series forecast- ing. Advances in Neural Information Processing Systems, 33:17766-17778, 2020.","title":"Spectral temporal graph neural network for multivariate time-series forecasting","authors":"Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Yuzhou Chen, Ignacio Segovia, and Yu- lia R Gel. Z-gcnets: Time zigzags at graph convolutional networks for time series forecasting. In International Con- ference on Machine Learning, pages 1684-1694. PMLR, 2021.","title":"Z-gcnets: Time zigzags at graph convolutional networks for time series forecasting","authors":"Yuzhou Chen, Ignacio Segovia, Yulia R Gel","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chung et al., 2014] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evalua- tion of gated recurrent neural networks on sequence mod- eling. arXiv preprint arXiv:1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cirstea et al., 2021] Razvan-Gabriel Cirstea, Tung Kieu, Chenjuan Guo, Bin Yang, and Sinno Jialin Pan. En- hancenet: Plugin neural networks for enhancing corre- lated time series forecasting. In 2021 IEEE 37th Inter- national Conference on Data Engineering (ICDE), pages 1739-1750. IEEE, 2021.","title":"Enhancenet: Plugin neural networks for enhancing correlated time series forecasting","authors":"Razvan-Gabriel Cirstea, Tung Kieu, Chenjuan Guo, Bin Yang, Sinno Jialin Pan","year":"2021","venue":"IEEE 37th International Conference on Data Engineering (ICDE)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cirstea et al., 2022] Razvan-Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu, and Shirui Pan. Towards spatio-temporal aware traffic time series forecasting. In 2022 IEEE 38th International Conference on Data Engi- neering (ICDE), pages 2900-2913. IEEE, 2022.","title":"Towards spatio-temporal aware traffic time series forecasting","authors":"Razvan-Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu, Shirui Pan","year":"2022","venue":"IEEE 38th International Conference on Data Engineering (ICDE)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Deng et al., 2021] Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W Tsang. St-norm: Spatial and tem- poral normalization for multi-variate time series forecast- ing. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 269-278, 2021.","title":"St-norm: Spatial and temporal normalization for multi-variate time series forecasting","authors":"Deng","year":"2021","venue":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deng et al., 2022] Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W Tsang. A multi-view multi-task learning framework for multi-variate time series forecast- ing. IEEE Transactions on Knowledge and Data Engineer- ing, 2022.","title":"A multi-view multi-task learning framework for multi-variate time series forecasting","authors":"Deng","year":"2022","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Deng et al., 2024] Jinliang Deng, Xiusi Chen, Renhe Jiang, Du Yin, Yi Yang, Xuan Song, and Ivor W Tsang. Disen- tangling structured components: Towards adaptive, inter- pretable and scalable time series forecasting. IEEE Trans- actions on Knowledge and Data Engineering, 2024.","title":"Disentangling structured components: Towards adaptive, inter-pretable and scalable time series forecasting","authors":"Deng","year":"2024","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fang et al., 2021] Zheng Fang, Qingqing Long, Guojie Song, and Kunqing Xie. Spatial-temporal graph ode net- works for traffic flow forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 364-373, 2021.","title":"Spatial-temporal graph ode networks for traffic flow forecasting","authors":"Zheng Fang, Qingqing Long, Guojie Song, Kunqing Xie","year":"2021","venue":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo et al., 2019] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 922-929, 2019.","title":"Attention based spatialtemporal graph convolutional networks for traffic flow forecasting","authors":"Guo","year":"2019","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guo et al., 2021a] Kan Guo, Yongli Hu, Yanfeng Sun, Sean Qian, Junbin Gao, and Baocai Yin. Hierarchical graph convolution networks for traffic forecasting. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 35, pages 151-159, 2021.","title":"Hierarchical graph convolution networks for traffic forecasting","authors":"Guo","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Guo et al., 2021b] Shengnan Guo, Youfang Lin, Huaiyu Wan, Xiucheng Li, and Gao Cong. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering, 2021.","title":"Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting","authors":"Guo","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Han et al., 2021] Liangzhe Han, Bowen Du, Leilei Sun, Yanjie Fu, Yisheng Lv, and Hui Xiong. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 547-555, 2021.","title":"Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting","authors":"Han","year":"2021","venue":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[He et al., 2022] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.","title":"Masked autoencoders are scalable vision learners","authors":"Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick","year":"2022","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Jiang et al., 2021] Renhe Jiang, Du Yin, Zhaonan Wang, Yizhuo Wang, Jiewen Deng, Hangchen Liu, Zekun Cai, Jinliang Deng, Xuan Song, and Ryosuke Shibasaki. Dl- traff: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 4515-4525, 2021.","title":"Dltraff: Survey and benchmark of deep learning models for urban traffic prediction","authors":"Renhe Jiang, Du Yin, Zhaonan Wang, Yizhuo Wang, Jiewen Deng, Hangchen Liu, Zekun Cai, Jinliang Deng, Xuan Song, Ryosuke Shibasaki","year":"2021","venue":"Proceedings of the 30th ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jiang et al., 2023a] Jiawei Jiang, Chengkai Han, Wayne Xin Zhao, and Jingyuan Wang. Pdformer: Propagation delay- aware dynamic long-range transformer for traffic flow pre- diction. In AAAI. AAAI Press, 2023.","title":"Pdformer: Propagation delayaware dynamic long-range transformer for traffic flow prediction","authors":"Jiang","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Jiang et al., 2023b] Renhe Jiang, Zhaonan Wang, Jiawei Yong, Puneet Jeph, Quanjun Chen, Yasumasa Kobayashi, Xuan Song, Shintaro Fukushima, and Toyotaro Suzumura. Spatio-temporal meta-graph learning for traffic forecast- ing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8078-8086, 2023.","title":"Spatio-temporal meta-graph learning for traffic forecasting","authors":"Jiang","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kenton and Toutanova, 2019] Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. In Proceedings of NAACL-HLT, pages 4171-4186, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton, Lee Kristina, Toutanova","year":"2019","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lan et al., 2019] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Sori- cut. Albert: A lite bert for self-supervised learning of lan- guage representations. arXiv preprint arXiv:1909.11942, 2019.","title":"Albert: A lite bert for self-supervised learning of language representations","authors":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lan et al., 2022] Shiyong Lan, Yitong Ma, Weikang Huang, Wenwu Wang, Hongyu Yang, and Pyang Li. Dstagnn: Dynamic spatial-temporal aware graph neural network for traffic flow forecasting. In International conference on ma- chine learning, pages 11906-11917. PMLR, 2022.","title":"Dstagnn: Dynamic spatial-temporal aware graph neural network for traffic flow forecasting","authors":"Shiyong Lan, Yitong Ma, Weikang Huang, Wenwu Wang, Hongyu Yang, Pyang Li","year":"2022","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li and Zhu, 2021] Mengzhang Li and Zhanxing Zhu. Spatial-temporal fusion graph neural networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4189-4196, 2021.","title":"Spatial-temporal fusion graph neural networks for traffic flow forecasting","authors":"Zhu ; Li, Mengzhang Li, Zhanxing Zhu","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li et al., 2018] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. In International Conference on Learning Representations, 2018.","title":"Diffusion convolutional recurrent neural network: Data-driven traffic forecasting","authors":"Yaguang Li, Rose Yu, Cyrus Shahabi, Yan Liu","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Li et al., 2023] Zhe Li, Zhongwen Rao, Lujia Pan, Pengyun Wang, and Zenglin Xu. Ti-mae: Self-supervised masked time series autoencoders. arXiv preprint arXiv:2301.08871, 2023.","title":"Ti-mae: Self-supervised masked time series autoencoders","authors":"Zhe Li, Zhongwen Rao, Lujia Pan, Pengyun Wang, Zenglin Xu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Liu et al., 2023] Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 4125-4129, 2023.","title":"Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting","authors":"Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, Xuan Song","year":"2023","venue":"Proceedings of the 32nd ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nie et al., 2022] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Rep- resentations, 2022.","title":"A time series is worth 64 words: Long-term forecasting with transformers","authors":"Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, Jayant Kalagnanam","year":"2022","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Oord et al., 2016] Aaron van den Oord, Sander Diele- man, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw au- dio. arXiv preprint arXiv:1609.03499, 2016.","title":"Wavenet: A generative model for raw audio","authors":"Oord","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Pan et al., 2012] Bei Pan, Ugur Demiryurek, and Cyrus Shahabi. Utilizing real-world transportation data for ac- curate traffic prediction. In 2012 ieee 12th international conference on data mining, pages 595-604. IEEE, 2012.","title":"Utilizing real-world transportation data for accurate traffic prediction","authors":"Pan","year":"2012","venue":"ieee 12th international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Shao et al., 2022a] Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecast- ing. In Proceedings of the 31st ACM International Con- ference on Information & Knowledge Management, pages 4454-4458, 2022.","title":"Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting","authors":"Shao","year":"2022","venue":"Proceedings of the 31st ACM International Con-ference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Shao et al., 2022b] Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1567- 1577, 2022.","title":"Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting","authors":"Shao","year":"2022","venue":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Shao et al., 2023] Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Guangyin Jin, Xin Cao, Gao Cong, et al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. arXiv preprint arXiv:2310.06119, 2023.","title":"Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis","authors":"Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Guangyin Jin, Xin Cao, Gao Cong","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Song et al., 2020] Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial- temporal network data forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 914-921, 2020.","title":"Spatial-temporal synchronous graph convolutional networks: A new framework for spatialtemporal network data forecasting","authors":"Chao Song, Youfang Lin, Shengnan Guo, Huaiyu Wan","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"James H Stock and Mark W Wat- son. Vector autoregressions. Journal of Economic per- spectives, 15(4):101-115, 2001.","title":"Vector autoregressions","authors":"H James, Mark W Stock, Watson","year":"2001","venue":"Journal of Economic perspectives","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Tong et al., 2022] Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre- training. Advances in neural information processing sys- tems, 35:10078-10093, 2022.","title":"Videomae: Masked autoencoders are data-efficient learners for self-supervised video pretraining","authors":"Zhan Tong, Yibing Song, Jue Wang, Limin Wang","year":"2022","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wang and Liu, 2021] Zelun Wang and Jyh-Charn Liu. Translating math formula images to latex sequences using deep neural networks with sequence-level training. Inter- national Journal on Document Analysis and Recognition (IJDAR), 24(1-2):63-75, 2021.","title":"Translating math formula images to latex sequences using deep neural networks with sequence-level training","authors":"Wang, Liu ; ] Zelun Wang, Jyh-Charn Liu","year":"2021","venue":"International Journal on Document Analysis and Recognition","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wu et al., 2019] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In IJCAI, 2019.","title":"Graph wavenet for deep spatial-temporal graph modeling","authors":"Wu","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/264"},{"position":41,"raw":"[Wu et al., 2020] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Con- necting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discov- ery & Data Mining, pages 753-763, 2020.","title":"Connecting the dots: Multivariate time series forecasting with graph neural networks","authors":"Wu","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Xu et al., 2020] Mingxing Xu, Wenrui Dai, Chunmiao Liu, Xing Gao, Weiyao Lin, Guo-Jun Qi, and Hongkai Xiong. Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908, 2020.","title":"Spatial-temporal transformer networks for traffic flow forecasting","authors":"Mingxing Xu, Wenrui Dai, Chunmiao Liu, Xing Gao, Weiyao Lin, Guo-Jun Qi, Hongkai Xiong","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yu et al., 2018] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial In- telligence, pages 3634-3640, 2018.","title":"Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting","authors":"Yu","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/505"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}