{"type":"ScholarlyArticle","title":"SaSDim:Self-Adaptive Noise Scaling Diffusion Model for Spatial Time Series Imputation","authors":[{"name":"Shunyang Zhang","slug":"shunyang-zhang","given":"Shunyang","family":"Zhang","affiliation":"Central South University {224712166 , szwang , 224712155 , rzwang , 234712122 ,"},{"name":"Senzhang Wang","slug":"senzhang-wang","given":"Senzhang","family":"Wang","affiliation":"Central South University {224712166 , szwang , 224712155 , rzwang , 234712122 ,"},{"name":"Xianzhen Tan","slug":"xianzhen-tan","given":"Xianzhen","family":"Tan","affiliation":"Central South University {224712166 , szwang , 224712155 , rzwang , 234712122 ,"},{"name":"Renzhi Wang","slug":"renzhi-wang","given":"Renzhi","family":"Wang","affiliation":"Central South University {224712166 , szwang , 224712155 , rzwang , 234712122 ,"},{"name":"Ruochen Liu","slug":"ruochen-liu","given":"Ruochen","family":"Liu","affiliation":"Central South University {224712166 , szwang , 224712155 , rzwang , 234712122 ,"},{"name":"Jian Zhang","slug":"jian-zhang","given":"Jian","family":"Zhang","affiliation":"Central South University {224712166 , szwang , 224712155 , rzwang , 234712122 ,"},{"name":"Jianxin Wang","slug":"jianxin-wang","given":"Jianxin","family":"Wang","affiliation":"Central South University {224712166 , szwang , 224712155 , rzwang , 234712122 ,"}],"abstract":"Spatial time series imputation is of great importance to various real-world applications. As the state-of-the-art generative models, diffusion models (e.g. CSDI) have outperformed statistical and autoregressive based models in time series imputation. However, diffusion models may introduce unstable noise owing to the inherent uncertainty in sampling, leading to the generated noise deviating from the intended Gaussian distribution. Consequently, the imputed data may deviate from the real data. To this end, we propose a Self-adaptive noise Scaling Diffusion Model named SaSDim for spatial time series imputation. Specifically, we introduce a novel Probabilistic High-Order SDE Solver Module to scale the noise following the standard Gaussian distribution. The noise scaling operation helps the noise prediction module of the diffusion model to more accurately estimate the variance of noise. To effectively learn the spatial and temporal features, a Spatial guided Global Convolution Module (SgGConv) for multi-periodic temporal dependencies learning with the Fast Fourier Transformation and dynamic spatial dependencies learning with dynamic graph convolution is also proposed. Extensive experiments conducted on three real-world spatial time series datasets verify the effectiveness of SaSDim.","doi":"10.24963/ijcai.2024/283","url":"https://ijcai.org/proceedings/2024/283","pdf":"https://ijcai.org/proceedings/2024/0283.pdf","markdown":"https://ijcai.org/proceedings/2024/283.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/283","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":283,"pages":{"first":2561,"last":2569},"keywords":["Data Mining: DM: Mining spatial and/or temporal data"],"videos":[],"references":[{"position":1,"raw":"[Alcaraz and Strodthoff, 2022] Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series impu- tation and forecasting with structured state space models. arXiv preprint arXiv:2208.09399, 2022.","title":"Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models","authors":"Strodthoff Alcaraz","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2018] Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent im- putation for time series. Advances in neural information processing systems, 31, 2018.","title":"Brits: Bidirectional recurrent imputation for time series","authors":"Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, Yitan Li","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Che et al., 2018] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.","title":"Recurrent neural networks for multivariate time series with missing values","authors":"Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, Yan Liu","year":"2018","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cini et al., 2022] Andrea Cini, Ivan Marisca, and Cesare Alippi. Filling the g ap s: Multivariate time series im- putation by graph neural networks. In ICLR, 2022.","title":"Filling the g ap s: Multivariate time series imputation by graph neural networks","authors":"Andrea Cini, Ivan Marisca, Cesare Alippi","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du et al., 2023] Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-based imputation for time series. Expert Systems with Applications, 219:119619, 2023.","title":"Saits: Self-attention-based imputation for time series","authors":"Wenjie Du, David Côté, Yan Liu","year":"2023","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goel et al., 2022] Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré. It's raw! audio generation with state- space models. In Proceedings of International Conference on Machine Learning, pages 7616-7633. PMLR, 2022.","title":"It's raw! audio generation with statespace models","authors":"Karan Goel, Albert Gu, Chris Donahue, Christopher Ré","year":"2022","venue":"Proceedings of International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gu et al., 2021] Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021.","title":"Efficiently modeling long sequences with structured state spaces","authors":"Albert Gu, Karan Goel, Christopher Ré","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hewage et al., 2020] Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu. Temporal con- volutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, 24:16453-16482, 2020.","title":"Temporal convolutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station","authors":"Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, Yonghuai Liu","year":"2020","venue":"Soft Computing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2022] Yuhong Li, Tianle Cai, Yi Zhang, Deming Chen, and Debadeepta Dey. What makes convolutional models great on long sequence modeling? arXiv preprint arXiv:2210.09298, 2022.","title":"What makes convolutional models great on long sequence modeling","authors":"Yuhong Li, Tianle Cai, Yi Zhang, Deming Chen, Debadeepta Dey","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lim and Zohren, 2021] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a sur- vey. Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.","title":"Time-series forecasting with deep learning: a survey","authors":"Bryan Lim, Stefan Zohren","year":"2021","venue":"Philosophical Transactions of the Royal Society A","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Luo et al., 2018] Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, 31, 2018.","title":"Multivariate time series imputation with generative adversarial networks","authors":"Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Marisca et al., 2022] Ivan Marisca, Andrea Cini, and Ce- sare Alippi. Learning to reconstruct missing data from spa- tiotemporal graphs with sparse observations. Advances in Neural Information Processing Systems, 35:32069-32082, 2022.","title":"Learning to reconstruct missing data from spatiotemporal graphs with sparse observations","authors":"Ivan Marisca, Andrea Cini, Cesare Alippi","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nelson, 1998] Brian K Nelson. Time series analysis using autoregressive integrated moving average (arima) models. Academic emergency medicine, 5(7):739-744, 1998.","title":"Time series analysis using autoregressive integrated moving average (arima) models","authors":"Brian K Nelson","year":"1998","venue":"Academic emergency medicine","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nguyen et al., 2022] Eric Nguyen, Karan Goel, Albert Gu, Gordon Downs, Preey Shah, Tri Dao, Stephen Baccus, and Christopher Ré. S4nd: Modeling images and videos as multidimensional signals with state spaces. Advances in neural information processing systems, 35:2846-2861, 2022.","title":"S4nd: Modeling images and videos as multidimensional signals with state spaces","authors":"Eric Nguyen, Karan Goel, Albert Gu, Gordon Downs, Preey Shah, Tri Dao, Stephen Baccus, Christopher Ré","year":"2022","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Song et al., 2020] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.","title":"Score-based generative modeling through stochastic differential equations","authors":"Yang Song, Jascha Sohl-Dickstein, P Diederik, Abhishek Kingma, Stefano Kumar, Ben Ermon, Poole","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tashiro et al., 2021] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score- based diffusion models for probabilistic time series impu- tation. Advances in Neural Information Processing Sys- tems, 34:24804-24816, 2021.","title":"Csdi: Conditional scorebased diffusion models for probabilistic time series imputation","authors":"Yusuke Tashiro, Jiaming Song, Yang Song, Stefano Ermon","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2020] Senzhang Wang, Jiannong Cao, and S Yu Philip. Deep learning for spatio-temporal data min- ing: A survey. IEEE transactions on knowledge and data engineering, 34(8):3681-3700, 2020.","title":"Deep learning for spatio-temporal data mining: A survey","authors":"Wang","year":"2020","venue":"IEEE transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2021] Qinfen Wang, Siyuan Ren, Yong Xia, and Longbing Cao. Bicmts: Bidirectional coupled mul- tivariate learning of irregular time series with missing val- ues. In Proceedings of the 30th ACM International Con- ference on Information & Knowledge Management, pages 3493-3497, 2021.","title":"Bicmts: Bidirectional coupled multivariate learning of irregular time series with missing values","authors":"Wang","year":"2021","venue":"Proceedings of the 30th ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2022a] Senzhang Wang, Jiyue Li, Hao Miao, Junbo Zhang, Junxing Zhu, and Jianxin Wang. Generative- free urban flow imputation. In Proceedings of the 31st ACM International Conference on Information & Knowl- edge Management, pages 2028-2037, 2022.","title":"Generativefree urban flow imputation","authors":"Wang","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2022b] Senzhang Wang, Jiyue Li, Hao Miao, Junbo Zhang, Junxing Zhu, and Jianxin Wang. Generative- free urban flow imputation. In Proceedings of the 31st ACM International Conference on Information & Knowl- edge Management, pages 2028-2037, 2022.","title":"Generativefree urban flow imputation","authors":"Wang","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2022] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2022.","title":"Timesnet: Temporal 2d-variation modeling for general time series analysis","authors":"Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long","year":"2022","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2022] Xin Xu, Liangliang Zhang, Qi Kong, Chen- guang Gui, and Xing Zhang. Enhanced-historical aver- age for long-term prediction. In 2022 2nd International Conference on Computer, Control and Robotics (ICCCR), pages 115-119. IEEE, 2022.","title":"Enhanced-historical average for long-term prediction","authors":"Xin Xu, Liangliang Zhang, Qi Kong, Chenguang Gui, Xing Zhang","year":"2022","venue":"2022 2nd International Conference on Computer, Control and Robotics (ICCCR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2022] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applica- tions. arXiv preprint arXiv:2209.00796, 2022.","title":"Diffusion models: A comprehensive survey of methods and applications","authors":"Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, Ming-Hsuan Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yi et al., 2016] Xiuwen Yi, Yu Zheng, Junbo Zhang, and Tianrui Li. St-mvl: filling missing values in geo-sensory time series data. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016.","title":"St-mvl: filling missing values in geo-sensory time series data","authors":"Xiuwen Yi, Yu Zheng, Junbo Zhang, Tianrui Li","year":"2016","venue":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/384"},{"position":25,"raw":"[Yoon et al., 2017] Jinsung Yoon, William R Zame, and Mi- haela van der Schaar. Multi-directional recurrent neural networks: A novel method for estimating missing data. In Proceedings of Time series workshop in international con- ference on machine learning, 2017.","title":"Multi-directional recurrent neural networks: A novel method for estimating missing data","authors":"Yoon","year":"2017","venue":"Proceedings of Time series workshop in international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yoon et al., 2018] Jinsung Yoon, James Jordon, and Mi- haela Schaar. Gain: Missing data imputation using genera- tive adversarial nets. In Proceedings of International con- ference on machine learning, pages 5689-5698. PMLR, 2018.","title":"Gain: Missing data imputation using generative adversarial nets","authors":"Yoon","year":"2018","venue":"Proceedings of International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2016] Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. Advances in neu- ral information processing systems, 29, 2016.","title":"Temporal regularized matrix factorization for high-dimensional time series prediction","authors":"Hsiang-Fu Yu, Nikhil Rao, Inderjit S Dhillon","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}