{"type":"ScholarlyArticle","title":"Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputation","authors":[{"name":"Shunyang Zhang","slug":"shunyang-zhang","given":"Shunyang","family":"Zhang","affiliation":"Central South University"},{"name":"Senzhang Wang","slug":"senzhang-wang","given":"Senzhang","family":"Wang","affiliation":"Central South University"},{"name":"Hao Miao","slug":"hao-miao","given":"Hao","family":"Miao","affiliation":"Aalborg University"},{"name":"Hao Chen","slug":"hao-chen","given":"Hao","family":"Chen","affiliation":"The Hong Kong Polytechnic University"},{"name":"Changjun Fan","slug":"changjun-fan","given":"Changjun","family":"Fan","affiliation":"National University of Defense Technology"},{"name":"Jian Zhang","slug":"jian-zhang","given":"Jian","family":"Zhang","affiliation":"Central South University"}],"abstract":"Multivariant time series (MTS) data are usually incomplete in real scenarios, and imputing the incomplete MTS is practically important to facilitate various time series mining tasks. Recently, diffusion model-based MTS imputation methods have achieved promising results by utilizing CNN or attention mechanisms for temporal features learning. However, it is hard to adaptively trade off the diverse effects of local and global temporal features by simply combining CNN and attention. To address this issue, we propose a Score-weighted Convolutional Diffusion Model (Score-CDM for short), whose backbone consists of a Score-weighted Convolution Module (SCM) and an Adaptive Reception Module (ARM). SCM adopts a score map to capture the global temporal features in the time domain, while ARM uses a Spectral2Time Window Block (S2TWB) to convolve the local time series data in the spectral domain. Benefiting from the time convolution properties of Fast Fourier Transformation, ARM can adaptively change the receptive field of the score map, and thus effectively balance the local and global temporal features. We conduct extensive evaluations on three real MTS datasets of different domains, and the result verifies the effectiveness of the proposed Score-CDM.","doi":"10.24963/ijcai.2024/282","url":"https://ijcai.org/proceedings/2024/282","pdf":"https://ijcai.org/proceedings/2024/0282.pdf","markdown":"https://ijcai.org/proceedings/2024/282.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/282","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":282,"pages":{"first":2551,"last":2560},"keywords":["Data Mining: DM: Mining spatial and/or temporal data"],"videos":[],"references":[{"position":1,"raw":"[Acevedo and Masuoka, 1997] William Acevedo and Penny Masuoka. Time-series animation techniques for visualiz- ing urban growth. Computers & Geosciences, 23(4):423- 435, 1997.","title":"Time-series animation techniques for visualizing urban growth","authors":"Masuoka Acevedo, William Acevedo, Penny Masuoka","year":"1997","venue":"Computers & Geosciences","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alcaraz and Strodthoff, 2022] Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series impu- tation and forecasting with structured state space models. arXiv preprint arXiv:2208.09399, 2022.","title":"Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models","authors":"Strodthoff Alcaraz","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2018] Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent im- putation for time series. Advances in neural information processing systems, 31, 2018.","title":"Brits: Bidirectional recurrent imputation for time series","authors":"Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, Yitan Li","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Che et al., 2018] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.","title":"Recurrent neural networks for multivariate time series with missing values","authors":"Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, Yan Liu","year":"2018","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choromanski et al., 2020] Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Col- well, and Adrian Weller. Rethinking attention with per- formers, 2020.","title":"Rethinking attention with performers","authors":"Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, Adrian Weller","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cini et al., 2022] Andrea Cini, Ivan Marisca, and Cesare Alippi. Filling the g ap s: Multivariate time series im- putation by graph neural networks. In ICLR, 2022.","title":"Filling the g ap s: Multivariate time series imputation by graph neural networks","authors":"Andrea Cini, Ivan Marisca, Cesare Alippi","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du et al., 2023] Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-based imputation for time series. Expert Systems with Applications, 219:119619, 2023.","title":"Saits: Self-attention-based imputation for time series","authors":"Wenjie Du, David Côté, Yan Liu","year":"2023","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2021] Ziquan Fang, Lu Pan, Lu Chen, Yuntao Du, and Yunjun Gao. Mdtp: A multi-source deep traffic prediction framework over spatio-temporal trajectory data. Proceedings of the VLDB Endowment, 14(8):1289-1297, 2021.","title":"Mdtp: A multi-source deep traffic prediction framework over spatio-temporal trajectory data","authors":"Fang","year":"2021","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gu et al., 2022] Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the parameterization and initialization of diagonal state space models. Advances in Neural Infor- mation Processing Systems, 35:35971-35983, 2022.","title":"On the parameterization and initialization of diagonal state space models","authors":"Albert Gu, Karan Goel, Ankit Gupta, Christopher Ré","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33:6840-6851, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Katharopoulos et al., 2020] Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Franc ¸ois Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. In International conference on machine learning, pages 5156-5165. PMLR, 2020.","title":"Transformers are rnns: Fast autoregressive transformers with linear attention","authors":"Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, Franc ¸ois, Fleuret","year":"2020","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kitaev et al., 2019] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In International Conference on Learning Representations, 2019.","title":"Reformer: The efficient transformer","authors":"Nikita Kitaev, Lukasz Kaiser, Anselm Levskaya","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee and Fambro, 1999] Sangsoo Lee and Daniel B Fambro. Application of subset autoregressive integrated moving av- erage model for short-term freeway traffic volume fore- casting. Transportation research record, 1678(1):179- 188, 1999.","title":"Application of subset autoregressive integrated moving average model for short-term freeway traffic volume forecasting","authors":"Sangsoo Lee, Daniel B Fambro","year":"1999","venue":"Transportation research record","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2022] Jiyue Li, Senzhang Wang, Jiaqiang Zhang, Hao Miao, Junbo Zhang, and S Yu Philip. Fine-grained urban flow inference with incomplete data. IEEE Transac- tions on Knowledge and Data Engineering, 35(6):5851- 5864, 2022.","title":"Fine-grained urban flow inference with incomplete data","authors":"Li","year":"2022","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lim and Zohren, 2021] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a sur- vey. Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.","title":"Time-series forecasting with deep learning: a survey","authors":"Bryan Lim, Stefan Zohren","year":"2021","venue":"Philosophical Transactions of the Royal Society A","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2022] Minhao Liu, Ailing Zeng, Muxi Chen, Zhi- jian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convo- lution and interaction. Advances in Neural Information Processing Systems, 35:5816-5828, 2022.","title":"Scinet: Time series modeling and forecasting with sample convolution and interaction","authors":"Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, Qiang Xu","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2023] Mingzhe Liu, Han Huang, Hao Feng, Leilei Sun, Bowen Du, and Yanjie Fu. Pristi: A condi- tional diffusion framework for spatiotemporal imputation. 2023 IEEE 39th International Conference on Data Engi- neering (ICDE), pages 1927-1939, 2023.","title":"Pristi: A conditional diffusion framework for spatiotemporal imputation","authors":"Liu","year":"2023","venue":"IEEE 39th International Conference on Data Engineering (ICDE)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Luo et al., 2018] Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, 31, 2018.","title":"Multivariate time series imputation with generative adversarial networks","authors":"Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Marisca et al., 2022] Ivan Marisca, Andrea Cini, and Ce- sare Alippi. Learning to reconstruct missing data from spa- tiotemporal graphs with sparse observations. Advances in Neural Information Processing Systems, 35:32069-32082, 2022.","title":"Learning to reconstruct missing data from spatiotemporal graphs with sparse observations","authors":"Ivan Marisca, Andrea Cini, Cesare Alippi","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Miao et al., 2024] Hao Miao, Yan Zhao, Chenjuan Guo, Bin Yang, Zheng Kai, Feiteng Huang, Jiandong Xie, and Christian S Jensen. A unified replay-based continu- ous learning framework for spatio-temporal prediction on streaming data. In ICDE, 2024.","title":"A unified replay-based continuous learning framework for spatio-temporal prediction on streaming data","authors":"Hao Miao, Yan Zhao, Chenjuan Guo, Bin Yang, Zheng Kai, Feiteng Huang, Jiandong Xie, Christian S Jensen","year":"2024","venue":"ICDE","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nelson, 1998] Brian K Nelson. Time series analysis using autoregressive integrated moving average (arima) models. Academic emergency medicine, 5(7):739-744, 1998.","title":"Time series analysis using autoregressive integrated moving average (arima) models","authors":"Brian K Nelson","year":"1998","venue":"Academic emergency medicine","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Peng et al., 2023] Bo Peng, Eric Alcaide, Quentin An- thony, Alon Albalak, Samuel Arcadinho, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran GV, et al. Rwkv: Reinventing rnns for the transformer era. arXiv preprint arXiv:2305.13048, 2023.","title":"Rwkv: Reinventing rnns for the transformer era","authors":"Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran, G V","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Peterson, 2009] Leif E Peterson. K-nearest neighbor. Schol- arpedia, 4(2):1883, 2009.","title":"K-nearest neighbor","authors":"Leif E Peterson","year":"2009","venue":"Scholarpedia","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shen and Kwok, 2023] Lifeng Shen and James Kwok. Non- autoregressive conditional diffusion models for time series prediction. arXiv preprint arXiv:2306.05043, 2023.","title":"Nonautoregressive conditional diffusion models for time series prediction","authors":"Kwok ; ] Lifeng Shen, James Shen, Kwok","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Si et al., 2022] Chenyang Si, Weihao Yu, Pan Zhou, Yichen Zhou, Xinchao Wang, and Shuicheng Yan. Inception transformer. Advances in Neural Information Processing Systems, 35:23495-23509, 2022.","title":"Inception transformer","authors":"Chenyang Si, Weihao Yu, Pan Zhou, Yichen Zhou, Xinchao Wang, Shuicheng Yan","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tashiro et al., 2021] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. NeurIPS, 34:24804-24816, 2021.","title":"Csdi: Conditional score-based diffusion models for probabilistic time series imputation","authors":"Yusuke Tashiro, Jiaming Song, Yang Song, Stefano Ermon","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2020] Senzhang Wang, Jiannong Cao, and S Yu Philip. Deep learning for spatio-temporal data min- ing: A survey. IEEE transactions on knowledge and data engineering, 34(8):3681-3700, 2020.","title":"Deep learning for spatio-temporal data mining: A survey","authors":"Wang","year":"2020","venue":"IEEE transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2021] Qinfen Wang, Siyuan Ren, Yong Xia, and Longbing Cao. Bicmts: Bidirectional coupled mul- tivariate learning of irregular time series with missing val- ues. In Proceedings of the 30th ACM International Con- ference on Information & Knowledge Management, pages 3493-3497, 2021.","title":"Bicmts: Bidirectional coupled multivariate learning of irregular time series with missing values","authors":"Wang","year":"2021","venue":"Proceedings of the 30th ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2022] Senzhang Wang, Jiyue Li, Hao Miao, Junbo Zhang, Junxing Zhu, and Jianxin Wang. Generative- free urban flow imputation. In Proceedings of the 31st ACM International Conference on Information & Knowl- edge Management, pages 2028-2037, 2022.","title":"Generativefree urban flow imputation","authors":"Wang","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2022] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2022.","title":"Timesnet: Temporal 2d-variation modeling for general time series analysis","authors":"Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long","year":"2022","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2022] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 2022.","title":"Diffusion models: A comprehensive survey of methods and applications","authors":"Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, Ming-Hsuan Yang","year":"2022","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yao et al., 2023] Yuanyuan Yao, Dimeng Li, Hailiang Jie, Hailiang Jie, Tianyi Li, Jie Chen, Jiaqi Wang, Feifei Li, and Yunjun Gao. Simplets: An efficient and universal model selection framework for time series forecasting. Proceedings of the VLDB Endowment, 16(12):3741-3753, 2023.","title":"Simplets: An efficient and universal model selection framework for time series forecasting","authors":"Yuanyuan Yao, Dimeng Li, Hailiang Jie, Hailiang Jie, Tianyi Li, Jie Chen, Jiaqi Wang, Feifei Li, Yunjun Gao","year":"2023","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yi et al., 2016] Xiuwen Yi, Yu Zheng, Junbo Zhang, and Tianrui Li. St-mvl: filling missing values in geo-sensory time series data. In IJCAI, 2016.","title":"St-mvl: filling missing values in geo-sensory time series data","authors":"Xiuwen Yi, Yu Zheng, Junbo Zhang, Tianrui Li","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/384"},{"position":35,"raw":"[Yoon et al., 2017] Jinsung Yoon, William R Zame, and Mi- haela van der Schaar. Multi-directional recurrent neural networks: A novel method for estimating missing data. In Proceedings of Time series workshop in international con- ference on machine learning, 2017.","title":"Multi-directional recurrent neural networks: A novel method for estimating missing data","authors":"Yoon","year":"2017","venue":"Proceedings of Time series workshop in international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhai et al., 2021] Shuangfei Zhai, Walter Talbott, Nitish Srivastava, Chen Huang, Hanlin Goh, Ruixiang Zhang, and Josh Susskind. An attention free transformer. arXiv preprint arXiv:2105.14103, 2021.","title":"An attention free transformer","authors":"Zhai","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}