{"type":"ScholarlyArticle","title":"DiffAR: Adaptive Conditional Diffusion Model for Temporal-augmented Human Activity Recognition","authors":[{"name":"Shuokang Huang","slug":"shuokang-huang","given":"Shuokang","family":"Huang","affiliation":"Imperial College London"},{"name":"Po-Yu Chen","slug":"po-yu-chen","given":"Po-Yu","family":"Chen","affiliation":"Imperial College London; JPMorgan Chase & Co."},{"name":"Julie McCann","slug":"julie-mccann","given":"Julie","family":"McCann","affiliation":"Imperial College London"}],"abstract":"Human activity recognition (HAR) is a fundamental sensing and analysis technique that supports diverse applications, such as smart homes and healthcare. In device-free and non-intrusive HAR, WiFi channel state information (CSI) captures wireless signal variations caused by human interference without the need for video cameras or on-body sensors. However, current CSI-based HAR performance is hampered by incomplete CSI recordings due to fixed window sizes in CSI collection and human/machine errors that incur missing values in CSI. To address these issues, we propose DiffAR, a temporal-augmented HAR approach that improves HAR performance by augmenting CSI. DiffAR devises a novel Adaptive Conditional Diffusion Model (ACDM) to synthesize augmented CSI, which tackles the issue of fixed windows by forecasting and handles missing values with imputation. Compared to existing diffusion models, ACDM improves the synthesis quality by guiding progressive synthesis with step-specific conditions. DiffAR further exploits an ensemble classifier for activity recognition using both raw and augmented CSI. Extensive experiments on four public datasets show that DiffAR achieves the best synthesis quality of augmented CSI and outperforms state-of-the-art CSI-based HAR methods in recognition performance. The source code of DiffAR is available at https://github.com/huangshk/DiffAR.","doi":"10.24963/ijcai.2023/424","url":"https://ijcai.org/proceedings/2023/424","pdf":"https://ijcai.org/proceedings/2023/0424.pdf","markdown":"https://ijcai.org/proceedings/2023/424.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/424","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":424,"pages":{"first":3812,"last":3820},"keywords":["Machine Learning: ML: Semi-supervised learning","Machine Learning: ML: Applications","Data Mining: DM: Mining spatial and/or temporal data","Machine Learning: ML: Time series and data streams"],"videos":[],"references":[{"position":1,"raw":"[Alazrai et al., 2020] Rami Alazrai, Ali Awad, Alsaify Baha'A, Mohammad Hababeh, and Mohammad I Daoud. A dataset for wi-fi-based human-to-human interaction recognition. Data in brief, 31:105668, 2020.","title":"A dataset for wi-fi-based human-to-human interaction recognition","authors":"Rami Alazrai, Ali Awad, Alsaify Baha, ' A, Mohammad Hababeh, Mohammad I Daoud","year":"2020","venue":"Data in brief","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[An and Ogras, 2021] Sizhe An and Umit Y Ogras. Mars: mmwave-based assistive rehabilitation system for smart healthcare. ACM Transactions on Embedded Computing Systems (TECS), 20(5s):1-22, 2021.","title":"Sizhe An and Umit Y Ogras. Mars: mmwave-based assistive rehabilitation system for smart healthcare","authors":"An, Ogras","year":"2021","venue":"ACM Transactions on Embedded Computing Systems (TECS)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bianchi et al., 2019] Valentina Bianchi, Marco Bassoli, Gi- anfranco Lombardo, Paolo Fornacciari, Monica Mor- donini, and Ilaria De Munari. Iot wearable sensor and deep learning: An integrated approach for personalized human activity recognition in a smart home environment. IEEE Internet of Things Journal, 6(5):8553-8562, 2019.","title":"Iot wearable sensor and deep learning: An integrated approach for personalized human activity recognition in a smart home environment","authors":"Valentina Bianchi, Marco Bassoli, Gianfranco Lombardo, Paolo Fornacciari, Monica Mordonini, Ilaria De, Munari","year":"2019","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018] Zhenghua Chen, Le Zhang, Chaoyang Jiang, Zhiguang Cao, and Wei Cui. Wifi csi based passive human activity recognition using attention based blstm. IEEE Transactions on Mobile Computing, 18(11):2714- 2724, 2018.","title":"Wifi csi based passive human activity recognition using attention based blstm","authors":"Zhenghua Chen, Le Zhang, Chaoyang Jiang, Zhiguang Cao, Wei Cui","year":"2018","venue":"IEEE Transactions on Mobile Computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009.00713, 2020.","title":"Wavegrad: Estimating gradients for waveform generation","authors":"Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, William Chan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Esteban et al., 2017] Cristóbal Esteban, Stephanie L Hy- land, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.02633, 2017.","title":"Real-valued (medical) time series generation with recurrent conditional gans","authors":"Cristóbal Esteban, Stephanie L Hyland, Gunnar Rätsch","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2020] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial networks. Communications of the ACM, 63(11):139-144, 2020.","title":"Generative adversarial networks","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2020","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ho and Salimans, 2021] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Work- shop on Deep Generative Models and Downstream Appli- cations, 2021.","title":"Classifier-free diffusion guidance","authors":"Jonathan Ho, Tim Salimans","year":"2021","venue":"NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hussain et al., 2020] Zawar Hussain, Quan Z Sheng, and Wei Emma Zhang. A review and categorization of tech- niques on device-free human activity recognition. Jour- nal of Network and Computer Applications, 167:102738, 2020.","title":"A review and categorization of techniques on device-free human activity recognition","authors":"Zawar Hussain, Wei Emma Quan Z Sheng, Zhang","year":"2020","venue":"Journal of Network and Computer Applications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kong et al., 2021] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versa- tile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.","title":"Diffwave: A versatile diffusion model for audio synthesis","authors":"Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, Bryan Catanzaro","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2021] Bing Li, Wei Cui, Wei Wang, Le Zhang, Zhenghua Chen, and Min Wu. Two-stream convolution augmented transformer for human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pages 286-293, 2021.","title":"Two-stream convolution augmented transformer for human activity recognition","authors":"Li","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2020] Yuxiang Lin, Yi Gao, Bingji Li, and Wei Dong. Revisiting indoor intrusion detection with wifi sig- nals: do not panic over a pet! IEEE Internet of Things Journal, 7(10):10437-10449, 2020.","title":"Revisiting indoor intrusion detection with wifi signals: do not panic over a pet!","authors":"Yuxiang Lin, Yi Gao, Bingji Li, Wei Dong","year":"2020","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2018] Yongsen Ma, Gang Zhou, Shuangquan Wang, Hongyang Zhao, and Woosub Jung. Signfi: Sign language recognition using wifi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Tech- nologies, 2(1):1-21, 2018.","title":"Signfi: Sign language recognition using wifi","authors":"Yongsen Ma, Gang Zhou, Shuangquan Wang, Hongyang Zhao, Woosub Jung","year":"2018","venue":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma et al., 2019] Yongsen Ma, Gang Zhou, and Shuangquan Wang. Wifi sensing with channel state information: A sur- vey. ACM Computing Surveys (CSUR), 52(3):1-36, 2019.","title":"Wifi sensing with channel state information: A survey","authors":"Ma","year":"2019","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Matheson and Winkler, 1976] James E Matheson and Robert L Winkler. Scoring rules for continuous probabil- ity distributions. Management science, 22(10):1087-1096, 1976.","title":"Scoring rules for continuous probability distributions","authors":"E James, Robert L Matheson, Winkler","year":"1976","venue":"Management science","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mogren, 2016] Olof Mogren. C-rnn-gan: Continuous re- current neural networks with adversarial training. arXiv preprint arXiv:1611.09904, 2016.","title":"C-rnn-gan: Continuous recurrent neural networks with adversarial training","authors":"Mogren","year":"2016","venue":"Olof Mogren","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Moshiri et al., 2021] Parisa Fard Moshiri, Mohammad Nabati, Reza Shahbazian, and Seyed Ali Ghorashi. Csi- based human activity recognition using convolutional neu- ral networks. In 2021 11th International Conference on Computer Engineering and Knowledge (ICCKE), pages 7- 12. IEEE, 2021.","title":"Csibased human activity recognition using convolutional neural networks","authors":"Fard Parisa, Mohammad Moshiri, Reza Nabati, Seyed Shahbazian, Ghorashi Ali","year":"2021","venue":"11th International Conference on Computer Engineering and Knowledge (ICCKE)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nirmal et al., 2021] Isura Nirmal, Abdelwahed Khamis, Mahbub Hassan, Wen Hu, and Xiaoqing Zhu. Deep learn- ing for radio-based human sensing: Recent advances and future directions. IEEE Communications Surveys & Tuto- rials, 23(2):995-1019, 2021.","title":"Deep learning for radio-based human sensing: Recent advances and future directions","authors":"Nirmal","year":"2021","venue":"IEEE Communications Surveys & Tutorials","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Oord et al., 2016] Aaron van den Oord, Sander Diele- man, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw au- dio. arXiv preprint arXiv:1609.03499, 2016.","title":"Wavenet: A generative model for raw audio","authors":"Oord","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rasul et al., 2021] Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denois- ing diffusion models for multivariate probabilistic time se- ries forecasting. In International Conference on Machine Learning, pages 8857-8868. PMLR, 2021.","title":"Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting","authors":"Kashif Rasul, Calvin Seward, Ingmar Schuster, Roland Vollgraf","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.","title":"High-resolution image synthesis with latent diffusion models","authors":"Rombach","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shalaby et al., 2022] Eman Shalaby, Nada ElShennawy, and Amany Sarhan. Utilizing deep learning models in csi- based human activity recognition. Neural Computing and Applications, 34(8):5993-6010, 2022.","title":"Utilizing deep learning models in csibased human activity recognition","authors":"Eman Shalaby, Nada Elshennawy, Amany Sarhan","year":"2022","venue":"Neural Computing and Applications","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tan et al., 2022] Sheng Tan, Yili Ren, Jie Yang, and Yingy- ing Chen. Commodity wifi sensing in ten years: Status, challenges, and opportunities. IEEE Internet of Things Journal, 9(18):17832-17843, 2022.","title":"Commodity wifi sensing in ten years: Status, challenges, and opportunities","authors":"Sheng Tan, Yili Ren, Jie Yang, Yingying Chen","year":"2022","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tashiro et al., 2021] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score- based diffusion models for probabilistic time series impu- tation. Advances in Neural Information Processing Sys- tems, 34:24804-24816, 2021.","title":"Csdi: Conditional scorebased diffusion models for probabilistic time series imputation","authors":"Yusuke Tashiro, Jiaming Song, Yang Song, Stefano Ermon","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2015] Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st annual international conference on mobile computing and networking, pages 65-76, 2015.","title":"Understanding and modeling of wifi signal based human activity recognition","authors":"Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, Sanglu Lu","year":"2015","venue":"Proceedings of the 21st annual international conference on mobile computing and networking","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2019] Fei Wang, Jianwei Feng, Yinliang Zhao, Xiaobin Zhang, Shiyuan Zhang, and Jinsong Han. Joint activity recognition and indoor localization with wifi fin- gerprints. IEEE Access, 7:80058-80068, 2019.","title":"Joint activity recognition and indoor localization with wifi fingerprints","authors":"Fei Wang, Jianwei Feng, Yinliang Zhao, Xiaobin Zhang, Shiyuan Zhang, Jinsong Han","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wen et al., 2021] Qingsong Wen, Liang Sun, Fan Yang, Xi- aomin Song, Jingkun Gao, Xue Wang, and Huan Xu. Time series data augmentation for deep learning: A sur- vey. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4653-4660. International Joint Confer- ences on Artificial Intelligence Organization, 8 2021. Sur- vey Track.","title":"Time series data augmentation for deep learning: A survey","authors":"Qingsong Wen, Liang Sun, Fan Yang, Xiaomin Song, Jingkun Gao, Xue Wang, Huan Xu","year":"2021","venue":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21","doi":null,"url":null,"internal_url":"/proceedings/2021/631"},{"position":31,"raw":"[Yang et al., 2018] Jianfei Yang, Han Zou, Hao Jiang, and Lihua Xie. Device-free occupant activity sensing using wifi-enabled iot devices for smart homes. IEEE Internet of Things Journal, 5(5):3991-4002, 2018.","title":"Device-free occupant activity sensing using wifi-enabled iot devices for smart homes","authors":"Jianfei Yang, Han Zou, Hao Jiang, Lihua Xie","year":"2018","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2022] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applica- tions. arXiv preprint arXiv:2209.00796, 2022.","title":"Diffusion models: A comprehensive survey of methods and applications","authors":"Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, Ming-Hsuan Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yoon et al., 2019] Jinsung Yoon, Daniel Jarrett, and Mi- haela Van der Schaar. Time-series generative adversarial networks. Advances in neural information processing sys- tems, 32, 2019.","title":"Time-series generative adversarial networks","authors":"Yoon","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yousefi et al., 2017] Siamak Yousefi, Hirokazu Narui, Sankalp Dayal, Stefano Ermon, and Shahrokh Valaee. A survey on behavior recognition using wifi channel state information. IEEE Communications Magazine, 55(10):98-104, 2017.","title":"A survey on behavior recognition using wifi channel state information","authors":"Siamak Yousefi, Hirokazu Narui, Sankalp Dayal, Stefano Ermon, Shahrokh Valaee","year":"2017","venue":"IEEE Communications Magazine","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zeiler et al., 2010] Matthew D Zeiler, Dilip Krishnan, Gra- ham W Taylor, and Rob Fergus. Deconvolutional net- works. In 2010 IEEE Computer Society Conference on computer vision and pattern recognition, pages 2528- 2535. IEEE, 2010.","title":"Deconvolutional networks","authors":"Dilip Matthew D Zeiler, Graham W Krishnan, Rob Taylor, Fergus","year":"2010","venue":"IEEE Computer Society Conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2021] Yi Zhang, Yue Zheng, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. Widar3. 0: Zero-effort cross-domain gesture recog- nition with wi-fi. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.","title":"Widar3. 0: Zero-effort cross-domain gesture recognition with wi-fi","authors":"Yi Zhang, Yue Zheng, Guidong Kun Qian, Yunhao Zhang, Chenshu Liu, Zheng Wu, Yang","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}