Proceedings

Communication-Efficient Federated Video Human Activity Recognition via Discriminative Subspace Discovery

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 1695-1703.

https://doi.org/10.24963/ijcai.2026/189

Abstract

Federated learning (FL) enables privacy-preserving video human activity recognition without centralizing data. Despite its potential, FL remains fundamentally constrained by the high communication overhead incurred during training.

Existing methods, such as model quantization, gradient sparsification and low-rank approximation, aim to mitigate this overhead but often at the cost of model expressivity, leading to reduced classification accuracy.

In this work, we take a representation-centric perspective and show that for video-base activity recognition using pretrained embeddings, class-discriminative information lies in a very low-dimensional subspace, that preserves linear seperability between activity classes. Motivated by this observation, we propose Federated discriminative subspace discovery, a framework that collaboratively identify and operate within such a subspace.

Instead of communicating full classifier updates, each client first projects its local video embeddings using a shared random projection matrix. It then computes the covariance of the projected embeddings and transmits this statistic to the server for aggregation. The server reconstructs a global covariance matrix, performs singular value decomposition, and selects the top-k eigenvectors that define the global class-discriminative subspace. Subsequent communication occurs only in this compact subspace. Extensive experiments on UCF101, HMDB51, and Toyota-SmartHome demonstrate that, learning and sharing the classifier in the discovered subspace, reduces communication costs by up to 61%, with a moderate drop in activity recognition accuracy compared to the full model.

  1. [Alistarh et al., 2017] Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. Qsgd: Communication-efficient sgd via gradient quantization and encoding. Advances in neural information processing systems, 30, 2017.
  2. [Alistarh et al., 2018] Dan Alistarh, Torsten Hoefler, Mikael Johansson, Nikola Konstantinov, Sarit Khirirat, and Cédric Renggli. The convergence of sparsified gradient methods. Advances in Neural Information Processing Systems, 31, 2018.
  3. [Amiri et al., 2020] Mohammad Mohammadi Amiri, Deniz Gunduz, Sanjeev R Kulkarni, and H Vincent Poor. Feder- ated learning with quantized global model updates. arXiv preprint arXiv:2006.10672, 2020.
  4. [Azar et al., 2019] Sina Mokhtarzadeh Azar, Mina Ghadimi Atigh, Ahmad Nickabadi, and Alexandre Alahi. Convo- lutional relational machine for group activity recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7892-7901, 2019.
  5. [Benfold and Reid, 2011] Ben Benfold and Ian Reid. Stable multi-target tracking in real-time surveillance video. In CVPR 2011, pages 3457-3464. IEEE, 2011.
  6. [Das et al., 2019] Srijan Das, Rui Dai, Michal Koperski, Luca Minciullo, Lorenzo Garattoni, Francois Bremond, and Gianpiero Francesca. Toyota smarthome: Real-world activities of daily living. In Proceedings of the IEEE/CVF international conference on computer vision, pages 833- 842, 2019.
  7. [Doshi and Yilmaz, 2022] Keval Doshi and Yasin Yilmaz. Federated learning-based driver activity recognition for edge devices. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 3338-3346, 2022.
  8. [Fan et al., 2021] Haoqi Fan, Bo Xiong, Karttikeya Man- galam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer. Multiscale vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6824-6835, 2021.
  9. [Fanì et al., 2024] Eros Fanì, Raffaello Camoriano, Barbara Caputo, and Marco Ciccone. Accelerating heterogeneous federated learning with closed-form classifiers. Proceed- ings of the International Conference on Machine Learn- ing, 2024.
  10. [Gautam et al., 2024] Vinay Gautam, Himani Maheshwari, Raj Gaurang Tiwari, Ambuj Kumar Agarwal, and Naresh Kumar Trivedi. Federated learning empowered vi- olence recognition in cctv footage: A yolo and resnet-50 fusion approach. In 2024 2nd International Conference on Intelligent Data Communication Technologies and Inter- net of Things (IDCIoT), pages 01-06. IEEE, 2024.
  11. [Gondara and Wang, 2020] Lovedeep Gondara and Ke Wang. Differentially private small dataset release using random projections. In Conference on Uncertainty in Artificial Intelligence, pages 639-648. PMLR, 2020.
  12. [Grammenos et al., 2020] Andreas Grammenos, Rodrigo Mendoza Smith, Jon Crowcroft, and Cecilia Mascolo. Federated principal component analysis. Advances in neu- ral information processing systems, 33:6453-6464, 2020.
  13. [Grativol et al., 2024] Lucas Grativol, Mathieu Leonardon, Guillaume Muller, Virginie Fresse, and Matthieu Arzel. Flocora: Federated learning compression with low-rank adaptation. In 2024 32nd European Signal Processing Conference (EUSIPCO), pages 1786-1790. IEEE, 2024.
  14. [Guo et al., 2024] Mingzhao Guo, Dongzhu Liu, Osvaldo Simeone, and Dingzhu Wen. Efficient wireless federated learning via low-rank gradient factorization. IEEE Trans- actions on Vehicular Technology, 2024.
  15. [Han et al., 2015] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015.
  16. [Hara et al., 2018] Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotemporal 3d cnns retrace the his- tory of 2d cnns and imagenet? In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 6546-6555, 2018.
  17. [He et al., 2020] Chaoyang He, Murali Annavaram, and Salman Avestimehr. Group knowledge transfer: Feder- ated learning of large cnns at the edge. Advances in neural information processing systems, 33:14068-14080, 2020.
  18. [Horvóth et al., 2022] Samuel Horvóth, Chen-Yu Ho, Lu- dovit Horvath, Atal Narayan Sahu, Marco Canini, and Pe- ter Richtárik. Natural compression for distributed deep learning. In Mathematical and Scientific Machine Learn- ing, pages 129-141. PMLR, 2022.
  19. [Ijaz et al., 2022] Momal Ijaz, Renato Diaz, and Chen Chen. Multimodal transformer for nursing activity recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2065-2074, 2022.
  20. [Jiang et al., 2022] Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K Leung, and Le- andros Tassiulas. Model pruning enables efficient feder- ated learning on edge devices. IEEE Transactions on Neu- ral Networks and Learning Systems, 34(12):10374-10386, 2022.
  21. [Kannan et al., 2014] Ravi Kannan, Santosh Vempala, and David Woodruff. Principal component analysis and higher correlations for distributed data. In Conference on Learn- ing Theory, pages 1040-1057. PMLR, 2014.
  22. [Kay et al., 2017] Will Kay, Joao Carreira, Karen Si- monyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017.
  23. [Kuehne et al., 2011] Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre. Hmdb: a large video database for human motion recogni- tion. In 2011 International conference on computer vision, pages 2556-2563. IEEE, 2011.
  24. [Luo et al., 2024] Zelun Luo, Yuliang Zou, Yijin Yang, Zane Durante, De-An Huang, Zhiding Yu, Chaowei Xiao, Li Fei-Fei, and Animashree Anandkumar. Differentially private video activity recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 6657-6667, 2024.
  25. [McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
  26. [Molchanov et al., 2019] Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz. Importance estimation for neural network pruning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11264-11272, 2019.
  27. [Narayanamurthy et al., 2022] Praneeth Narayanamurthy, Namrata Vaswani, and Aditya Ramamoorthy. Federated over-air subspace tracking from incomplete and cor- rupted data. IEEE Transactions on Signal Processing, 70:3906-3920, 2022.
  28. [Nken et al., 2025] Allassan Tchangmena A Nken, Susan McKeever, Peter Corcoran, and Ihsan Ullah. Video- dprp: A differentially private approach for visual privacy- preserving video human activity recognition. In Joint Eu- ropean Conference on Machine Learning and Knowledge Discovery in Databases, pages 345-362. Springer, 2025.
  29. [Rehman et al., 2022] Yasar Abbas Ur Rehman, Yan Gao, Ji- ajun Shen, Pedro Porto Buarque de Gusmao, and Nicholas Lane. Federated self-supervised learning for video under- standing. In European Conference on Computer Vision, pages 506-522. Springer, 2022.
  30. [Reisizadeh et al., 2020] Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani, Ali Jadbabaie, and Ramtin Pedarsani. Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization. In International conference on artificial intelligence and statistics, pages 2021-2031. PMLR, 2020.
  31. [Sanyal et al., 2018] Amartya Sanyal, Varun Kanade, Philip HS Torr, and Puneet K Dokania. Robustness via deep low-rank representations. arXiv preprint arXiv:1804.07090, 2018.
  32. [Soomro et al., 2012] Khurram Soomro, Amir Roshan Za- mir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012.
  33. [Tu et al., 2024] Nguyen Anh Tu, Assanali Abu, Nartay Aikyn, Nursultan Makhanov, Min-Ho Lee, Khiem Le- Huy, and Kok-Seng Wong. Fedfslar: A federated learning framework for few-shot action recognition. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 270-279, 2024.
  34. [Wu et al., 2024] Xinghao Wu, Xuefeng Liu, Jianwei Niu, Haolin Wang, Shaojie Tang, and Guogang Zhu. Fed- lora: When personalized federated learning meets low- rank adaptation. 2024.
  35. [Xu et al., 2023] Jian Xu, Xinyi Tong, and Shao-Lun Huang. Personalized federated learning with feature alignment and classifier collaboration. arXiv preprint arXiv:2306.11867, 2023.
  36. [Yang et al., 2022] Xiaoshan Yang, Baochen Xiong, Yi Huang, and Changsheng Xu. Cross-modal federated human activity recognition via modality-agnostic and modality-specific representation learning. In Proceed- ings of the AAAI conference on artificial intelligence, volume 36, pages 3063-3071, 2022.
  37. [Yang et al., 2025] Yiyuan Yang, Guodong Long, Qinghua Lu, Liming Zhu, Jing Jiang, and Chengqi Zhang. Feder- ated low-rank adaptation for foundation models: A survey. arXiv preprint arXiv:2505.13502, 2025. IJCAI

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards