{"type":"ScholarlyArticle","title":"Scalable Federated Unlearning via Isolated and Coded Sharding","authors":[{"name":"Yijing Lin","slug":"yijing-lin","given":"Yijing","family":"Lin","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications; The Future Network of Intelligence Institute , The Chinese University of Hong Kong (Shenzhen )"},{"name":"Zhipeng Gao","slug":"zhipeng-gao","given":"Zhipeng","family":"Gao","affiliation":"State Key Laboratory of Networking and Switching Technology , Beijing University of Posts and Telecommunications"},{"name":"Hongyang Du","slug":"hongyang-du","given":"Hongyang","family":"Du","affiliation":"College of Computing and Data Science , Nanyang Technological University"},{"name":"Dusit Niyato","slug":"dusit-niyato","given":"Dusit","family":"Niyato","affiliation":"College of Computing and Data Science , Nanyang Technological University"},{"name":"Gui Gui","slug":"gui-gui","given":"Gui","family":"Gui","affiliation":"School of Automation , Central South University"},{"name":"Shuguang Cui","slug":"shuguang-cui","given":"Shuguang","family":"Cui","affiliation":"School of Science and Engineering , The Chinese University of Hong Kong (Shenzhen ); The Future Network of Intelligence Institute , The Chinese University of Hong Kong (Shenzhen )"},{"name":"Jinke Ren","slug":"jinke-ren","given":"Jinke","family":"Ren","affiliation":"The Future Network of Intelligence Institute , The Chinese University of Hong Kong (Shenzhen ); School of Science and Engineering , The Chinese University of Hong Kong (Shenzhen )"}],"abstract":"Federated unlearning has emerged as a promising paradigm to erase the client-level data effect without affecting the performance of collaborative learning models. However, the federated unlearning process often introduces extensive storage overhead and consumes substantial computational resources, thus hindering its implementation in practice. To address this issue, this paper proposes a scalable federated unlearning framework based on isolated sharding and coded computing. We first divide distributed clients into multiple isolated shards across stages to reduce the number of clients being affected. Then, to reduce the storage overhead of the central server, we develop a coded computing mechanism by compressing the model parameters across different shards. In addition, we provide the theoretical analysis of time efficiency and storage effectiveness for the isolated and coded sharding. Finally, extensive experiments on two typical learning tasks, i.e., classification and generation, demonstrate that our proposed framework can achieve better performance than three state-of-the-art frameworks in terms of accuracy, retraining time, storage overhead, and F1 scores for resisting membership inference attacks.","doi":"10.24963/ijcai.2024/503","url":"https://ijcai.org/proceedings/2024/503","pdf":"https://ijcai.org/proceedings/2024/0503.pdf","markdown":"https://ijcai.org/proceedings/2024/503.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/503","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":503,"pages":{"first":4551,"last":4559},"keywords":["Machine Learning: ML: Federated learning","Machine Learning: ML: Trustworthy machine learning"],"videos":[],"references":[{"position":1,"raw":"[Bourtoule et al., 2021] Lucas Bourtoule, Varun Chan- drasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pages 141-159. IEEE, 2021.","title":"Machine unlearning","authors":"Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, Nicolas Papernot","year":"2021","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen and Yang, 2023] Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning for llms. In EMNLP 2023-Empirical Methods in Natural Language Processing, 2023.","title":"Unlearn what you want to forget: Efficient unlearning for llms","authors":"Chen, Yang ; ] Jiaao Chen, Diyi Yang","year":"2023","venue":"EMNLP 2023-Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Jang et al., 2022] Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating pri- vacy risks in language models. 2022.","title":"Knowledge unlearning for mitigating privacy risks in language models","authors":"Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, Minjoon Seo","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kurmanji et al., 2023] Meghdad Kurmanji, Peter Triantafil- lou, and Eleni Triantafillou. Towards unbounded machine unlearning. arXiv preprint arXiv:2302.09880, 2023.","title":"Towards unbounded machine unlearning","authors":"Kurmanji","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Li et al., 2020] Songze Li, Mingchao Yu, Chien-Sheng Yang, Amir Salman Avestimehr, Sreeram Kannan, and Pramod Viswanath. Polyshard: Coded sharding achieves linearly scaling efficiency and security simultaneously. IEEE Transactions on Information Forensics and Security, 16:249-261, 2020.","title":"Polyshard: Coded sharding achieves linearly scaling efficiency and security simultaneously","authors":"Li","year":"2020","venue":"IEEE Transactions on Information Forensics and Security","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2021] Gaoyang Liu, Xiaoqiang Ma, Yang Yang, Chen Wang, and Jiangchuan Liu. Federaser: Enabling ef- ficient client-level data removal from federated learning models. In 2021 IEEE/ACM 29th International Sympo- sium on Quality of Service (IWQOS), pages 1-10. IEEE, 2021.","title":"Federaser: Enabling efficient client-level data removal from federated learning models","authors":"Liu","year":"2021","venue":"IEEE/ACM 29th International Symposium on Quality of Service (IWQOS)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2022] Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, and Bo Li. The right to be forgotten in feder- ated learning: An efficient realization with rapid retrain- ing. In IEEE INFOCOM 2022-IEEE Conference on Com- puter Communications, pages 1749-1758. IEEE, 2022.","title":"The right to be forgotten in federated learning: An efficient realization with rapid retraining","authors":"Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, Bo Li","year":"2022","venue":"IEEE INFOCOM 2022-IEEE Conference on Computer Communications","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera Y Arcas","year":"2017","venue":"Artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Roth, 2006] Ron M Roth. Introduction to coding theory. IET Communications, 47(18-19):4, 2006.","title":"Introduction to coding theory","authors":"Ron M Roth","year":"2006","venue":"IET Communications","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Salton and Buckley, 1988] Gerard Salton and Christopher Buckley. Term-weighting approaches in automatic text retrieval. Information processing & management, 24(5):513-523, 1988.","title":"Term-weighting approaches in automatic text retrieval","authors":"Buckley Salton, Gerard Salton, Christopher Buckley","year":"1988","venue":"Information processing & management","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Shokri et al., 2017] Reza Shokri, Marco Stronati, Con- gzheng Song, and Vitaly Shmatikov. Membership infer- ence attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3- 18. IEEE, 2017.","title":"Membership inference attacks against machine learning models","authors":"Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov","year":"2017","venue":"IEEE symposium on security and privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Su and Li, 2023] Ningxin Su and Baochun Li. Asyn- chronous federated unlearning. In IEEE INFOCOM 2023- IEEE Conference on Computer Communications, pages 1- 10. IEEE, 2023.","title":"Asynchronous federated unlearning","authors":"Li ; Su, Ningxin Su, Baochun Li","year":"2023","venue":"IEEE INFOCOM 2023-IEEE Conference on Computer Communications","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2020] Hao Wang, Zakhary Kaplan, Di Niu, and Baochun Li. Optimizing federated learning on non-iid data with reinforcement learning. In IEEE INFOCOM 2020- IEEE Conference on Computer Communications, pages 1698-1707. IEEE, 2020.","title":"Optimizing federated learning on non-iid data with reinforcement learning","authors":"Hao Wang, Zakhary Kaplan, Di Niu, Baochun Li","year":"2020","venue":"IEEE INFOCOM 2020-IEEE Conference on Computer Communications","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2022] Junxiao Wang, Song Guo, Xin Xie, and Heng Qi. Federated unlearning via class-discriminative pruning. In Proceedings of the ACM Web Conference 2022, pages 622-632, 2022.","title":"Federated unlearning via class-discriminative pruning","authors":"Wang","year":"2022","venue":"Proceedings of the ACM Web Conference 2022","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2023] Weiqi Wang, Zhiyi Tian, Chenhan Zhang, An Liu, and Shui Yu. Bfu: Bayesian federated unlearning with parameter self-sharing. In Proceedings of the 2023 ACM Asia Conference on Computer and Com- munications Security, pages 567-578, 2023.","title":"Bfu: Bayesian federated unlearning with parameter self-sharing","authors":"Weiqi Wang, Zhiyi Tian, Chenhan Zhang, An Liu, Shui Yu","year":"2023","venue":"Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2022] Leijie Wu, Song Guo, Junxiao Wang, Zi- cong Hong, Jie Zhang, and Yaohong Ding. Federated un- learning: Guarantee the right of clients to forget. IEEE Network, 36(5):129-135, 2022.","title":"Federated unlearning: Guarantee the right of clients to forget","authors":"Leijie Wu, Song Guo, Junxiao Wang, Zicong Hong, Jie Zhang, Yaohong Ding","year":"2022","venue":"IEEE Network","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2023] Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S Yu. Machine unlearning: A survey. ACM Computing Surveys, 56(1):1-36, 2023.","title":"Machine unlearning: A survey","authors":"Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, Philip S Yu","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yan et al., 2022] Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. Arcane: An efficient architecture for exact machine unlearning. In Proceedings of the Thirty-First International Joint Con- ference on Artificial Intelligence, IJCAI-22, pages 4006- 4013, 2022.","title":"Arcane: An efficient architecture for exact machine unlearning","authors":"Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, Xiaodong Lin","year":"2022","venue":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22","doi":null,"url":null,"internal_url":"/proceedings/2022/556"},{"position":24,"raw":"[Zhang et al., 2023] Lefeng Zhang, Tianqing Zhu, Haibin Zhang, Ping Xiong, and Wanlei Zhou. Fedrecovery: Dif- ferentially private machine unlearning for federated learn- ing frameworks. IEEE Transactions on Information Foren- sics and Security, 2023.","title":"Fedrecovery: Differentially private machine unlearning for federated learning frameworks","authors":"Zhang","year":"2023","venue":"IEEE Transactions on Information Forensics and Security","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}