{"type":"ScholarlyArticle","title":"Denial-of-Service or Fine-Grained Control: Towards Flexible Model Poisoning Attacks on Federated Learning","authors":[{"name":"Hangtao Zhang","slug":"hangtao-zhang","given":"Hangtao","family":"Zhang","affiliation":"Xiangtan University"},{"name":"Zeming Yao","slug":"zeming-yao","given":"Zeming","family":"Yao","affiliation":"Swinburne University of Technology"},{"name":"Leo Yu Zhang","slug":"leo-yu-zhang","given":"Leo Yu","family":"Zhang","affiliation":"Griffith University"},{"name":"Shengshan Hu","slug":"shengshan-hu","given":"Shengshan","family":"Hu","affiliation":"School of Cyber Science and Engineering , Huazhong University of Science and Technology; National Engineering Research Center for Big Data Technology and System; Services Computing Technology and System Lab; Hubei Key Laboratory of Distributed System Security; Hubei Engineering Research Center on Big Data Security"},{"name":"Chao Chen","slug":"chao-chen","given":"Chao","family":"Chen","affiliation":"RMIT University"},{"name":"Alan Liew","slug":"alan-liew","given":"Alan","family":"Liew","affiliation":"Griffith University"},{"name":"Zhetao Li","slug":"zhetao-li","given":"Zhetao","family":"Li","affiliation":"Xiangtan University"}],"abstract":"Federated learning (FL) is vulnerable to poisoning attacks, where adversaries corrupt the global aggregation results and cause denial-of-service (DoS). Unlike recent model poisoning attacks that optimize the amplitude of malicious perturbations along certain prescribed directions to cause DoS, we propose a flexible model poisoning attack (FMPA) that can achieve versatile attack goals. We consider a practical threat scenario where no extra knowledge about the FL system (e.g., aggregation rules or updates on benign devices) is available to adversaries. FMPA exploits the global historical information to construct an estimator that predicts the next round of the global model as a benign reference. It then fine-tunes the reference model to obtain the desired poisoned model with low accuracy and small perturbations. Besides the goal of causing DoS, FMPA can be naturally extended to launch a fine-grained controllable attack, making it possible to precisely reduce the global accuracy. Armed with precise control, malicious FL service providers can gain advantages over their competitors without getting noticed, hence opening a new attack surface in FL other than DoS. Even for the purpose of DoS, experiments show that FMPA significantly decreases the global accuracy, outperforming six state-of-the-art attacks.","doi":"10.24963/ijcai.2023/508","url":"https://ijcai.org/proceedings/2023/508","pdf":"https://ijcai.org/proceedings/2023/0508.pdf","markdown":"https://ijcai.org/proceedings/2023/508.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/508","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":508,"pages":{"first":4567,"last":4575},"keywords":["Machine Learning: ML: Adversarial machine learning","AI Ethics, Trust, Fairness: ETF: Safety and robustness","Machine Learning: ML: Federated learning"],"videos":[],"references":[{"position":1,"raw":"[Baruch et al., 2019] Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing de- fenses for distributed learning. In Advances in Neural In- formation Processing Systems, pages 8632-8642, 2019.","title":"A little is enough: Circumventing defenses for distributed learning","authors":"Baruch","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blanchard et al., 2017] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems.","title":"Machine learning with adversaries: Byzantine tolerant gradient descent","authors":"Blanchard","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown and Meyer, 1961] Robert G. Brown and Richard F. Meyer. The fundamental theorem of exponential smooth- ing. Operations Research, 1961.","title":"The fundamental theorem of exponential smoothing","authors":"Robert G Brown, Richard F Meyer","year":"1961","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao and Gong, 2023] Xiaoyu Cao and Neil Zhenqiang Gong. Mpaf: Model poisoning attacks to federated learn- ing based on fake clients. In Computer Vision and Pattern Recognition (CVPR), 2023.","title":"Mpaf: Model poisoning attacks to federated learning based on fake clients","authors":"Gong ; ] Xiaoyu Cao, Neil Cao, Gong Zhenqiang","year":"2023","venue":"Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cao et al., 2021] Xiaoyu Cao, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. Fltrust: Byzantine-robust federated learning via trust bootstrapping. In 28th Annual Network and Distributed System Security Symposium, NDSS 2021. The Internet Society, 2021.","title":"Fltrust: Byzantine-robust federated learning via trust bootstrapping","authors":"Xiaoyu Cao, Minghong Fang, Jia Liu, Neil Zhenqiang, Gong","year":"2021","venue":"28th Annual Network and Distributed System Security Symposium, NDSS 2021","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy, pages 39-57. IEEE, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"IEEE Symposium on Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cohen et al., 2017] Gregory Cohen, Saeed Afshar, Jonathan Tapson, van Schaik, et al. Emnist: Extending mnist to handwritten letters. In 2017 International Joint Con- ference on Neural Networks (IJCNN), pages 2921-2926, 2017.","title":"Emnist: Extending mnist to handwritten letters","authors":"Gregory Cohen, Saeed Afshar, Jonathan Tapson, Van Schaik","year":"2017","venue":"2017 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2020] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Local model poisoning at- tacks to byzantine-robust federated learning. In Proceed- ings of the 29th USENIX Conference on Security Sympo- sium. USENIX Association, 2020.","title":"Local model poisoning attacks to byzantine-robust federated learning","authors":"Fang","year":"2020","venue":"Proceedings of the 29th USENIX Conference on Security Symposium","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Karimireddy et al., 2021] Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Learning from history for byzantine robust optimization. In Proceedings of the 38th Interna- tional Conference on Machine Learning, 2021.","title":"Learning from history for byzantine robust optimization","authors":"Karimireddy","year":"2021","venue":"Proceedings of the 38th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lecun et al., 1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Y Lecun, L Bottou, Y Bengio, P Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2019] Liping Li, Wei Xu, Tianyi Chen, Geor- gios B. Giannakis, and Qing Ling. Rsa: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets. In Proceedings of the Thirty- Third AAAI Conference on Artificial Intelligence, 2019.","title":"Rsa: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets","authors":"Liping Li, Wei Xu, Tianyi Chen, Georgios B Giannakis, Qing Ling","year":"2019","venue":"Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[McMahan and Ramage, 2017] H. Brendan McMahan and Daniel Ramage. Federated learning: Col- laborative machine learning without centralized training data. https://ai.googleblog.com/2017/04/ federated-learning-collaborative.html, 2017. Accessed: 2022-08-14.","title":"Federated learning: Collaborative machine learning without centralized training data","authors":"H, Brendan Mcmahan, Daniel Ramage","year":"2017","venue":null,"doi":null,"url":"https://ai.googleblog.com/2017/04/federated-learning-collaborative.html","internal_url":null},{"position":14,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th Inter- national Conference on Artificial Intelligence and Statis- tics, AISTATS 2017, pages 1273-1282, 2017.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera Y Arcas","year":"2017","venue":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mhamdi et al., 2018] El Mahdi El Mhamdi, Rachid Guer- raoui, and Sébastien Rouault. The hidden vulnerability of distributed learning in byzantium. In Proceedings of the 35th International Conference on Machine Learning, pages 3518-3527, 2018.","title":"El Mahdi El Mhamdi, Rachid Guerraoui, and Sébastien Rouault. The hidden vulnerability of distributed learning in byzantium","authors":"Mhamdi","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Muñoz-González et al., 2019] Luis Muñoz-González, Ken- neth T. Co, and Emil C. Lupu. Byzantine-robust feder- ated machine learning through adaptive model averaging. CoRR, abs/1909.05125, 2019.","title":"Byzantine-robust federated machine learning through adaptive model averaging","authors":"Luis Muñoz-González, Kenneth T Co, Emil C Lupu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Panda et al., 2022] Ashwinee Panda, Saeed Mahloujifar, Arjun Nitin Bhagoji, Supriyo Chakraborty, and Prateek Mittal. Sparsefed: Mitigating model poisoning attacks in federated learning with sparsification. In Proceedings of The 25th International Conference on Artificial Intelli- gence and Statistics, 2022.","title":"Sparsefed: Mitigating model poisoning attacks in federated learning with sparsification","authors":"Panda","year":"2022","venue":"Proceedings of The 25th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Prakash and Avestimehr, 2020] Saurav Prakash and Amir Salman Avestimehr. Mitigating byzantine attacks in federated learning. arXiv: Distributed, Parallel, and Cluster Computing, 2020.","title":"Mitigating byzantine attacks in federated learning","authors":"Saurav Prakash, Amir Salman, Avestimehr","year":"2020","venue":"arXiv: Distributed, Parallel, and Cluster Computing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shejwalkar and Houmansadr, 2021] Virat Shejwalkar and Amir Houmansadr. Manipulating the byzantine: Opti- mizing model poisoning attacks and defenses for federated learning. In Network and Distributed System Security Sym- posium, 2021.","title":"Virat Shejwalkar and Amir Houmansadr. Manipulating the byzantine: Optimizing model poisoning attacks and defenses for federated learning","authors":"Houmansadr Shejwalkar","year":"2021","venue":"Network and Distributed System Security Symposium","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sun et al., 2019] Ziteng Sun, Peter Kairouz, and Ananda Theertha Suresh. Can you really backdoor federated learning? CoRR, abs/1911.07963, 2019.","title":"Can you really backdoor federated learning?","authors":"Sun","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, et al. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations ICLR, 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"2nd International Conference on Learning Representations ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tolpegin et al., 2020] Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, and Ling Liu. Data poisoning attacks against federated learning systems. In Proceed- ings of the 25th European Symposium on Research in Computer Security, ESORICS 2020, 2020.","title":"Data poisoning attacks against federated learning systems","authors":"Tolpegin","year":"2020","venue":"Proceedings of the 25th European Symposium on Research in Computer Security, ESORICS 2020","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wan et al., 2022] Wei Wan, Shengshan Hu, jianrong Lu, Leo Yu Zhang, Hai Jin, and Yuanyuan He. Shielding fed- erated learning: Robust aggregation with adaptive client selection. In Proceedings of the Thirty-First Interna- tional Joint Conference on Artificial Intelligence, IJCAI- 22, pages 753-760, 2022.","title":"Shielding federated learning: Robust aggregation with adaptive client selection","authors":"Wei Wan, Shengshan Hu, Leo Yu Lu, Hai Zhang, Yuanyuan Jin, He","year":"2022","venue":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22","doi":null,"url":null,"internal_url":"/proceedings/2022/106"},{"position":24,"raw":"[Wang et al., 2020] Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papailiopou- los. Attack of the tails: Yes, You really can backdoor federated learning. In Proceedings of the 34th Inter- national Conference on Neural Information Processing Systems, 2020.","title":"Attack of the tails: Yes, You really can backdoor federated learning","authors":"Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-Yong Sohn, Kangwook Lee, Dimitris Papailiopoulos","year":"2020","venue":"Proceedings of the 34th International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xia et al., 2019] Qi Xia, Zeyi Tao, Zijiang Hao, and Qun Li. Faba: An algorithm for fast aggregation against byzantine attacks in distributed neural networks. In Proceedings of the Twenty-Eighth International Joint Conference on Arti- ficial Intelligence, IJCAI-19, pages 4824-4830, 2019.","title":"Faba: An algorithm for fast aggregation against byzantine attacks in distributed neural networks","authors":"Qi Xia, Zeyi Tao, Zijiang Hao, Qun Li","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19","doi":null,"url":null,"internal_url":"/proceedings/2019/670"},{"position":26,"raw":"[Xie et al., 2018] Cong Xie, Oluwasanmi Koyejo, and In- dranil Gupta. Generalized byzantine-tolerant SGD. CoRR, abs/1802.10116, 2018.","title":"Generalized byzantine-tolerant SGD","authors":"Cong Xie, Oluwasanmi Koyejo","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xie et al., 2020] Cong Xie, Oluwasanmi Koyejo, and In- dranil Gupta. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation. In Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, pages 261-270, 2020.","title":"Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation","authors":"Cong Xie, Oluwasanmi Koyejo, Indranil Gupta","year":"2020","venue":"Proceedings of The 35th Uncertainty in Artificial Intelligence Conference","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yin et al., 2018] Dong Yin, Yudong Chen, Kannan Ram- chandran, and Peter L. Bartlett. Byzantine-robust dis- tributed learning: Towards optimal statistical rates. In Pro- ceedings of the 35th International Conference on Machine Learning, ICML 2018, pages 5636-5645, 2018.","title":"Byzantine-robust distributed learning: Towards optimal statistical rates","authors":"Dong Yin, Yudong Chen, Kannan Ramchandran, Peter L Bartlett","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning, ICML 2018","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Wu, 2020] Zekun Zhang and Tianfu Wu. Learn- ing ordered top-k adversarial attacks via adversarial distil- lation. In 2020 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition Workshops (CVPRW), pages 3364-3373, 2020.","title":"Learning ordered top-k adversarial attacks via adversarial distillation","authors":"Wu ; ] Zekun Zhang, Tianfu Zhang, Wu","year":"2020","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2022] Zaixi Zhang, Xiaoyu Cao, Jin Jia, and Neil Zhenqiang Gong. Fldetector: Defending federated learning against model poisoning attacks via detecting ma- licious clients. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022.","title":"Fldetector: Defending federated learning against model poisoning attacks via detecting malicious clients","authors":"Zhang","year":"2022","venue":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhou et al., 2021] X. Zhou, M. Xu, Y. Wu, and N. Zheng. Deep model poisoning attack on federated learning. Fu- ture Internet, 13(3):73, 2021.","title":"Deep model poisoning attack on federated learning","authors":"Zhou","year":"2021","venue":"Future Internet","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}