{"type":"ScholarlyArticle","title":"Bridging Differential Privacy and Byzantine-Robustness via Model Aggregation","authors":[{"name":"Heng Zhu","slug":"heng-zhu","given":"Heng","family":"Zhu","affiliation":"University of California , San Diego"},{"name":"Qing Ling","slug":"qing-ling","given":"Qing","family":"Ling","affiliation":"sen University"}],"abstract":"This paper aims at jointly addressing two seemly conflicting issues in federated learning: differential privacy (DP) and Byzantine-robustness, which are particularly challenging when the distributed data are non-i.i.d. (independent and identically distributed). The standard DP mechanisms add noise to the transmitted messages, and entangles with robust stochastic gradient aggregation to defend against Byzantine attacks. In this paper, we decouple the two issues via robust stochastic model aggregation, in the sense that  our proposed DP mechanisms and the defense against Byzantine attacks have separated influence on the learning performance. Leveraging robust stochastic model aggregation, at each iteration, each worker calculates the difference between the local model and the global one, followed by sending the element-wise signs to the master node, which enables robustness to Byzantine attacks. Further, we design two DP mechanisms to perturb the uploaded signs for the purpose of privacy preservation, and prove that they are (epsilon,0)-DP by exploiting the properties of noise distributions. With the tools of Moreau envelop and proximal point projection, we establish the convergence of the proposed algorithm when the cost function is nonconvex. We analyze the trade-off between privacy preservation and learning performance, and show that the influence of our proposed DP mechanisms is decoupled with that of robust stochastic model aggregation. Numerical experiments demonstrate the effectiveness of the proposed algorithm.","doi":"10.24963/ijcai.2022/337","url":"https://ijcai.org/proceedings/2022/337","pdf":"https://ijcai.org/proceedings/2022/0337.pdf","markdown":"https://ijcai.org/proceedings/2022/337.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/337","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":337,"pages":{"first":2427,"last":2433},"keywords":["Machine Learning: Robustness","AI Ethics, Trust, Fairness: Safety & Robustness","Data Mining: Federated Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986934","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985077","length_sec":353,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martin Abadi, Andy Chu, Ian Goodfel- low, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In ACM SIGSAC Conference on Computer and Communications Security, pages 308-318, 2016.","title":"Deep learning with differential privacy","authors":"Martin Abadi, Andy Chu, Ian Goodfellow, Brendan Mcmahan, Ilya Mironov, Kunal Talwar, Li Zhang","year":"2016","venue":"ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agarwal et al., 2018] Naman Agarwal, Ananda Theertha Suresh, Felix Yu, Sanjiv Kumar, and Brendan Mcmahan. cpSGD: Communication-efficient and differentially-private distributed SGD. arXiv preprint arXiv:1805.10559, 2018.","title":"cpSGD: Communication-efficient and differentially-private distributed SGD","authors":"Naman Agarwal, Ananda Theertha Suresh, Felix Yu, Sanjiv Kumar, Brendan Mcmahan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blanchard et al., 2017] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, pages 119-129, 2017.","title":"Machine learning with adversaries: Byzantine tolerant gradient descent","authors":"Blanchard","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017] Yudong Chen, Lili Su, and Jiaming Xu. Distributed statistical machine learning in adversarial set- tings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Sys- tems, 1(2):1-25, 2017.","title":"Distributed statistical machine learning in adversarial settings: Byzantine gradient descent","authors":"Yudong Chen, Lili Su, Jiaming Xu","year":"2017","venue":"Proceedings of the ACM on Measurement and Analysis of Computing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Davis and Drusvyatskiy, 2019] Damek Davis and Dmitriy Drusvyatskiy. Stochastic model-based minimization of weakly convex functions. SIAM Journal on Optimization, 29(1):207-239, 2019.","title":"Damek Davis and Dmitriy Drusvyatskiy. Stochastic model-based minimization of weakly convex functions","authors":"Drusvyatskiy Davis","year":"2019","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Davis and Grimmer, 2019] Damek Davis and Benjamin Grimmer. Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems. SIAM Jour- nal on Optimization, 29(3):1908-1930, 2019.","title":"Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems","authors":"Grimmer ; Davis, Damek Davis, Benjamin Grimmer","year":"2019","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dwork and Roth, 2014] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foun- dations and Trends® in Theoretical Computer Science, 9(3):211-407, 2014.","title":"The algorithmic foundations of differential privacy","authors":"Roth ; Dwork, Cynthia Dwork, Aaron Roth","year":"2014","venue":"Foundations and Trends® in Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guerraoui et al., 2021a] Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, Sebastien Rouault, and John Stephan. Com- bining differential privacy and byzantine resilience in dis- tributed SGD. arXiv preprint arXiv:2110.03991, 2021.","title":"Combining differential privacy and byzantine resilience in distributed SGD","authors":"Guerraoui","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guerraoui et al., 2021b] Rachid Guerraoui, Nirupam Gupta, Rafaël Pinot, Sébastien Rouault, and John Stephan. Differential privacy and byzantine resilience in sgd: Do they add up? arXiv preprint arXiv:2102.08166, 2021.","title":"Differential privacy and byzantine resilience in sgd: Do they add up","authors":"Guerraoui","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jin et al., 2020] Richeng Jin, Yufan Huang, Xiaofan He, Tianfu Wu, and Huaiyu Dai. Stochastic-sign sgd for fed- erated learning with theoretical guarantees. arXiv preprint arXiv:2002.10940, 2020.","title":"Stochastic-sign sgd for federated learning with theoretical guarantees","authors":"Jin","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kairouz and McMahan, 2021] Peter Kairouz and Brendan McMahan. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1):1-210, 2021.","title":"Advances and open problems in federated learning","authors":"Mcmahan Kairouz, Peter Kairouz, Brendan Mcmahan","year":"2021","venue":"Foundations and Trends® in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kairouz et al., 2015] Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The composition theorem for differ- ential privacy. In International Conference on Machine Learning, pages 1376-1385, 2015.","title":"The composition theorem for differential privacy","authors":"Peter Kairouz, Sewoong Oh, Pramod Viswanath","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Karimireddy et al., 2020] Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Learning from history for Byzantine robust optimization. arXiv preprint arXiv:2012.10333, 2020.","title":"Sai Praneeth Karimireddy","authors":"Karimireddy","year":"2020","venue":"Learning from history for Byzantine robust optimization","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Konečnỳ et al., 2016] Jakub Konečnỳ, Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016.","title":"Federated learning: Strategies for improving communication efficiency","authors":"Jakub Konečnỳ, Brendan Mcmahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, Dave Bacon","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lamport et al., 1982] Leslie Lamport, Robert E. Shostak, and Marshall C. Pease. The Byzantine generals problem. ACM Transactions on Programming Languages and Sys- tems, 4(3):382-401, 1982.","title":"The Byzantine generals problem","authors":"Leslie Lamport, Robert E Shostak, Marshall C Pease","year":"1982","venue":"ACM Transactions on Programming Languages and Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2019] Liping Li, Wei Xu, Tianyi Chen, Geor- gios B Giannakis, and Qing Ling. RSA: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets. In AAAI Conference on Ar- tificial Intelligence, pages 1544-1551, 2019.","title":"RSA: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets","authors":"Liping Li, Wei Xu, Tianyi Chen, Georgios B Giannakis, Qing Ling","year":"2019","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mhamdi et al., 2018] El Mahdi El Mhamdi, Rachid Guer- raoui, and Sébastien Rouault. The hidden vulnerability of distributed learning in Byzantium. In International Con- ference on Machine Learning, pages 3521-3530, 2018.","title":"El Mahdi El Mhamdi, Rachid Guerraoui, and Sébastien Rouault. The hidden vulnerability of distributed learning in Byzantium","authors":"Mhamdi","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mironov, 2017] Ilya Mironov. Rényi differential privacy. In IEEE Computer Security Foundations Symposium, pages 263-275, 2017.","title":"Rényi differential privacy","authors":"Ilya Mironov","year":"2017","venue":"IEEE Computer Security Foundations Symposium","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pinelis, 2019] Iosif Pinelis. Exact bounds on the inverse Mill's ratio and its derivatives. Complex Analysis and Op- erator Theory, 13(4):1643-1651, 2019.","title":"Exact bounds on the inverse Mill's ratio and its derivatives","authors":"Iosif Pinelis","year":"2019","venue":"Complex Analysis and Operator Theory","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sampford, 1953] Michael R Sampford. Some inequalities on Mill's ratio and related functions. The Annals of Math- ematical Statistics, 24(1):130-132, 1953.","title":"Some inequalities on Mill's ratio and related functions","authors":"Michael R Sampford","year":"1953","venue":"The Annals of Mathematical Statistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2013] Shuang Song, Kamalika Chaudhuri, and Anand Sarwate. Stochastic gradient descent with differ- entially private updates. In IEEE Global Conference on Signal and Information Processing, pages 245-248, 2013.","title":"Stochastic gradient descent with differentially private updates","authors":"Shuang Song, Kamalika Chaudhuri, Anand Sarwate","year":"2013","venue":"IEEE Global Conference on Signal and Information Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wei et al., 2020] Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony Quek, and Vincent Poor. Federated learning with differential pri- vacy: Algorithms and performance analysis. IEEE Trans- actions on Information Forensics and Security, 15:3454- 3469, 2020.","title":"Federated learning with differential privacy: Algorithms and performance analysis","authors":"Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Jin Shi, Tony Quek, Vincent Poor","year":"2020","venue":"IEEE Transactions on Information Forensics and Security","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2020] Zhaoxian Wu, Qing Ling, Tianyi Chen, and Georgios B Giannakis. Federated variance-reduced stochastic gradient descent with robustness to Byzan- tine attacks. IEEE Transactions on Signal Processing, 68:4583-4596, 2020.","title":"Federated variance-reduced stochastic gradient descent with robustness to Byzantine attacks","authors":"Zhaoxian Wu, Qing Ling, Tianyi Chen, Georgios B Giannakis","year":"2020","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yin et al., 2018] Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In Interna- tional Conference on Machine Learning, pages 5650- 5659, 2018.","title":"Byzantine-robust distributed learning: Towards optimal statistical rates","authors":"Dong Yin, Yudong Chen, Ramchandran Kannan, Peter Bartlett","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu and Ling, 2022] Heng Zhu and Qing Ling. Bridging differential privacy and Byzantine-robustness via model aggregation. arXiv preprint, 2022.","title":"Bridging differential privacy and Byzantine-robustness via model aggregation","authors":"Heng Zhu, Qing Ling","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}