{"type":"ScholarlyArticle","title":"Practical Hybrid Gradient Compression for Federated Learning Systems","authors":[{"name":"Sixu Hu","slug":"sixu-hu","given":"Sixu","family":"Hu","affiliation":"National University of Singapore"},{"name":"Linshan Jiang","slug":"linshan-jiang","given":"Linshan","family":"Jiang","affiliation":"National University of Singapore"},{"name":"Bingsheng He","slug":"bingsheng-he","given":"Bingsheng","family":"He","affiliation":"National University of Singapore"}],"abstract":"The high communication cost is a major challenge in the federated learning (FL) training process. Several methods have been proposed to reduce communication costs on the uplink channel, primarily sparsification-based methods, which have overlooked the impact of downlink channels. However, model accuracy and communication cost issues arise when applying them in practical FL applications, especially when the bandwidth is limited both on the uplink and downlink channels. In this paper, we propose a novel secure-FL-compatible hybrid gradient compression framework (HGC) that handles both uplink and downlink communication. Specifically, HGC identifies and exploits three types of redundancies in the FL training process. With proposed optimization methods based on compression ratio correction and dynamic momentum correction, HGC improves the trade-off between communication cost and model performance. The extensive theoretical and empirical analysis demonstrates the effectiveness of our framework in achieving a high compression ratio for both uplink and downlink communications with negligible loss of model accuracy, surpassing the state-of-the-art compression methods.","doi":"10.24963/ijcai.2024/458","url":"https://ijcai.org/proceedings/2024/458","pdf":"https://ijcai.org/proceedings/2024/0458.pdf","markdown":"https://ijcai.org/proceedings/2024/458.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/458","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":458,"pages":{"first":4147,"last":4155},"keywords":["Machine Learning: ML: Federated learning"],"videos":[],"references":[{"position":1,"raw":"[Abdi and Fekri, 2019] Afshin Abdi and Faramarz Fekri. Nested dithered quantization for communication reduction in distributed training. arXiv:1904.01197, 2019.","title":"Nested dithered quantization for communication reduction in distributed training","authors":"Afshin Abdi, Faramarz Fekri","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aji and Heafield, 2017] Alham Fikri Aji and Kenneth Heafield. Sparse Communication for Distributed Gradient Descent. In Proceedings of the 2017 Conference on Empir- ical Methods in Natural Language Processing, pages 440- 445. Association for Computational Linguistics, 2017.","title":"Sparse Communication for Distributed Gradient Descent","authors":"Heafield ; Aji, Fikri Alham, Kenneth Aji, Heafield","year":"2017","venue":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Alistarh et al., 2017] Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Pro- cessing Systems 30, pages 1709-1720.","title":"QSGD: Communication-efficient SGD via gradient quantization and encoding","authors":"Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, Milan Vojnovic","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Alistarh et al., 2018] Dan Alistarh, Torsten Hoefler, Mikael Johansson, Sarit Khirirat, Nikola Konstantinov, and Cédric Renggli. The convergence of sparsified gradient methods. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pages 5977-5987.","title":"The convergence of sparsified gradient methods","authors":"Dan Alistarh, Torsten Hoefler, Mikael Johansson, Sarit Khirirat, Nikola Konstantinov, Cédric Renggli","year":null,"venue":"Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Basu et al., 2020] Debraj Basu, Deepesh Data, Can Karakus, and Suhas N. Diggavi. Qsparse-local-SGD: Distributed SGD with quantization, sparsification, and local computations. IEEE Journal on Selected Areas in Information Theory, 1(1):217-226, 2020.","title":"Qsparse-local-SGD: Distributed SGD with quantization, sparsification, and local computations","authors":"Debraj Basu, Deepesh Data, Can Karakus, Suhas N Diggavi","year":"2020","venue":"IEEE Journal on Selected Areas in Information Theory","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bernstein et al., 2018] Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, and Animashree Anandku- mar. signSGD: Compressed Optimisation for Non-Convex Problems. In Proceedings of the 35th International Con- ference on Machine Learning, pages 560-569. PMLR, 2018.","title":"signSGD: Compressed Optimisation for Non-Convex Problems","authors":"Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, Animashree Anandkumar","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Caldas et al., 2018] Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv:1812.01097, 2018.","title":"Leaf: A benchmark for federated settings","authors":"Sebastian Caldas, Sai Meher, Karthik Duddu, Peter Wu, Tian Li, Jakub Konečnỳ, Brendan Mcmahan, Virginia Smith, Ameet Talwalkar","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2018] Chia-Yu Chen, Jungwook Choi, Daniel Brand, Ankur Agrawal, Wei Zhang, and Kailash Gopalakr- ishnan. AdaComp : Adaptive residual gradient compres- sion for data-parallel distributed training. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018.","title":"AdaComp : Adaptive residual gradient compression for data-parallel distributed training","authors":"Chia-Yu Chen, Jungwook Choi, Daniel Brand, Ankur Agrawal, Wei Zhang, Kailash Gopalakrishnan","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chen et al., 2020] Chia-Yu Chen, Jiamin Ni, Songtao Lu, Xiaodong Cui, Pin-Yu Chen, Xiao Sun, Naigang Wang, Swagath Venkataramani, Vijayalakshmi (Viji) Srinivasan, Wei Zhang, and Kailash Gopalakrishnan. ScaleCom: Scal- able sparsified gradient compression for communication- efficient distributed training. volume 33, pages 13551- 13563.","title":"ScaleCom: Scalable sparsified gradient compression for communicationefficient distributed training","authors":"Chia-Yu Chen, Jiamin Ni, Songtao Lu, Xiaodong Cui, Pin-Yu Chen, Xiao Sun, Naigang Wang, Swagath Venkataramani, ( Vijayalakshmi, ) Viji, Wei Srinivasan, Kailash Zhang, Gopalakrishnan","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Deutsch, 1996] L. Peter Deutsch. DEFLATE Compressed Data Format Specification. Request for Comments RFC 1951, Internet Engineering Task Force, 1996.","title":"DEFLATE Compressed Data Format Specification","authors":"L Peter Deutsch","year":"1996","venue":"Request for Comments RFC 1951","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Golomb, 1966] Solomon Golomb. Run-Length Encodings (Corresp.). IEEE Transactions on Information Theory, 12(3):399-401, 1966.","title":"Solomon Golomb. Run-Length Encodings (Corresp.)","authors":"Golomb","year":"1966","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778. IEEE, 2016.","title":"Deep Residual Learning for Image Recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ivkin et al., 2019] Nikita Ivkin, Daniel Rothchild, Enayat Ullah, Vladimir Braverman, Ion Stoica, and Raman Arora. Communication-efficient distributed SGD with sketching. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, number 1178, pages 13142-13152.","title":"Communication-efficient distributed SGD with sketching","authors":"Nikita Ivkin, Daniel Rothchild, Enayat Ullah, Vladimir Braverman, Ion Stoica, Raman Arora","year":"1178","venue":"Proceedings of the 33rd International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jiang et al., 2018] Jiawei Jiang, Fangcheng Fu, Tong Yang, and Bin Cui. SketchML: Accelerating Distributed Ma- chine Learning with Data Sketches. In Proceedings of the 2018 International Conference on Management of Data, SIGMOD '18, pages 1269-1284. Association for Comput- ing Machinery, 2018.","title":"SketchML: Accelerating Distributed Machine Learning with Data Sketches","authors":"Jiawei Jiang, Fangcheng Fu, Tong Yang, Bin Cui","year":"2018","venue":"Proceedings of the 2018 International Conference on Management of Data, SIGMOD '18","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Karimireddy et al., 2019] Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian Stich, and Martin Jaggi. Error Feedback Fixes SignSGD and other Gradient Compres- sion Schemes. In Proceedings of the 36th International Conference on Machine Learning, pages 3252-3261.","title":"Error Feedback Fixes SignSGD and other Gradient Compression Schemes","authors":"Praneeth Sai, Quentin Karimireddy, Sebastian Rebjock, Martin Stich, Jaggi","year":null,"venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"PMLR, 2019.","title":null,"authors":"Pmlr","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kingma and Ba, 2017] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1412.6980, 2017.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. Technical report, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2020] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, volume 2, pages 429-450, 2020.","title":"Federated optimization in heterogeneous networks","authors":"Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith","year":"2020","venue":"Proceedings of Machine Learning and Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2022] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-IID data silos: An experimental study. In IEEE International Conference on Data Engineering, 2022.","title":"Federated learning on non-IID data silos: An experimental study","authors":"Li","year":"2022","venue":"IEEE International Conference on Data Engineering","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lin et al., 2018] Yujun Lin, Song Han, Huizi Mao, Yu Wang, and Bill Dally. Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training. In International Conference on Learning Repre- sentations, 2018.","title":"Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training","authors":"Yujun Lin, Song Han, Huizi Mao, Yu Wang, Bill Dally","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Marcus et al., 1993] Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330, 1993.","title":"Building a large annotated corpus of English: The Penn Treebank","authors":"Mitchell P Marcus, Beatrice Santorini, Mary Ann Marcinkiewicz","year":"1993","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th Inter- national Conference on Artificial Intelligence and Statis- tics, volume 54 of Proceedings of Machine Learning Re- search, pages 1273-1282. PMLR, 2017.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera Y Arcas","year":"2017","venue":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Phansalkar and Sastry, 1994] Vijay V Phansalkar and P Shanti Sastry. Analysis of the back-propagation algo- rithm with momentum. IEEE Transactions on Neural Networks, 5(3):505-506, 1994.","title":"Analysis of the back-propagation algorithm with momentum","authors":"V Vijay, P Shanti Phansalkar, Sastry","year":"1994","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Philippenko and Dieuleveut, 2022] Constantin Philippenko and Aymeric Dieuleveut. Bidirectional compression in heterogeneous settings for distributed or federated learn- ing with partial participation: Tight convergence guaran- tees. arXiv:2006.14591, 2022.","title":"Bidirectional compression in heterogeneous settings for distributed or federated learning with partial participation: Tight convergence guarantees","authors":"Constantin Philippenko, Aymeric Dieuleveut","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Press and Wolf, 2017] Ofir Press and Lior Wolf. Us- ing the output embedding to improve language models. arXiv:1608.05859, 2017.","title":"Using the output embedding to improve language models","authors":null,"year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Qian, 1999] Ning Qian. On the momentum term in gradient descent learning algorithms. Neural Networks, 12(1):145- 151, 1999.","title":"On the momentum term in gradient descent learning algorithms","authors":"Ning Qian","year":"1999","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sattler et al., 2019] Felix Sattler, Simon Wiedemann, Klaus-Robert Muller, and Wojciech Samek. Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication. In 2019 International Joint Conference on Neural Networks, pages 1-8. IEEE, 2019.","title":"Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication","authors":"Felix Sattler, Simon Wiedemann, Klaus-Robert Muller, Wojciech Samek","year":"2019","venue":"2019 International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sattler et al., 2020] Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data. IEEE Transactions on Neural Networks and Learning Systems, 31(9):14, 2020.","title":"Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data","authors":"Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, Wojciech Samek","year":"2020","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Seide et al., 2014] Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu. 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs. In Interspeech 2014, pages 1058-1062. ISCA, 2014.","title":"1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs","authors":"Frank Seide, Hao Fu, Jasha Droppo, Gang Li, Dong Yu","year":"2014","venue":"Interspeech 2014","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Stich et al., 2018] Sebastian U Stich, Jean-Baptiste Cordon- nier, and Martin Jaggi. Sparsified SGD with memory. In Advances in Neural Information Processing Systems, vol- ume 31.","title":"Sparsified SGD with memory","authors":"Jean-Baptiste Sebastian U Stich, Martin Cordonnier, Jaggi","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"Curran Associates, Inc., 2018.","title":null,"authors":"Curran Associates, Inc","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Strom, 2015] Nikko Strom. Scalable distributed DNN train- ing using commodity GPU cloud computing. In Sixteenth Annual Conference of the International Speech Communi- cation Association, 2015.","title":"Scalable distributed DNN training using commodity GPU cloud computing","authors":"Nikko Strom","year":"2015","venue":"Sixteenth Annual Conference of the International Speech Communication Association","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Tang et al., 2019] Hanlin Tang, Chen Yu, Xiangru Lian, Tong Zhang, and Ji Liu. DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-pass Error- Compensated Compression. In Proceedings of the 36th International Conference on Machine Learning, pages 6155-6165. PMLR, 2019.","title":"DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-pass Error-Compensated Compression","authors":"Hanlin Tang, Chen Yu, Xiangru Lian, Tong Zhang, Ji Liu","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Tang et al., 2023] Zhenheng Tang, Shaohuai Shi, Bo Li, and Xiaowen Chu. GossipFL: A decentralized federated learn- ing framework with sparsified and adaptive communica- tion. IEEE Transactions on Parallel and Distributed Sys- tems, 34(3):909-922, 2023.","title":"GossipFL: A decentralized federated learning framework with sparsified and adaptive communica-tion","authors":"Tang","year":"2023","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2020] Jun-Kun Wang, Chi-Heng Lin, and Ja- cob Abernethy. Escaping Saddle Points Faster with Stochastic Momentum. In International Conference on Learning Representations, 2020.","title":"Escaping Saddle Points Faster with Stochastic Momentum","authors":"Wang","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wen et al., 2017] Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. TernGrad: Ternary gradients to reduce communication in distributed deep learning. In Advances in Neural Information Pro- cessing Systems 30, pages 1509-1519.","title":"TernGrad: Ternary gradients to reduce communication in distributed deep learning","authors":"Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, Hai Li","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Witten et al., 1987] Ian H. Witten, Radford M. Neal, and John G. Cleary. Arithmetic coding for data compression. Commun. ACM, 30(6):520-540, 1987.","title":"Arithmetic coding for data compression","authors":"Ian H Witten, M Radford, John G Neal, Cleary","year":"1987","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Wu et al., 2018] Jiaxiang Wu, Weidong Huang, Junzhou Huang, and Tong Zhang. Error Compensated Quantized SGD and its Applications to Large-scale Distributed Op- timization. In Proceedings of the 35th International Con- ference on Machine Learning, pages 5325-5333. PMLR, 2018.","title":"Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization","authors":"Jiaxiang Wu, Weidong Huang, Junzhou Huang, Tong Zhang","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wu et al., 2022] Donglei Wu, Xiangyu Zou, Shuyu Zhang, Haoyu Jin, Wen Xia, and Binxing Fang. SmartIdx: Reduc- ing communication cost in federated learning by exploit- ing the CNNs structures. Proceedings of the AAAI Confer- ence on Artificial Intelligence, 36(4):4254-4262, 2022.","title":"SmartIdx: Reducing communication cost in federated learning by exploiting the CNNs structures","authors":"Donglei Wu, Xiangyu Zou, Shuyu Zhang, Haoyu Jin, Wen Xia, Binxing Fang","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Xu et al., 2021] Hang Xu, Kelly Kostopoulou, Aritra Dutta, Xin Li, Alexandros Ntoulas, and Panos Kalnis. DeepRe- duce: A Sparse-tensor Communication Framework for Federated Deep Learning. In Advances in Neural Informa- tion Processing Systems, volume 34, pages 21150-21163.","title":"DeepReduce: A Sparse-tensor Communication Framework for Federated Deep Learning","authors":"Hang Xu, Kelly Kostopoulou, Aritra Dutta, Xin Li, Alexandros Ntoulas, Panos Kalnis","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"Curran Associates, Inc., 2021.","title":null,"authors":"Curran Associates, Inc","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Yang et al., 2019] Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learn- ing. Synthesis Lectures on Artificial Intelligence and Ma- chine Learning, 13(3):1-207, 2019.","title":"BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning","authors":"Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, Han Yu","year":"2019","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Yu et al., 2019] Yue Yu, Jiaxiang Wu, and Junzhou Huang. Exploring Fast and Communication-Efficient Algorithms in Large-Scale Distributed Networks. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, pages 674-683. PMLR, 2019.","title":"Exploring Fast and Communication-Efficient Algorithms in Large-Scale Distributed Networks","authors":"Yue Yu, Jiaxiang Wu, Junzhou Huang","year":"2019","venue":"Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yue et al., 2022] Kai Yue, Richeng Jin, Chau-Wai Wong, and Huaiyu Dai. Communication-efficient federated learn- ing via predictive coding. IEEE Journal of Selected Topics in Signal Processing, 16(3):369-380, 2022.","title":"Communication-efficient federated learning via predictive coding","authors":"Kai Yue, Richeng Jin, Chau-Wai Wong, Huaiyu Dai","year":"2022","venue":"IEEE Journal of Selected Topics in Signal Processing","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhang et al., 2020] Xiongtao Zhang, Xiaomin Zhu, Ji Wang, Hui Yan, Huangke Chen, and Weidong Bao. Federated Learning with Adaptive Communication Compression Under Dynamic Bandwidth and Unreliable Networks. Information Sciences, 540:242-262, 2020.","title":"Federated Learning with Adaptive Communication Compression Under Dynamic Bandwidth and Unreliable Networks","authors":"Zhang","year":"2020","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zheng et al., 2019] Shuai Zheng, Ziyue Huang, and James Kwok. Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback. In Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019.","title":"Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback","authors":"Shuai Zheng, Ziyue Huang, James Kwok","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}