{"type":"ScholarlyArticle","title":"FedCG: Leverage Conditional GAN for Protecting Privacy and Maintaining Competitive Performance in Federated Learning","authors":[{"name":"Yuezhou Wu","slug":"yuezhou-wu","given":"Yuezhou","family":"Wu","affiliation":"Sun Yat-sen University , China; WeBank , China"},{"name":"Yan Kang","slug":"yan-kang","given":"Yan","family":"Kang","affiliation":"WeBank , China"},{"name":"Jiahuan Luo","slug":"jiahuan-luo","given":"Jiahuan","family":"Luo","affiliation":"WeBank , China"},{"name":"Yuanqin He","slug":"yuanqin-he","given":"Yuanqin","family":"He","affiliation":"WeBank , China"},{"name":"Lixin Fan","slug":"lixin-fan","given":"Lixin","family":"Fan","affiliation":"WeBank , China"},{"name":"Rong Pan","slug":"rong-pan","given":"Rong","family":"Pan","affiliation":"Sun Yat-sen University , China"},{"name":"Qiang Yang","slug":"qiang-yang","given":"Qiang","family":"Yang","affiliation":"WeBank , China; Hong Kong University of Science and Technology , China"}],"abstract":"Federated learning (FL) aims to protect data privacy by enabling clients to build machine learning models collaboratively without sharing their private data. Recent works demonstrate that information exchanged during FL is subject to gradient-based privacy attacks and, consequently, a variety of privacy-preserving methods have been adopted to thwart such attacks. However, these defensive methods either introduce orders of magnitudes more computational and communication overheads (e.g., with homomorphic encryption) or incur substantial model performance losses in terms of prediction accuracy (e.g., with differential privacy). In this work, we propose FEDCG, a novel federated learning method that leverages conditional generative adversarial networks to achieve high-level privacy protection while still maintaining competitive model performance. FEDCG decomposes each client's local network into a private extractor and a public classifier and keeps the extractor local to protect privacy. Instead of exposing extractors, FEDCG shares clients' generators with the server for aggregating clients' shared knowledge aiming to enhance the performance of each client's local networks. Extensive experiments demonstrate that FEDCG can achieve competitive model performance compared with FL baselines, and privacy analysis shows that FEDCG has a high-level privacy-preserving capability.","doi":"10.24963/ijcai.2022/324","url":"https://ijcai.org/proceedings/2022/324","pdf":"https://ijcai.org/proceedings/2022/0324.pdf","markdown":"https://ijcai.org/proceedings/2022/324.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/324","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":324,"pages":{"first":2334,"last":2340},"keywords":["Data Mining: Federated Learning","Machine Learning: Generative Adverserial Networks","Knowledge Representation and Reasoning: Reasong about actions","Data Mining: Privacy Preserving Data Mining"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986664","length_sec":83,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984807","length_sec":319,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Dwork et al., 2014] Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Founda- tions and Trends in Theoretical Computer Science, 9:211- 407, 2014.","title":"The algorithmic foundations of differential privacy","authors":"Cynthia Dwork, Aaron Roth","year":"2014","venue":"Foundations and Trends in Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Fredrikson et al., 2015] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322- 1333, 2015.","title":"Model inversion attacks that exploit confidence information and basic countermeasures","authors":"Matt Fredrikson, Somesh Jha, Thomas Ristenpart","year":"2015","venue":"Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Geiping et al., 2020] Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. Inverting gradients -how easy is it to break privacy in federated learning? In Advances in Neural Information Processing Systems, volume 33, pages 16937-16947.","title":"Inverting gradients -how easy is it to break privacy in federated learning?","authors":"Geiping","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gentry and others, 2009] Craig Gentry et al. A fully homo- morphic encryption scheme, volume 20. Stanford univer- sity Stanford, 2009.","title":"A fully homomorphic encryption scheme","authors":"Craig Gentry","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gong et al., 2012] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE conference on computer vision and pattern recognition, pages 2066-2073. IEEE, 2012.","title":"Geodesic flow kernel for unsupervised domain adaptation","authors":"Gong","year":"2012","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gu et al., 2021] Hanlin Gu, Lixin Fan, Bowen Li, Yan Kang, Yuan Yao, and Qiang Yang. Federated deep learning with bayesian privacy. arXiv preprint arXiv:2109.13012, 2021.","title":"Federated deep learning with bayesian privacy","authors":"Hanlin Gu, Lixin Fan, Bowen Li, Yan Kang, Yuan Yao, Qiang Yang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gupta and Raskar, 2018] Otkrist Gupta and Ramesh Raskar. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 116:1-8, 2018.","title":"Otkrist Gupta and Ramesh Raskar. Distributed learning of deep neural network over multiple agents","authors":"Raskar Gupta","year":"2018","venue":"Journal of Network and Computer Applications","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hardy et al., 2019] Corentin Hardy, Erwan Le Merrer, and Bruno Sericola. Md-gan: Multi-discriminator generative adversarial networks for distributed datasets. In 2019 IEEE international parallel and distributed processing symposium (IPDPS), pages 866-877. IEEE, 2019.","title":"Md-gan: Multi-discriminator generative adversarial networks for distributed datasets","authors":"Corentin Hardy, Erwan Le Merrer, Bruno Sericola","year":"2019","venue":"IEEE international parallel and distributed processing symposium (IPDPS)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. In NIPS 2014 Deep Learning Workshop, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":"NIPS 2014 Deep Learning Workshop","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hitaj et al., 2017] Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: in- formation leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 603-618, 2017.","title":"Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: information leakage from collaborative deep learning","authors":"Hitaj","year":"2017","venue":"Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86:2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Lecun","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2020] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, volume 2, pages 429-450, 2020.","title":"Federated optimization in heterogeneous networks","authors":"Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith","year":"2020","venue":"Proceedings of Machine Learning and Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2021] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. arXiv preprint arXiv:2102.02079, 2021.","title":"Federated learning on non-iid data silos: An experimental study","authors":"Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2020] Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. In Advances in Neural Infor- mation Processing Systems, volume 33, pages 2351-2363.","title":"Ensemble distillation for robust model fusion in federated learning","authors":"Tao Lin, Lingjing Kong, Sebastian U Stich, Martin Jaggi","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mao et al., 2019] Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, and Ming-Hsuan Yang. Mode seeking genera- tive adversarial networks for diverse image synthesis. In IEEE Conference on Computer Vision and Pattern Recog- nition, 2019.","title":"Mode seeking generative adversarial networks for diverse image synthesis","authors":"Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, Ming-Hsuan Yang","year":"2019","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282, 2017.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera Y Arcas","year":"2017","venue":"Artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mirza and Osindero, 2014] Mehdi Mirza and Simon Osin- dero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.","title":"Conditional generative adversarial nets","authors":"Mehdi Mirza, Simon Osindero","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Peng et al., 2019] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment match- ing for multi-source domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 1406-1415, 2019.","title":"Moment matching for multi-source domain adaptation","authors":"Peng","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rasouli et al., 2020] Mohammad Rasouli, Tao Sun, and Ram Rajagopal. Fedgan: Federated generative adver- sarial networks for distributed data. arXiv preprint arXiv:2006.07228, 2020.","title":"Fedgan: Federated generative adversarial networks for distributed data","authors":"Mohammad Rasouli, Tao Sun, Ram Rajagopal","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ren et al., 2021] Hanchi Ren, Jingjing Deng, and Xianghua Xie. Grnn: Generative regression neural network-a data leakage attack for federated learning. arXiv preprint arXiv:2105.00529, 2021.","title":"Grnn: Generative regression neural network-a data leakage attack for federated learning","authors":"Ren","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhu and Han, 2020] Ligeng Zhu and Song Han. Deep leak- age from gradients. In Federated Learning, pages 17-31. Springer, 2020.","title":"Ligeng Zhu and Song Han. Deep leakage from gradients","authors":"Han Zhu","year":"2020","venue":"Federated Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu et al., 2021] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. arXiv preprint arXiv:2105.10056, 2021.","title":"Data-free knowledge distillation for heterogeneous federated learning","authors":"Zhu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}