{"type":"ScholarlyArticle","title":"FedPass: Privacy-Preserving Vertical Federated Deep Learning with Adaptive Obfuscation","authors":[{"name":"Hanlin Gu","slug":"hanlin-gu","given":"Hanlin","family":"Gu","affiliation":"Hong Kong University of Science and Technology , Hong Kong"},{"name":"Jiahuan Luo","slug":"jiahuan-luo","given":"Jiahuan","family":"Luo","affiliation":"Hong Kong University of Science and Technology , Hong Kong"},{"name":"Yan Kang","slug":"yan-kang","given":"Yan","family":"Kang","affiliation":"Hong Kong University of Science and Technology , Hong Kong"},{"name":"Lixin Fan","slug":"lixin-fan","given":"Lixin","family":"Fan","affiliation":"Hong Kong University of Science and Technology , Hong Kong"},{"name":"Qiang Yang","slug":"qiang-yang","given":"Qiang","family":"Yang","affiliation":"Hong Kong University of Science and Technology , Hong Kong"}],"abstract":"Vertical federated learning (VFL) allows an active party with labeled data to leverage auxiliary features from the passive parties to improve model performance. Concerns about the private feature and label leakage in both the training and inference phases of VFL have drawn wide research attention. In this paper, we propose a general privacy-preserving vertical federated deep learning framework called FedPass, which leverages adaptive obfuscation to protect the feature and label simultaneously.  Strong privacy-preserving capabilities about private features and labels are theoretically proved (in Theorems 1 and 2). \r\nExtensive experimental results with different datasets and network architectures also justify the superiority of FedPass against existing methods in light of its near-optimal trade-off between privacy and model performance.","doi":"10.24963/ijcai.2023/418","url":"https://ijcai.org/proceedings/2023/418","pdf":"https://ijcai.org/proceedings/2023/0418.pdf","markdown":"https://ijcai.org/proceedings/2023/418.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/418","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":418,"pages":{"first":3759,"last":3767},"keywords":["Computer Vision: CV: Bias, fairness and privacy","Machine Learning: ML: Federated learning"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martin Abadi, Andy Chu, Ian Goodfel- low, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on com- puter and communications security, pages 308-318, 2016.","title":"Deep learning with differential privacy","authors":"Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan Mcmahan, Ilya Mironov, Kunal Talwar, Li Zhang","year":"2016","venue":"Proceedings of the 2016 ACM SIGSAC conference on computer and communications security","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aji and Heafield, 2017] Alham Fikri Aji and Kenneth Heafield. Sparse communication for distributed gradient descent. In Proceedings of the 2017 Conference on Em- pirical Methods in Natural Language Processing, pages 440-445, 2017.","title":"Sparse communication for distributed gradient descent","authors":"Heafield ; Aji, Fikri Alham, Kenneth Aji, Heafield","year":"2017","venue":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2021] K. Cheng, T. Fan, Y. Jin, Y. Liu, T. Chen, D. Papadopoulos, and Q. Yang. Secureboost: A lossless federated learning framework. IEEE Intelligent Systems, 36(06):87-98, nov 2021.","title":"Secureboost: A lossless federated learning framework","authors":"K Cheng, T Fan, Y Jin, Y Liu, T Chen, D Papadopoulos, Q Yang","year":"2021","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dryden et al., 2016] Nikoli Dryden, Tim Moon, Sam Ade Jacobs, and Brian Van Essen. Communication quantiza- tion for data-parallel training of deep neural networks. In 2016 2nd Workshop on Machine Learning in HPC Envi- ronments (MLHPC), pages 1-8. IEEE, 2016.","title":"Communication quantization for data-parallel training of deep neural networks","authors":"Nikoli Dryden, Tim Moon, Sam Ade Jacobs, Brian Van Essen","year":"2016","venue":"2016 2nd Workshop on Machine Learning in HPC Environments (MLHPC)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2019] Lixin Fan, Kam Woh Ng, and Chee Seng Chan. Rethinking deep neural network ownership verifi- cation: Embedding passports to defeat ambiguity attacks. Advances in neural information processing systems, 32, 2019.","title":"Rethinking deep neural network ownership verification: Embedding passports to defeat ambiguity attacks","authors":"Lixin Fan, Kam Woh Ng, Chee Seng, Chan","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan et al., 2020] Lixin Fan, Kam Woh Ng, Ce Ju, Tianyu Zhang, Chang Liu, Chee Seng Chan, and Qiang Yang. Re- thinking privacy preserving deep learning: How to eval- uate and thwart privacy attacks. In Federated Learning, pages 32-50. Springer, 2020.","title":"Rethinking privacy preserving deep learning: How to evaluate and thwart privacy attacks","authors":"Lixin Fan, Kam Woh Ng, Ce Ju, Tianyu Zhang, Chang Liu, Chee Seng Chan, Qiang Yang","year":"2020","venue":"Federated Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fan et al., 2021] Lixin Fan, Kam Woh Ng, Chee Seng Chan, and Qiang Yang. Deepip: Deep neural network intellectual property protection with passports. IEEE Transactions on Pattern Analysis & Machine Intelligence, (01):1-1, 2021.","title":"Deepip: Deep neural network intellectual property protection with passports","authors":"Lixin Fan, Kam Woh Ng, Chee Seng Chan, Qiang Yang","year":"2021","venue":"IEEE Transactions on Pattern Analysis & Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fu et al., 2022a] Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X Liu, and Ting Wang. Label inference attacks against vertical federated learning. In 31st USENIX Secu- rity Symposium (USENIX Security 22), Boston, MA, 2022.","title":"Label inference attacks against vertical federated learning","authors":"Fu","year":"2022","venue":"31st USENIX Security Symposium (USENIX Security 22)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fu et al., 2022b] Fangcheng Fu, Huanran Xue, Yong Cheng, Yangyu Tao, and Bin Cui. Blindfl: Vertical fed- erated machine learning without peeking into your data. In Proceedings of the 2022 International Conference on Management of Data, SIGMOD '22, page 1316-1330, New York, NY, USA, 2022. Association for Computing Machinery. Gascón et al., 2016]","title":"Blindfl: Vertical federated machine learning without peeking into your data","authors":"Fu","year":"2016","venue":"Proceedings of the 2022 International Conference on Management of Data, SIGMOD '22","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Adrià Gascón, Phillipp Schoppmann, Borja Balle, Mariana Raykova, Jack Doerner, Samee Za- hur, and David Evans. Secure linear regression on ver- tically partitioned datasets. IACR Cryptol. ePrint Arch., 2016:892, 2016.","title":"Secure linear regression on vertically partitioned datasets","authors":"Adrià Gascón, Phillipp Schoppmann, Borja Balle, Mariana Raykova, Jack Doerner, Samee Zahur, David Evans","year":"2016","venue":"IACR Cryptol. ePrint Arch","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hardy et al., 2017] Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guil- laume Smith, and Brian Thorne. Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv preprint arXiv:1711.10677, 2017.","title":"Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption","authors":"Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, Brian Thorne","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2019] Zecheng He, Tianwei Zhang, and Ruby B Lee. Model inversion attacks against collaborative infer- ence. In Proceedings of the 35th Annual Computer Secu- rity Applications Conference, pages 148-162, 2019.","title":"Model inversion attacks against collaborative inference","authors":"He","year":"2019","venue":"Proceedings of the 35th Annual Computer Security Applications Conference","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2022] Yuanqin He, Yan Kang, Jiahuan Luo, Lixin Fan, and Qiang Yang. A hybrid self-supervised learning framework for vertical federated learning. arXiv preprint arXiv:2208.08934, 2022.","title":"A hybrid self-supervised learning framework for vertical federated learning","authors":"He","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2022] Yuzheng Hu, Tianle Cai, Jinyong Shan, Shange Tang, Chaochao Cai, Ethan Song, Bo Li, and Dawn Song. Is vertical logistic regression privacy- preserving? a comprehensive privacy analysis and beyond. arXiv preprint arXiv:2207.09087, 2022.","title":"Is vertical logistic regression privacypreserving? a comprehensive privacy analysis and beyond","authors":"Hu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang et al., 2020] Yangsibo Huang, Zhao Song, Kai Li, and Sanjeev Arora. Instahide: Instance-hiding schemes for private distributed learning. In International conference on machine learning, pages 4507-4518. PMLR, 2020.","title":"Instahide: Instance-hiding schemes for private distributed learning","authors":"Yangsibo Huang, Zhao Song, Kai Li, Sanjeev Arora","year":"2020","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jin et al., 2021] Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia- Mu Yu, and Tianyi Chen. Cafe: Catastrophic data leak- age in vertical federated learning. NeurIPS, 34:994-1006, 2021.","title":"Cafe: Catastrophic data leakage in vertical federated learning","authors":"Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, Tianyi Chen","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kang et al., 2022a] Y. Kang, Y. He, J. Luo, T. Fan, Y. Liu, and Q. Yang. Privacy-preserving federated adversarial do- main adaptation over feature groups for interpretability. IEEE Transactions on Big Data, (01):1-12, jul 2022.","title":"Privacy-preserving federated adversarial domain adaptation over feature groups for interpretability","authors":"Kang","year":"2022","venue":"IEEE Transactions on Big Data","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kang et al., 2022b] Yan Kang, Yang Liu, and Xinle Liang. FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training. ACM Transactions on Intelli- gent Systems and Technology (TIST), May 2022.","title":"FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training","authors":"Kang","year":"2022","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kang et al., 2022c] Yan Kang, Jiahuan Luo, Yuanqin He, Xiaojin Zhang, Lixin Fan, and Qiang Yang. A framework for evaluating privacy-utility trade-off in vertical federated learning. arXiv preprint arXiv:2209.03885, 2022.","title":"A framework for evaluating privacy-utility trade-off in vertical federated learning","authors":"Kang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neu- ral Information Processing Systems, volume 25.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"Curran Associates, Inc., 2012.","title":null,"authors":"Curran Associates, Inc","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Krizhevsky et al., 2014] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset. online: http://www. cs. toronto. edu/kriz/cifar. html, 55(5), 2014.","title":"The cifar-10 dataset","authors":"Alex Krizhevsky, Vinod Nair, Geoffrey Hinton","year":"2014","venue":null,"doi":null,"url":"http://www.cs.toronto.edu/kriz/cifar.html,55","internal_url":null},{"position":24,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[LeCun et al., 2010] Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010.","title":"Mnist handwritten digit database","authors":"Yann Lecun, Corinna Cortes, Burges","year":"2010","venue":null,"doi":null,"url":"http://yann.lecun.com/exdb/mnist,2","internal_url":null},{"position":26,"raw":"[Li et al., 2020] Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. A review of applications in federated learning. Computers & Industrial Engineering, 149:106854, 2020.","title":"A review of applications in federated learning","authors":"Li Li, Yuxi Fan, Mike Tse, Kuo-Yi Lin","year":"2020","venue":"Computers & Industrial Engineering","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Li et al., 2022a] Bowen Li, Lixin Fan, Hanlin Gu, Jie Li, and Qiang Yang. Fedipr: Ownership verification for fed- erated deep neural network models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.","title":"Fedipr: Ownership verification for federated deep neural network models","authors":"Li","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Li et al., 2022b] Oscar Li, Jiankai Sun, Xin Yang, Weihao Gao, Hongyi Zhang, Junyuan Xie, Virginia Smith, and Chong Wang. Label leakage and protection in two-party split learning. In International Conference on Learning Representations, 2022.","title":"Label leakage and protection in two-party split learning","authors":"Li","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lin et al., 2018] Yujun Lin, Song Han, Huizi Mao, Yu Wang, and Bill Dally. Deep gradient compression: Re- ducing the communication bandwidth for distributed train- ing. In International Conference on Learning Representa- tions, 2018.","title":"Deep gradient compression: Reducing the communication bandwidth for distributed training","authors":"Yujun Lin, Song Han, Huizi Mao, Yu Wang, Bill Dally","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Liu et al., 2020] Yang Liu, Yan Kang, Chaoping Xing, Tianjian Chen, and Qiang Yang. A secure federated transfer learning framework. IEEE Intelligent Systems, 35(4):70-82, 2020.","title":"A secure federated transfer learning framework","authors":"Yang Liu, Yan Kang, Chaoping Xing, Tianjian Chen, Qiang Yang","year":"2020","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Liu et al., 2021] Yang Liu, Zhihao Yi, Yan Kang, Yuanqin He, Wenhan Liu, Tianyuan Zou, and Qiang Yang. Defend- ing label inference and backdoor attacks in vertical feder- ated learning. arXiv preprint arXiv:2112.05409, 2021.","title":"Defending label inference and backdoor attacks in vertical federated learning","authors":"Yang Liu, Zhihao Yi, Yan Kang, Yuanqin He, Wenhan Liu, Tianyuan Zou, Qiang Yang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Luo et al., 2021] Xinjian Luo, Yuncheng Wu, Xiaokui Xiao, and Beng Chin Ooi. Feature inference attack on model predictions in vertical federated learning. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, apr 2021.","title":"Feature inference attack on model predictions in vertical federated learning","authors":"Xinjian Luo, Yuncheng Wu, Xiaokui Xiao, Beng Chin Ooi","year":"2021","venue":"IEEE 37th International Conference on Data Engineering (ICDE)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wu et al., 2015] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 1912-1920, 2015.","title":"3d shapenets: A deep representation for volumetric shapes","authors":"Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, Jianxiong Xiao","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2019] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1-19, 2019.","title":"Federated machine learning: Concept and applications","authors":"Qiang Yang, Yang Liu, Tianjian Chen, Yongxin Tong","year":"2019","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2018] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.","title":"mixup: Beyond empirical risk minimization","authors":"Hongyi Zhang, Moustapha Cisse, David Yann N Dauphin, Lopez-Paz","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2022] Xiaojin Zhang, Yan Kang, Kai Chen, Lixin Fan, and Qiang Yang. Trading off privacy, util- ity and efficiency in federated learning. arXiv preprint arXiv:2209.00230, 2022.","title":"Trading off privacy, utility and efficiency in federated learning","authors":"Xiaojin Zhang, Yan Kang, Kai Chen, Lixin Fan, Qiang Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhu et al., 2019] Ligeng Zhu, Zhijian Liu, , and Song Han. Deep leakage from gradients. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2019.","title":"Deep leakage from gradients","authors":"Ligeng Zhu, Zhijian Liu, Song Han","year":"2019","venue":"Annual Conference on Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zou et al., 2022] Tianyuan Zou, Yang Liu, Yan Kang, Wen- han Liu, Yuanqin He, Zhihao Yi, Qiang Yang, and Ya-Qin Zhang. Defending batch-level label inference and replace- ment attacks in vertical federated learning. IEEE Transac- tions on Big Data, 2022.","title":"Defending batch-level label inference and replacement attacks in vertical federated learning","authors":"Tianyuan Zou, Yang Liu, Yan Kang, Wenhan Liu, Yuanqin He, Zhihao Yi, Qiang Yang, Ya-Qin Zhang","year":"2022","venue":"IEEE Transactions on Big Data","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}