{"type":"ScholarlyArticle","title":"When Fairness Meets Privacy: Exploring Privacy Threats in Fair Binary Classifiers via Membership Inference Attacks","authors":[{"name":"Huan Tian","slug":"huan-tian","given":"Huan","family":"Tian","affiliation":"University of Technology Sydney"},{"name":"Guangsheng Zhang","slug":"guangsheng-zhang","given":"Guangsheng","family":"Zhang","affiliation":"University of Technology Sydney"},{"name":"Bo Liu","slug":"bo-liu","given":"Bo","family":"Liu","affiliation":"University of Technology Sydney"},{"name":"Tianqing Zhu","slug":"tianqing-zhu","given":"Tianqing","family":"Zhu","affiliation":"City University of Macau"},{"name":"Ming Ding","slug":"ming-ding","given":"Ming","family":"Ding","affiliation":"Data61 CSIRO"},{"name":"Wanlei Zhou","slug":"wanlei-zhou","given":"Wanlei","family":"Zhou","affiliation":"City University of Macau"}],"abstract":"While in-processing fairness approaches show promise in mitigating bias predictions, their potential impact on privacy leakage remains under-explored. We aim to address this gap by assessing the privacy risks of fairness-enhanced binary classifiers with membership inference attacks (MIAs). Surprisingly, our results reveal that these fairness interventions exhibit increased resilience against existing attacks, indicating that enhancing fairness does not necessarily lead to privacy compromises. However, we find current attack methods are ineffective as they typically degrade into simple threshold models with limited attack effectiveness. Following this observation, we discover a novel threat dubbed Fairness Discrepancy Membership Inference Attacks (FD-MIA) that exploits prediction discrepancies between fair and biased models. This attack reveals more potent vulnerabilities and poses significant privacy risks to model privacy. Extensive experiments across multiple datasets, attack methods, and representative fairness approaches confirm our findings and demonstrate the efficacy of the proposed attack method. Our study exposes the overlooked privacy threats in fairness studies, advocating for thorough evaluations of potential security vulnerabilities before model deployments.","doi":"10.24963/ijcai.2024/57","url":"https://ijcai.org/proceedings/2024/57","pdf":"https://ijcai.org/proceedings/2024/0057.pdf","markdown":"https://ijcai.org/proceedings/2024/57.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/57","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":57,"pages":{"first":512,"last":520},"keywords":["AI Ethics, Trust, Fairness: ETF: Fairness and diversity","AI Ethics, Trust, Fairness: ETF: Trustworthy AI","AI Ethics, Trust, Fairness: ETF: Safety and robustness"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martin Abadi, Andy Chu, Ian Goodfel- low, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In CCS, pages 308-318, 2016.","title":"Deep learning with differential privacy","authors":"Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan Mcmahan, Ilya Mironov, Kunal Talwar, Li Zhang","year":"2016","venue":"CCS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carlini et al., 2022] Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In SP, pages 1897-1914, 2022.","title":"Membership inference attacks from first principles","authors":"Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, Florian Tramer","year":"2022","venue":"SP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chai and Wang, 2022] Junyi Chai and Xiaoqian Wang. Fairness with adaptive weights. In ICML, pages 2853- 2866, 2022.","title":"Fairness with adaptive weights","authors":"Junyi Chai, Xiaoqian Wang","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chang and Shokri, 2021] Hongyan Chang and Reza Shokri. On the privacy risks of algorithmic fairness. In 2021 IEEE European Symposium on Security and Privacy (Eu- roS&P), pages 292-303, 2021.","title":"On the privacy risks of algorithmic fairness","authors":"Chang, Shokri ; ] Hongyan Chang, Reza Shokri","year":"2021","venue":"IEEE European Symposium on Security and Privacy (Eu-roS&P)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ching-Yao Chuang, 2021] Youssef Mroueh Ching- Yao Chuang. Fair mixup: Fairness via interpolation. In ICLR, 2021.","title":"Fair mixup: Fairness via interpolation","authors":"Ching-Yao Chuang, ; Youssef Mroueh, Ching-Yao Chuang","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choquette-Choo et al., 2021] Christopher A. Choquette- Choo, Florian Tramer, Nicholas Carlini, and Nicolas Pa- pernot. Label-Only Membership Inference Attacks. In ICML, pages 1964-1974, 2021.","title":"Label-Only Membership Inference Attacks","authors":"A Christopher, Florian Choquette-Choo, Nicholas Tramer, Nicolas Carlini, Papernot","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Creager et al., 2019] Elliot Creager, David Madras, Joern- Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. In ICML, pages 1436-1445, 2019.","title":"Flexibly fair representation learning by disentanglement","authors":"Elliot Creager, David Madras, Joern-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, Richard Zemel","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cruz et al., 2023] André Cruz, Catarina G Belém, João Bravo, Pedro Saleiro, and Pedro Bizarro. FairGBM: Gra- dient boosting with fairness constraints. In ICLR, 2023.","title":"FairGBM: Gradient boosting with fairness constraints","authors":"Cruz","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Du et al., 2021] Mengnan Du, Subhabrata Mukherjee, Guanchu Wang, Ruixiang Tang, Ahmed Awadallah, and Xia Hu. Fairness via representation neutralization. In NeurIPS, 2021.","title":"Fairness via representation neutralization","authors":"Du","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dwork et al., 2006] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptog- raphy: Third Theory of Cryptography Conference, pages 265-284, 2006.","title":"Calibrating noise to sensitivity in private data analysis","authors":"Dwork","year":"2006","venue":"Theory of Cryptography: Third Theory of Cryptography Conference","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gao et al., 2023] Junyao Gao, Xinyang Jiang, Huishuai Zhang, Yifan Yang, Shuguang Dou, Dongsheng Li, Duo- qian Miao, Cheng Deng, and Cairong Zhao. Similarity Distribution Based Membership Inference Attack on Per- son Re-identification. In AAAI, pages 14820-14828, 2023.","title":"Similarity Distribution Based Membership Inference Attack on Person Re-identification","authors":"Gao","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Geralds, 2017] J Geralds. Utkface large scale face dataset. github. com, 2017.","title":"Utkface large scale face dataset","authors":"Geralds","year":"2017","venue":null,"doi":null,"url":"github.com","internal_url":null},{"position":13,"raw":"[Han et al., 2024] Xiaotian Han, Jianfeng Chi, Yu Chen, Qi- fan Wang, Han Zhao, Na Zou, and Xia Hu. FFB: A fair fairness benchmark for in-processing group fairness meth- ods. In ICLR, 2024.","title":"FFB: A fair fairness benchmark for in-processing group fairness methods","authors":"Xiaotian Han, Jianfeng Chi, Yu Chen, Qifan Wang, Han Zhao, Na Zou, Xia Hu","year":"2024","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hardt et al., 2016] Moritz Hardt, Eric Price, and Nati Sre- bro. Equality of opportunity in supervised learning. In NeurIPS, pages 3315-3323, 2016.","title":"Equality of opportunity in supervised learning","authors":"Moritz Hardt, Eric Price, Nati Srebro","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[He et al., 2022] Xinlei He, Hongbin Liu, Neil Zhenqiang Gong, and Yang Zhang. Semi-Leak: Membership In- ference Attacks Against Semi-supervised Learning. In ECCV, pages 365-381, 2022.","title":"Semi-Leak: Membership Inference Attacks Against Semi-supervised Learning","authors":"He","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hu et al., 2022] Pingyi Hu, Zihan Wang, Ruoxi Sun, Hu Wang, and Minhui Xue. M4i: Multi-modal models membership inference. In NeurIPS, pages 1867-1882, 2022.","title":"M4i: Multi-modal models membership inference","authors":"Pingyi Hu, Zihan Wang, Ruoxi Sun, Hu Wang, Minhui Xue","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jung et al., 2023] Sangwon Jung, Taeeon Park, Sanghyuk Chun, and Taesup Moon. Re-weighting based group fair- ness regularization via classwise robust optimization. In ICLR, 2023.","title":"Re-weighting based group fairness regularization via classwise robust optimization","authors":"Sangwon Jung, Taeeon Park, Sanghyuk Chun, Taesup Moon","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Karkkainen and Joo, 2021] Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1548-1558, 2021.","title":"Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation","authors":"Joo ; Karkkainen, Kimmo Karkkainen, Jungseock Joo","year":"2021","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kim et al., 2019] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In CVPR, pages 9004-9012, 2019.","title":"Learning not to learn: Training deep neural networks with biased data","authors":"Kim","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lee et al., 2020] Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In CVPR, 2020.","title":"Maskgan: Towards diverse and interactive facial image manipulation","authors":"Ziwei Cheng-Han Lee, Lingyun Liu, Ping Wu, Luo","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2022] Zheng Li, Yiyong Liu, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Auditing Membership Leakages of Multi-Exit Networks. In CCS, pages 1917- 1931, 2022.","title":"Auditing Membership Leakages of Multi-Exit Networks","authors":"Zheng Li, Yiyong Liu, Xinlei He, Ning Yu, Michael Backes, Yang Zhang","year":"2022","venue":"CCS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2022] Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, and Yang Zhang. ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models. In USENIX Security, pages 4525-4542, 2022.","title":"ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models","authors":"Liu","year":"2022","venue":"USENIX Security","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Manisha and Gujar, 2020] Padala Manisha and Sujit Gujar. Fnnc: Achieving fairness through neural networks. In IJ- CAI, 2020.","title":"Fnnc: Achieving fairness through neural networks","authors":"Gujar Manisha, Padala Manisha, Sujit Gujar","year":"2020","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2020/315"},{"position":24,"raw":"[Mehrabi et al., 2021] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. CSUR, pages 1-35, 2021.","title":"A survey on bias and fairness in machine learning","authors":"Mehrabi","year":"2021","venue":"CSUR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Park et al., 2021] Sungho Park, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. Learning disentangled representa- tion for fair facial attribute classification via fairness-aware information alignment. In AAAI, pages 2403-2411, 2021.","title":"Learning disentangled representation for fair facial attribute classification via fairness-aware information alignment","authors":"Sungho Park, Sunhee Hwang, Dohyung Kim, Hyeran Byun","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Park et al., 2022] Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. Fair contrastive learning for facial attribute classification. In CVPR, pages 10389-10398, 2022.","title":"Fair contrastive learning for facial attribute classification","authors":"Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang, Dohyung Kim, Hyeran Byun","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Pinzón et al., 2022] Carlos Pinzón, Catuscia Palamidessi, Pablo Piantanida, and Frank Valencia. On the impossi- bility of non-trivial accuracy in presence of fairness con- straints. In AAAI, pages 7993-8000, 2022.","title":"On the impossibility of non-trivial accuracy in presence of fairness constraints","authors":"Carlos Pinzón, Catuscia Palamidessi, Pablo Piantanida, Frank Valencia","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Qi et al., 2022] Tao Qi, Fangzhao Wu, Chuhan Wu, Lingjuan Lyu, Tong Xu, Hao Liao, Zhongliang Yang, Yongfeng Huang, and Xing Xie. FairVFL: A fair vertical federated learning framework with contrastive adversarial learning. In NeurIPS, 2022.","title":"FairVFL: A fair vertical federated learning framework with contrastive adversarial learning","authors":"Tao Qi, Fangzhao Wu, Chuhan Wu, Lingjuan Lyu, Tong Xu, Hao Liao, Zhongliang Yang, Yongfeng Huang, Xing Xie","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Roh et al., 2021] Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. Sample selection for fair and robust training. In NeurIPS, pages 815-827, 2021.","title":"Sample selection for fair and robust training","authors":"Yuji Roh, Kangwook Lee, Steven Whang, Changho Suh","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sablayrolles et al., 2019] Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Herve Je- gou. White-box vs Black-box: Bayes Optimal Strategies for Membership Inference. In ICML, pages 5558-5567, 2019.","title":"White-box vs Black-box: Bayes Optimal Strategies for Membership Inference","authors":"Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, Herve Jegou","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shokri et al., 2017] Reza Shokri, Marco Stronati, Con- gzheng Song, and Vitaly Shmatikov. Membership infer- ence attacks against machine learning models. In SP, pages 3-18, 2017.","title":"Membership inference attacks against machine learning models","authors":"Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov","year":"2017","venue":"SP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tang et al., 2023] Pengwei Tang, Wei Yao, Zhicong Li, and Yong Liu. Fair scratch tickets: Finding fair sparse net- works without weight training. In CVPR, pages 24406- 24416, 2023.","title":"Fair scratch tickets: Finding fair sparse networks without weight training","authors":"Tang","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Truong et al., 2023] Thanh-Dat Truong, Ngan Le, Bhiksha Raj, Jackson Cothren, and Khoa Luu. Fredom: Fair- ness domain adaptation approach to semantic scene under- standing. In CVPR, 2023.","title":"Fredom: Fairness domain adaptation approach to semantic scene understanding","authors":"Thanh-Dat Truong, Ngan Le, Bhiksha Raj, Jackson Cothren, Khoa Luu","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang and Deng, 2020] Mei Wang and Weihong Deng. Mit- igating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning. In CVPR, page 10, 2020.","title":"Mitigating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning","authors":"Mei Wang, Weihong Deng","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2022] Zhibo Wang, Xiaowei Dong, Henry Xue, Zhifei Zhang, Weifeng Chiu, Tao Wei, and Kui Ren. Fairness-aware adversarial perturbation towards bias miti- gation for deployed deep models. In CVPR, pages 10379- 10388, 2022.","title":"Fairness-aware adversarial perturbation towards bias mitigation for deployed deep models","authors":"Zhibo Wang, Xiaowei Dong, Henry Xue, Zhifei Zhang, Weifeng Chiu, Tao Wei, Kui Ren","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yang et al., 2023] Ziqi Yang, Lijin Wang, Da Yang, Jie Wan, Ziming Zhao, Ee-Chien Chang, Fan Zhang, and Kui Ren. Purifier: Defending Data Inference Attacks via Transforming Confidence Scores. In AAAI, pages 10871- 10879, 2023.","title":"Purifier: Defending Data Inference Attacks via Transforming Confidence Scores","authors":"Ziqi Yang, Lijin Wang, Da Yang, Jie Wan, Ziming Zhao, Ee-Chien Chang, Fan Zhang, Kui Ren","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Ye et al., 2022] Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, and Reza Shokri. En- hanced membership inference attacks against machine learning models. In CCS, pages 3093-3106, 2022.","title":"Enhanced membership inference attacks against machine learning models","authors":"Ye","year":"2022","venue":"CCS","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yuan and Zhang, 2022] Xiaoyong Yuan and Lan Zhang. Membership Inference Attacks and Defenses in Neural Network Pruning. In USENIX Security, pages 4561-4578, 2022.","title":"Membership Inference Attacks and Defenses in Neural Network Pruning","authors":"Xiaoyong Yuan, Lan Zhang","year":"2022","venue":"USENIX Security","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zemel et al., 2013] Richard S. Zemel, Ledell Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. Learning fair representations. In ICML, 2013.","title":"Learning fair representations","authors":"Richard S Zemel, Yu Ledell, Kevin Wu, Toniann Swersky, Cynthia Pitassi, Dwork","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2023] Fengda Zhang, Kun Kuang, Long Chen, Yuxuan Liu, Chao Wu, and Jun Xiao. Fairness-aware contrastive learning with partially annotated sensitive at- tributes. In ICLR, 2023.","title":"Fairness-aware contrastive learning with partially annotated sensitive attributes","authors":"Zhang","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhao et al., 2017] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shop- ping: Reducing gender bias amplification using corpus- level constraints. In EMNLP, pages 2979-2989, 2017.","title":"Men also like shopping: Reducing gender bias amplification using corpuslevel constraints","authors":"Zhao","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhu et al., 2021] Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, and Jiebo Luo. Learning bias-invariant repre- sentation by cross-sample mutual information minimiza- tion. In ICCV, pages 15002-15012, 2021.","title":"Learning bias-invariant representation by cross-sample mutual information minimization","authors":"Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, Jiebo Luo","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zietlow et al., 2022] Dominik Zietlow, Michael Lohaus, Guha Balakrishnan, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, and Chris Russell. Level- ing down in computer vision: Pareto inefficiencies in fair deep classifiers. In CVPR, pages 10410-10421, 2022.","title":"Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers","authors":"Dominik Zietlow, Michael Lohaus, Guha Balakrishnan, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, Chris Russell","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}