{"type":"ScholarlyArticle","title":"To Trust or Not To Trust Prediction Scores for Membership Inference Attacks","authors":[{"name":"Dominik Hintersdorf","slug":"dominik-hintersdorf","given":"Dominik","family":"Hintersdorf","affiliation":"Department of Computer Science , Technical University of Darmstadt , Germany"},{"name":"Lukas Struppek","slug":"lukas-struppek","given":"Lukas","family":"Struppek","affiliation":"Department of Computer Science , Technical University of Darmstadt , Germany"},{"name":"Kristian Kersting","slug":"kristian-kersting","given":"Kristian","family":"Kersting","affiliation":"Department of Computer Science , Technical University of Darmstadt , Germany; Centre for Cognitive Science , Technical University of Darmstadt , Germany; Hessian Center for AI (hessian.AI) , Germany"}],"abstract":"Membership inference attacks (MIAs) aim to determine whether a specific sample was used to train a predictive model. Knowing this may indeed lead to a privacy breach. Most MIAs, however, make use of the model's prediction scores - the probability of each output given some input - following the intuition that the trained model tends to behave differently on its training data. We argue that this is a fallacy for many modern deep network architectures. Consequently, MIAs will miserably fail since overconfidence leads to high false-positive rates not only on known domains but also on out-of-distribution data and implicitly acts as a defense against MIAs. Specifically, using generative adversarial networks, we are able to produce a potentially infinite number of samples falsely classified as part of the training data. In other words, the threat of MIAs is overestimated, and less information is leaked than previously assumed. Moreover, there is actually a trade-off between the overconfidence of models and their susceptibility to MIAs: the more classifiers know when they do not know, making low confidence predictions, the more they reveal the training data.","doi":"10.24963/ijcai.2022/422","url":"https://ijcai.org/proceedings/2022/422","pdf":"https://ijcai.org/proceedings/2022/0422.pdf","markdown":"https://ijcai.org/proceedings/2022/422.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/422","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":422,"pages":{"first":3043,"last":3049},"keywords":["Machine Learning: Other","Machine Learning: Adversarial Machine Learning","Computer Vision: Bias, Fairness & Privacy","Machine Learning: Evaluation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986962","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985105","length_sec":684,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Arora et al., 2018] Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neu- ral networks with rectified linear units. In ICLR, 2018.","title":"Understanding deep neural networks with rectified linear units","authors":"Raman Arora, Amitabh Basu, Poorya Mianjy, Anirbit Mukherjee","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Choi et al., 2020] Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In CVPR, pages 8185-8194, 2020.","title":"Stargan v2: Diverse image synthesis for multiple domains","authors":"Choi","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Choquette-Choo et al., 2021] Christopher A. Choquette- Choo, Florian Tramer, Nicholas Carlini, and Nicolas Pa- pernot. Label-only membership inference attacks. In ICML, volume 139, pages 1964-1974, 18-24 Jul 2021.","title":"Label-only membership inference attacks","authors":"A Christopher, Florian Choquette-Choo, Nicholas Tramer, Nicolas Carlini, Papernot","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Coates et al., 2011] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In AISTATS, volume 15, pages 215-223, 2011.","title":"An analysis of single-layer networks in unsupervised feature learning","authors":"Adam Coates, Andrew Ng, Honglak Lee","year":"2011","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guo et al., 2017] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In ICML, volume 70, pages 1321-1330, 2017.","title":"On calibration of modern neural networks","authors":"Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q Weinberger","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hein et al., 2019] Matthias Hein, Maksym An- driushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In CVPR, pages 41-50, 2019.","title":"Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem","authors":"Matthias Hein, Maksym Andriushchenko, Julian Bitterwolf","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hendrycks and Gimpel, 2017] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of- distribution examples in neural networks. In ICLR, 2017.","title":"A baseline for detecting misclassified and out-ofdistribution examples in neural networks","authors":"Gimpel Hendrycks, Dan Hendrycks, Kevin Gimpel","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Karras et al., 2020] Tero Karras, Miika Aittala, Janne Hell- sten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In NeurIPS, 2020.","title":"Training generative adversarial networks with limited data","authors":"Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Aila","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khosla et al., 2011] Aditya Khosla, Nityananda Jayadevap- rakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In CVPR Workshop, June 2011.","title":"Novel dataset for fine-grained image categorization","authors":"Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, Li Fei-Fei","year":"2011","venue":"CVPR Workshop","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kristiadi et al., 2020] Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being bayesian, even just a bit, fixes overconfidence in relu networks. In ICML, volume 119, pages 5436-5446, 2020.","title":"Being bayesian, even just a bit, fixes overconfidence in relu networks","authors":"Agustinus Kristiadi, Matthias Hein, Philipp Hennig","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Technical report, Mas- ter's thesis, Department of Computer Science, University of Toronto, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Leibig et al., 2019] C. Leibig, V. Allken, M. S. Ayhan, P. Berens, and S. Wahl. Leveraging uncertainty informa- tion from deep neural networks for disease detection. Sci- entific Reports, 7(0), 2019.","title":"Leveraging uncertainty information from deep neural networks for disease detection","authors":"C Leibig, V Allken, M S Ayhan, P Berens, S Wahl","year":"2019","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Zhang, 2021] Zheng Li and Yang Zhang. Member- ship leakage in label-only exposures. In CCS, pages 880- 895, 2021.","title":"Membership leakage in label-only exposures","authors":"Li, Zhang ; ] Zheng Li, Yang Zhang","year":"2021","venue":"CCS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Müller et al., 2019] Rafael Müller, Simon Kornblith, and Geoffrey E. Hinton. When does label smoothing help? In NeurIPS, pages 4696-4705, 2019.","title":"When does label smoothing help?","authors":"Müller","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Naeini et al., 2015] Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In AAAI, pages 2901-2907, 2015.","title":"Obtaining well calibrated probabilities using bayesian binning","authors":"Naeini","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learn- ing. In NeurIPS Workshop, 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":"NeurIPS Workshop","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nguyen et al., 2015] Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, pages 427-436, 2015.","title":"Deep neural networks are easily fooled: High confidence predictions for unrecognizable images","authors":"Anh Mai Nguyen, Jason Yosinski, Jeff Clune","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ramachandran et al., 2018] Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Searching for activation functions. In ICLR, 2018.","title":"Searching for activation functions","authors":"Ramachandran","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rezaei and Liu, 2021] Shahbaz Rezaei and Xin Liu. On the difficulty of membership inference attacks. In CVPR, pages 7892-7900, 2021.","title":"On the difficulty of membership inference attacks","authors":"Shahbaz Rezaei, Xin Liu","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Salem et al., 2019] Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. Ml-leaks: Model and data independent member- ship inference attacks and defenses on machine learning models. In NDSS Symposium, 2019.","title":"Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models","authors":"Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, Michael Backes","year":"2019","venue":"NDSS Symposium","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shokri et al., 2017] Reza Shokri, Marco Stronati, Con- gzheng Song, and Vitaly Shmatikov. Membership infer- ence attacks against machine learning models. In IEEE Symposium on Security and Privacy, pages 3-18, 2017.","title":"Membership inference attacks against machine learning models","authors":"Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov","year":"2017","venue":"IEEE Symposium on Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Song and Mittal, 2021] Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learn- ing models. In USENIX Security Symposium, 2021.","title":"Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models","authors":"Mittal Song","year":"2021","venue":"USENIX Security Symposium","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Szegedy et al., 2016] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, pages 2818-2826, 2016.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tan and Le, 2019] Mingxing Tan and Quoc V. Le. Efficient- net: Rethinking model scaling for convolutional neural networks. In ICML, volume 97, pages 6105-6114, 2019.","title":"Efficientnet: Rethinking model scaling for convolutional neural networks","authors":"Mingxing Tan, V Quoc, Le","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Thulasidasan et al., 2019] Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibra- tion and predictive uncertainty for deep neural networks. In NeurIPS, 2019.","title":"On mixup training: Improved calibration and predictive uncertainty for deep neural networks","authors":"Gopinath Sunil Thulasidasan, Jeff A Chennupati, Tanmoy Bilmes, Sarah Bhattacharya, Michalak","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[van der Maaten and Hinton, 2008] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 9(86):2579-2605, 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yeom et al., 2018] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In IEEE CSF Symposium, pages 268-282, 2018.","title":"Privacy risk in machine learning: Analyzing the connection to overfitting","authors":"Yeom","year":"2018","venue":"IEEE CSF Symposium","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}