{"type":"ScholarlyArticle","title":"Measuring Data Leakage in Machine-Learning Models with Fisher Information (Extended Abstract)","authors":[{"name":"Awni Hannun","slug":"awni-hannun","given":"Awni","family":"Hannun","affiliation":"Meta AI Research"},{"name":"Chuan Guo","slug":"chuan-guo","given":"Chuan","family":"Guo","affiliation":"Meta AI Research"},{"name":"Laurens van der Maaten","slug":"laurens-van-der-maaten","given":"Laurens","family":"van der Maaten","affiliation":"Meta AI Research"}],"abstract":"Machine-learning models contain information about the data they were trained on. This information leaks either through the model itself or through predictions made by the model. Consequently, when the training data contains sensitive attributes, assessing the amount of information leakage is paramount. We propose a method to quantify this leakage using the Fisher information of the model about the data. Unlike the worst-case a priori guarantees of differential privacy, Fisher information loss measures leakage with respect to specific examples, attributes, or sub-populations within the dataset. We motivate Fisher information loss through the Cramer-Rao bound and delineate the implied threat model. We provide efficient methods to compute Fisher information loss for output-perturbed generalized linear models. Finally, we empirically validate Fisher information loss as a useful measure of information leakage.","doi":"10.24963/ijcai.2022/736","url":"https://ijcai.org/proceedings/2022/736","pdf":"https://ijcai.org/proceedings/2022/0736.pdf","markdown":"https://ijcai.org/proceedings/2022/736.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/736","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Sister Conferences Best Papers","track":"Sister Conferences Best Papers","number":736,"pages":{"first":5284,"last":5288},"keywords":["Artificial Intelligence: General"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987259","length_sec":81,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985402","length_sec":474,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Agrawal and Aggarwal, 2001] Dakshi Agrawal and Charu C Ag- garwal. On the design and quantification of privacy preserv- ing data mining algorithms. In Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 247-255, 2001.","title":"Dakshi Agrawal and Charu C Aggarwal. On the design and quantification of privacy preserving data mining algorithms","authors":"Aggarwal Agrawal","year":"2001","venue":"Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anderson, 1977] Harald Anderson. Efficiency versus protection in a general randomized response model. Scandinavian Journal of Statistics, pages 11-19, 1977.","title":"Efficiency versus protection in a general randomized response model","authors":"Anderson ;, Harald Anderson","year":"1977","venue":"Scandinavian Journal of Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carlini et al., 2019] Nicholas Carlini, Chang Liu, Úlfar Erlings- son, Jernej Kos, and Dawn Song. The secret sharer: Evaluat- ing and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19), pages 267-284, 2019.","title":"The secret sharer: Evaluating and testing unintended memorization in neural networks","authors":"Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, Dawn Song","year":"2019","venue":"28th USENIX Security Symposium (USENIX Security 19)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chaudhuri et al., 2011] Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. Differentially private empirical risk min- imization. Journal of Machine Learning Research, 12(3), 2011.","title":"Differentially private empirical risk minimization","authors":"Kamalika Chaudhuri, Claire Monteleoni, Anand D Sarwate","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cummings et al., 2019] Rachel Cummings, Varun Gupta, Dhamma Kimpara, and Jamie Morgenstern. On the com- patibility of privacy and fairness. In Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, pages 309-315, 2019.","title":"On the compatibility of privacy and fairness","authors":"Rachel Cummings, Varun Gupta, Dhamma Kimpara, Jamie Morgenstern","year":"2019","venue":"Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Daubechies et al., 2010] Ingrid Daubechies, Ronald DeVore, Mas- simo Fornasier, and C Sinan Güntürk. Iteratively reweighted least squares minimization for sparse recovery. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 63(1):1-38, 2010.","title":"Iteratively reweighted least squares minimization for sparse recovery","authors":"Ingrid Daubechies, Ronald Devore, Massimo Fornasier, C Sinan, Güntürk","year":"2010","venue":"Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dong et al., 2019] Jinshuo Dong, Aaron Roth, and Weijie J Su. Gaussian differential privacy. arXiv preprint arXiv:1905.02383, 2019.","title":"Gaussian differential privacy","authors":"Dong","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dwork and Roth, 2014] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3-4):211-407, 2014.","title":"The algorithmic foundations of differential privacy","authors":"Roth ; Dwork, Cynthia Dwork, Aaron Roth","year":"2014","venue":"Foundations and Trends® in Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dwork et al., 2006] Cynthia Dwork, Frank McSherry, Kobbi Nis- sim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265- 284. Springer, 2006.","title":"Calibrating noise to sensitivity in private data analysis","authors":"Cynthia Dwork, Frank Mcsherry, Kobbi Nissim, Adam Smith","year":"2006","venue":"Theory of cryptography conference","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ekstrand et al., 2018] Michael D Ekstrand, Rezvan Joshaghani, and Hoda Mehrpouyan. Privacy for all: Ensuring fair and eq- uitable privacy protections. In Conference on Fairness, Account- ability and Transparency, pages 35-47, 2018.","title":"Privacy for all: Ensuring fair and equitable privacy protections","authors":"Rezvan Michael D Ekstrand, Hoda Joshaghani, Mehrpouyan","year":"2018","venue":"Conference on Fairness, Accountability and Transparency","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ene and Vladu, 2019] Alina Ene and Adrian Vladu. Improved convergence for ℓ1 and ℓ∞ regression via iteratively reweighted least squares. In International Conference on Machine Learning, pages 1794-1801. PMLR, 2019.","title":"Improved convergence for ℓ1 and ℓ∞ regression via iteratively reweighted least squares","authors":"Alina Ene, Adrian Vladu","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Farokhi and Kaafar, 2020] Farhad Farokhi and Mohamed Ali Kaa- far. Modelling and quantifying membership information leakage in machine learning. arXiv preprint arXiv:2001.10648, 2020.","title":"Modelling and quantifying membership information leakage in machine learning","authors":"Farhad Farokhi, Mohamed Ali Kaafar","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Farokhi and Sandberg, 2017] Farhad Farokhi and Henrik Sand- berg. Fisher information as a measure of privacy: Preserving privacy of households with smart meters using batteries. IEEE Transactions on Smart Grid, 9(5):4726-4734, 2017.","title":"Farhad Farokhi and Henrik Sandberg. Fisher information as a measure of privacy: Preserving privacy of households with smart meters using batteries","authors":"Sandberg Farokhi","year":"2017","venue":"IEEE Transactions on Smart Grid","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Fredrikson et al., 2014] Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In 23rd USENIX Security Symposium (USENIX Security 14), pages 17-32, 2014.","title":"Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing","authors":"Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, Thomas Ristenpart","year":"2014","venue":"23rd USENIX Security Symposium (USENIX Security 14)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ghosh and Kleinberg, 2016] Arpita Ghosh and Robert Kleinberg. Inferential privacy guarantees for differentially private mecha- nisms. arXiv preprint arXiv:1603.01508, 2016.","title":"Inferential privacy guarantees for differentially private mechanisms","authors":"Kleinberg Ghosh, Arpita Ghosh, Robert Kleinberg","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Green, 1984] Peter J Green. Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resis- tant alternatives. Journal of the Royal Statistical Society: Series B (Methodological), 46(2):149-170, 1984.","title":"Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives","authors":"J Peter, Green","year":"1984","venue":"Journal of the Royal Statistical Society: Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kay, 1993] Steven M Kay. Fundamentals of statistical signal pro- cessing. Prentice Hall PTR, 1993.","title":"Fundamentals of statistical signal processing","authors":"M Steven, Kay","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2007] Ninghui Li, Tiancheng Li, and Suresh Venkata- subramanian. t-closeness: Privacy beyond k-anonymity and l- diversity. In 2007 IEEE 23rd International Conference on Data Engineering, pages 106-115. IEEE, 2007.","title":"Ninghui Li, Tiancheng Li, and Suresh Venkatasubramanian","authors":"Li","year":"2007","venue":"IEEE 23rd International Conference on Data Engineering","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2016] Changchang Liu, Supriyo Chakraborty, and Pra- teek Mittal. Dependence makes you vulnberable: Differential privacy under dependent tuples. In NDSS, volume 16, pages 21- 24, 2016.","title":"Dependence makes you vulnberable: Differential privacy under dependent tuples","authors":"Liu","year":"2016","venue":"NDSS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Long et al., 2018] Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen. Understanding membership inferences on well- generalized learning models. arXiv preprint arXiv:1802.04889, 2018.","title":"Understanding membership inferences on wellgeneralized learning models","authors":"Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, Kai Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Machanavajjhala et al., 2007] Ashwin Machanavajjhala, Daniel Kifer, Johannes Gehrke, and Muthuramakrishnan Venkitasubra- maniam. l-diversity: Privacy beyond k-anonymity. ACM Trans- actions on Knowledge Discovery from Data (TKDD), 1(1):3-es, 2007.","title":"Muthuramakrishnan Venkitasubramaniam. l-diversity: Privacy beyond k-anonymity","authors":"Ashwin Machanavajjhala, Daniel Kifer, Johannes Gehrke","year":"2007","venue":"ACM Transactions on Knowledge Discovery from Data (TKDD)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[McSherry and Talwar, 2007] Frank McSherry and Kunal Talwar. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07), pages 94-103. IEEE, 2007.","title":"Mechanism design via differential privacy","authors":"Talwar Mcsherry, Frank Mcsherry, Kunal Talwar","year":"2007","venue":"48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mironov, 2017] Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 263-275. IEEE, 2017.","title":"Rényi differential privacy","authors":"Ilya Mironov","year":"2017","venue":"IEEE 30th Computer Security Foundations Symposium (CSF)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Papernot et al., 2017] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pages 506-519, 2017.","title":"Practical black-box attacks against machine learning","authors":"Nicolas Papernot, Patrick Mcdaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, Ananthram Swami","year":"2017","venue":"Proceedings of the 2017 ACM on Asia conference on computer and communications security","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Samarati and Sweeney, 1998] P. Samarati and L. Sweeney. Pro- tecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression. Techni- cal report, 1998.","title":"Protecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression","authors":"P Samarati, L Sweeney","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shokri et al., 2017] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pages 3-18. IEEE, 2017.","title":"Membership inference attacks against machine learning models","authors":"Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov","year":"2017","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tramèr et al., 2016] Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In 25th USENIX Security Symposium (USENIX Security 16), pages 601-618, 2016.","title":"Stealing machine learning models via prediction apis","authors":"Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, Thomas Ristenpart","year":"2016","venue":"25th USENIX Security Symposium (USENIX Security 16)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yaghini et al., 2019] Mohammad Yaghini, Bogdan Kulynych, and Carmela Troncoso. Disparate vulnerability: On the unfairness of privacy attacks against machine learning. arXiv preprint arXiv:1906.00389, 2019.","title":"Disparate vulnerability: On the unfairness of privacy attacks against machine learning","authors":"Mohammad Yaghini, Bogdan Kulynych, Carmela Troncoso","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yeom et al., 2018] Samuel Yeom, Irene Giacomelli, Matt Fredrik- son, and Somesh Jha. Privacy risk in machine learning: Analyz- ing the connection to overfitting. In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pages 268-282. IEEE, 2018.","title":"Privacy risk in machine learning: Analyzing the connection to overfitting","authors":"Samuel Yeom, Irene Giacomelli, Matt Fredrikson, Somesh Jha","year":"2018","venue":"IEEE 31st Computer Security Foundations Symposium (CSF)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}