{"type":"ScholarlyArticle","title":"Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in Deep Learning with Provable Robustness","authors":[{"name":"NhatHai Phan","slug":"nhathai-phan","given":"NhatHai","family":"Phan","affiliation":"New Jersey Institute of Technology , Newark , New Jersey , USA"},{"name":"Minh N. Vu","slug":"minh-n-vu","given":"Minh N.","family":"Vu","affiliation":"University of Florida , Gainesville , Florida , USA"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"New Jersey Institute of Technology , Newark , New Jersey , USA"},{"name":"Ruoming Jin","slug":"ruoming-jin","given":"Ruoming","family":"Jin","affiliation":"Kent State University , Kent , Ohio , USA"},{"name":"Dejing Dou","slug":"dejing-dou","given":"Dejing","family":"Dou","affiliation":"University of Oregon , Eugene , Oregon , USA"},{"name":"Xintao Wu","slug":"xintao-wu","given":"Xintao","family":"Wu","affiliation":"University of Arkansas , Fayetteville , Arkansas , USA"},{"name":"My T. Thai","slug":"my-t-thai","given":"My T.","family":"Thai","affiliation":"University of Florida , Gainesville , Florida , USA"}],"abstract":"In this paper, we propose a novel Heterogeneous Gaussian Mechanism (HGM) to preserve differential privacy in deep neural networks, with provable robustness against adversarial examples. We first relax the constraint of the privacy budget in the traditional Gaussian Mechanism from (0, 1] to (0, infty), with a new bound of the noise scale to preserve differential privacy. The noise in our mechanism can be arbitrarily redistributed, offering a distinctive ability to address the trade-off between model utility and privacy loss. To derive provable robustness, our HGM is applied to inject Gaussian noise into the first hidden layer. Then, a tighter robustness bound is proposed. Theoretical analysis and thorough evaluations show that our mechanism notably improves the robustness of differentially private deep neural networks, compared with baseline approaches, under a variety of model attacks.","doi":"10.24963/ijcai.2019/660","url":"https://ijcai.org/proceedings/2019/660","pdf":"https://ijcai.org/proceedings/2019/0660.pdf","markdown":"https://ijcai.org/proceedings/2019/660.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/660","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":660,"pages":{"first":4753,"last":4759},"keywords":["Machine Learning: Deep Learning","Multidisciplinary Topics and Applications: Security and Privacy"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martin Abadi, Andy Chu, Ian Goodfel- low, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. arXiv:1607.00133, 2016.","title":"Deep learning with differential privacy","authors":"Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan Mcmahan, Ilya Mironov, Kunal Talwar, Li Zhang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bach et al., 2015] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10(7):e0130140, 07 2015.","title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","authors":"Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, Wojciech Samek","year":null,"venue":"PLoS ONE","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Balle and Wang, 2018] Borja Balle and Yu-Xiang Wang. Improving the Gaussian mechanism for differential pri- vacy: Analytical calibration and optimal denoising. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 394-403, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR.","title":"Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising","authors":"Borja Balle, Yu-Xiang Wang","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carlini and Wagner, 2017] N. Carlini and D. Wagner. To- wards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39-57, May 2017.","title":"Towards evaluating the robustness of neural networks","authors":"N Carlini, D Wagner","year":"2017","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chatzikokolakis et al., 2013] Konstantinos Chatzikoko- lakis, Miguel E. Andrés, Nicolás Emilio Bordenabe, and Catuscia Palamidessi. Broadening the scope of differential privacy using metrics. In Emiliano De Cristo- faro and Matthew Wright, editors, Privacy Enhancing Technologies, pages 82-102, 2013.","title":"Broadening the scope of differential privacy using metrics","authors":"Konstantinos Chatzikokolakis, Miguel E Andrés, Nicolás Emilio Bordenabe, Catuscia Palamidessi","year":"2013","venue":"Privacy Enhancing Technologies","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dong et al., 2017] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Discovering adversarial examples with momentum. CoRR, abs/1710.06081, 2017.","title":"Discovering adversarial examples with momentum","authors":"Dong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dwork and Roth, 2014] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foun- dations and Trends R in Theoretical Computer Science, 9(3-4):211-407, 2014.","title":"The algorithmic foundations of differential privacy","authors":"Roth ; Dwork, Cynthia Dwork, Aaron Roth","year":"2014","venue":"Foundations and Trends R in Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dwork et al., 2006] C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. Theory of Cryptography, pages 265-284, 2006.","title":"Calibrating noise to sensitivity in private data analysis","authors":"C Dwork, F Mcsherry, K Nissim, A Smith","year":"2006","venue":"Theory of Cryptography","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fredrikson et al., 2015] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security, CCS '15, pages 1322-1333, 2015.","title":"Model inversion attacks that exploit confidence information and basic countermeasures","authors":"Matt Fredrikson, Somesh Jha, Thomas Ristenpart","year":"2015","venue":"Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security, CCS '15","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/1412.6572, 2014.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hamm et al., 2017] J. Hamm, J. Luken, and Y. Xie. Crowd- ml: A library for privacy-preserving machine learning on smart devices. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6394-6398, 2017.","title":"Crowdml: A library for privacy-preserving machine learning on smart devices","authors":"J Hamm, J Luken, Y Xie","year":"2017","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hoeffding, 1963] Wassily Hoeffding. Probability inequali- ties for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.","title":"Wassily Hoeffding. Probability inequalities for sums of bounded random variables","authors":"Hoeffding","year":"1963","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kolter and Wong, 2017] J. Zico Kolter and Eric Wong. Provable defenses against adversarial examples via the convex outer adversarial polytope. CoRR, abs/1711.00851, 2017.","title":"Provable defenses against adversarial examples via the convex outer adversarial polytope","authors":"Kolter, ] J Zico Wong, Eric Kolter, Wong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krizhevsky and Hinton, 2009] Alex Krizhevsky and Geof- frey Hinton. Learning multiple layers of features from tiny images. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kurakin et al., 2016] Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. CoRR, abs/1607.02533, 2016.","title":"Adversarial examples in the physical world","authors":"Alexey Kurakin, Ian J Goodfellow, Samy Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lecun et al., 1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Y Lecun, L Bottou, Y Bengio, P Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lecuyer et al., 2018] Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Cer- tified robustness to adversarial examples with differential privacy. In arXiv:1802.03471, 2018.","title":"Certified robustness to adversarial examples with differential privacy","authors":"Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Suman Jana","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee and Kifer, 2018] Jaewoo Lee and Daniel Kifer. Con- centrated differentially private gradient descent with adap- tive per-iteration privacy budget. In Proceedings of the 24th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, pages 1656-1665, 2018.","title":"Concentrated differentially private gradient descent with adaptive per-iteration privacy budget","authors":"Kifer ; Lee, Jaewoo Lee, Daniel Kifer","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2016] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. CoRR, abs/1611.02770, 2016.","title":"Delving into transferable adversarial examples and black-box attacks","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Papernot et al., 2016] Nicolas Papernot, Patrick D. Mc- Daniel, Arunesh Sinha, and Michael P. Wellman. Towards the science of security and privacy in machine learning. CoRR, abs/1611.03814, 2016.","title":"Towards the science of security and privacy in machine learning","authors":"Nicolas Papernot, D Patrick, Arunesh Mc-Daniel, Michael P Sinha, Wellman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Phan et al., 2017] N. Phan, X. Wu, H. Hu, and D. Dou. Adaptive laplace mechanism: Differential privacy preser- vation in deep learning. In IEEE ICDM'17, 2017.","title":"Adaptive laplace mechanism: Differential privacy preservation in deep learning","authors":"N Phan, X Wu, H Hu, D Dou","year":"2017","venue":"IEEE ICDM'17","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Raghunathan et al., 2018] Aditi Raghunathan, Jacob Stein- hardt, and Percy Liang. Certified defenses against adver- sarial examples. CoRR, abs/1801.09344, 2018.","title":"Certified defenses against adversarial examples","authors":"Aditi Raghunathan, Jacob Steinhardt, Percy Liang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shokri et al., 2017] R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against ma- chine learning models. In 2017 IEEE Symposium on Secu- rity and Privacy (SP), pages 3-18, May 2017.","title":"Membership inference attacks against machine learning models","authors":"R Shokri, M Stronati, C Song, V Shmatikov","year":"2017","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2015] Yue Wang, Cheng Si, and Xintao Wu. Regression model fitting under differential privacy and model inversion attack. In Proceedings of the Twenty- Fourth International Joint Conference on Artificial Intel- ligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015, pages 1003-1009, 2015.","title":"Regression model fitting under differential privacy and model inversion attack","authors":"Yue Wang, Cheng Si, Xintao Wu","year":"2015","venue":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015","doi":null,"url":null,"internal_url":"/proceedings/2015/146"},{"position":26,"raw":"[Yu et al., 2019] L. Yu, L. Liu, C. Pu, M. Gursoy, and S. Truex. Differentially private model publishing for deep learning. In 2019 IEEE Symposium on Security and Pri- vacy (SP), pages 326-343, 2019.","title":"Differentially private model publishing for deep learning","authors":"L Yu, L Liu, C Pu, M Gursoy, S Truex","year":"2019","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}