{"type":"ScholarlyArticle","title":"Lower Bound of Locally Differentially Private Sparse Covariance Matrix Estimation","authors":[{"name":"Di Wang","slug":"di-wang","given":"Di","family":"Wang","affiliation":"Department of Computer Science and Engineering State University of New York at Buffalo , NY , USA."},{"name":"Jinhui Xu","slug":"jinhui-xu","given":"Jinhui","family":"Xu","affiliation":"Department of Computer Science and Engineering State University of New York at Buffalo , NY , USA."}],"abstract":"In this paper, we study the sparse covariance matrix estimation problem in the local differential privacy model, and give a non-trivial lower bound on the  non-interactive private minimax risk in the metric of squared spectral norm. We show that the lower bound is actually tight, as it matches a previous upper bound.\r\nOur main technique for achieving this lower bound is a general framework, called General Private Assouad Lemma, which is a considerable generalization of the previous private Assouad lemma and can be used as a general method for bounding the private minimax risk of  matrix-related estimation problems.","doi":"10.24963/ijcai.2019/665","url":"https://ijcai.org/proceedings/2019/665","pdf":"https://ijcai.org/proceedings/2019/0665.pdf","markdown":"https://ijcai.org/proceedings/2019/665.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/665","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":665,"pages":{"first":4788,"last":4794},"keywords":["Multidisciplinary Topics and Applications: Security and Privacy","Machine Learning: Trusted Machine Learning","Machine Learning: Feature Selection ; Learning Sparse Models"],"videos":[],"references":[{"position":1,"raw":"[Amin et al., 2018] Kareem Amin, Travis Dick, Alex Kulesza, Andres Munoz Medina, and Sergei Vassilvitskii. Private covariance estimation via iterative eigenvector sampling. 2018 NIPS workshop in Privacy-Preserving Machine Learning, 2018.","title":"Private covariance estimation via iterative eigenvector sampling","authors":"Kareem Amin, Travis Dick, Alex Kulesza, Andres Munoz Medina, Sergei Vassilvitskii","year":"2018","venue":"NIPS workshop in Privacy-Preserving Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2012] T Tony Cai, Harrison H Zhou, et al. Opti- mal rates of convergence for sparse covariance matrix esti- mation. The Annals of Statistics, 40(5):2389-2420, 2012.","title":"Optimal rates of convergence for sparse covariance matrix estimation","authors":"Tony Cai, Harrison H Zhou","year":"2012","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2016] T Tony Cai, Weidong Liu, Harrison H Zhou, et al. Estimating sparse precision matrix: Optimal rates of convergence and adaptive estimation. The Annals of Statistics, 44(2):455-488, 2016.","title":"Estimating sparse precision matrix: Optimal rates of convergence and adaptive estimation","authors":"Tony Cai, Weidong Liu, Harrison H Zhou","year":"2016","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2019] T Tony Cai, Yichen Wang, and Linjun Zhang. The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. arXiv preprint arXiv:1902.04495, 2019.","title":"The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy","authors":"Tony Cai, Yichen Wang, Linjun Zhang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Duchi and Rogers, 2019] John Duchi and Ryan Rogers. Lower bounds for locally private estimation via commu- nication complexity. arXiv preprint arXiv:1902.00582, 2019.","title":"Lower bounds for locally private estimation via communication complexity","authors":"Rogers ; Duchi, John Duchi, Ryan Rogers","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Duchi and Ruan, 2018] John C Duchi and Feng Ruan. The right complexity measure in locally private estima- tion: It is not the fisher information. arXiv preprint arXiv:1806.05756, 2018.","title":"The right complexity measure in locally private estimation: It is not the fisher information","authors":"C John, Feng Duchi, Ruan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duchi et al., 2013] John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical mini- max rates. In Foundations of Computer Science (FOCS), 2013 IEEE 54th Annual Symposium on, pages 429-438. IEEE, 2013.","title":"Local privacy and statistical minimax rates","authors":"Michael I John C Duchi, Martin J Jordan, Wainwright","year":"2013","venue":"Foundations of Computer Science (FOCS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duchi et al., 2018] John C Duchi, Michael I Jordan, and Martin J Wainwright. Minimax optimal procedures for lo- cally private estimation. Journal of the American Statisti- cal Association, 113(521):182-201, 2018.","title":"Minimax optimal procedures for locally private estimation","authors":"Michael I John C Duchi, Martin J Jordan, Wainwright","year":"2018","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dwork et al., 2006] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In TCC, volume 3876, pages 265-284. Springer, 2006.","title":"Calibrating noise to sensitivity in private data analysis","authors":"Cynthia Dwork, Frank Mcsherry, Kobbi Nissim, Adam Smith","year":"2006","venue":"TCC","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dwork et al., 2014] Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, and Li Zhang. Analyze gauss: op- timal bounds for privacy-preserving principal component analysis. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 11-20. ACM, 2014.","title":"Analyze gauss: optimal bounds for privacy-preserving principal component analysis","authors":"Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, Li Zhang","year":"2014","venue":"Proceedings of the forty-sixth annual ACM symposium on Theory of computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Erlingsson et al., 2014] Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC conference on computer and commu- nications security, pages 1054-1067. ACM, 2014.","title":"Aleksandra Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response","authors":"Erlingsson","year":"2014","venue":"Proceedings of the 2014 ACM SIGSAC conference on computer and communications security","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gaboardi et al., 2018] Marco Gaboardi, Ryan Rogers, and Or Sheffet. Locally private mean estimation: Z- test and tight confidence intervals. arXiv preprint arXiv:1810.08054, 2018.","title":"Locally private mean estimation: Ztest and tight confidence intervals","authors":"Marco Gaboardi, Ryan Rogers, Or Sheffet","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ge et al., 2018] Jason Ge, Zhaoran Wang, Mengdi Wang, and Han Liu. Minimax-optimal privacy-preserving sparse pca in distributed systems. In International Conference on Artificial Intelligence and Statistics, pages 1589-1598, 2018.","title":"Minimax-optimal privacy-preserving sparse pca in distributed systems","authors":"Jason Ge, Zhaoran Wang, Mengdi Wang, Han Liu","year":"2018","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Joseph et al., 2018] Matthew Joseph, Janardhan Kulkarni, Jieming Mao, and Zhiwei Steven Wu. Locally private gaus- sian estimation. arXiv preprint arXiv:1811.08382, 2018.","title":"Locally private gaussian estimation","authors":"Matthew Joseph, Janardhan Kulkarni, Jieming Mao, Zhiwei Steven Wu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kamath et al., 2018] Gautam Kamath, Jerry Li, Vikrant Singhal, and Jonathan Ullman. Privately learn- ing high-dimensional distributions. arXiv preprint arXiv:1805.00216, 2018.","title":"Privately learning high-dimensional distributions","authors":"Gautam Kamath, Jerry Li, Vikrant Singhal, Jonathan Ullman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Karwa and Vadhan, 2017] Vishesh Karwa and Salil Vadhan. Finite sample differentially private confidence intervals. arXiv preprint arXiv:1711.03908, 2017.","title":"Finite sample differentially private confidence intervals","authors":"Vadhan Karwa, Vishesh Karwa, Salil Vadhan","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Laurent and Massart, 2000] Beatrice Laurent and Pascal Massart. Adaptive estimation of a quadratic functional by model selection. Annals of Statistics, pages 1302-1338, 2000.","title":"Adaptive estimation of a quadratic functional by model selection","authors":"Massart Laurent, Beatrice Laurent, Pascal Massart","year":"2000","venue":"Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Near, 2018] Joe Near. Differential privacy at scale: Uber and berkeley collaboration. In Enigma 2018 (Enigma 2018), Santa Clara, CA, 2018. USENIX Association.","title":"Differential privacy at scale: Uber and berkeley collaboration","authors":"Joe Near","year":"2018","venue":"Enigma 2018)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Steinke and Ullman, 2017] Thomas Steinke and Jonathan Ullman. Tight lower bounds for differentially private se- lection. In 2017 IEEE 58th Annual Symposium on Founda- tions of Computer Science (FOCS), pages 552-563. IEEE, 2017.","title":"Tight lower bounds for differentially private selection","authors":"Ullman Steinke, Thomas Steinke, Jonathan Ullman","year":"2017","venue":"IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tang et al., 2017] Jun Tang, Aleksandra Korolova, Xiao- long Bai, Xueqiang Wang, and XiaoFeng Wang. Privacy loss in apple's implementation of differential privacy on macos 10.12. CoRR, abs/1709.02753, 2017.","title":"Privacy loss in apple's implementation of differential privacy on macos 10","authors":"Tang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tsybakov, 2008] Alexandre B. Tsybakov. Introduction to Nonparametric Estimation. Springer Publishing Company, Incorporated, 1st edition, 2008.","title":"Introduction to Nonparametric Estimation","authors":"B Alexandre, Tsybakov","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ullman, 2018] Jonathan Ullman. Tight lower bounds for locally differentially private selection. arXiv preprint arXiv:1802.02638, 2018.","title":"Tight lower bounds for locally differentially private selection","authors":"Jonathan Ullman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang and Xu, 2019a] Di Wang and Jinhui Xu. Differen- tially private high dimensional sparse covariance matrix estimation. CoRR, abs/1901.06413, 2019.","title":"Differentially private high dimensional sparse covariance matrix estimation","authors":"Wang, Xu, Di Wang, Jinhui Xu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang and Xu, 2019b] Di Wang and Jinhui Xu. On sparse linear regression in the local differential privacy model. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, California, USA, June 9-15, 2019, 2019.","title":"On sparse linear regression in the local differential privacy model","authors":"Wang, Xu, Di Wang, Jinhui Xu","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning, ICML 2019","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018] D Wang, M Huai, and J Xu. Differen- tially private sparse inverse covariance estimation. In 2018 IEEE Global Conference on Signal and Information Pro- cessing, GlobalSIP, pages 26-29, 2018.","title":"Differentially private sparse inverse covariance estimation","authors":"M Wang, Huai, Xu","year":"2018","venue":"IEEE Global Conference on Signal and Information Processing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wellner and others, 2013] Jon Wellner et al. Weak conver- gence and empirical processes: with applications to statis- tics. Springer Science & Business Media, 2013.","title":"Weak convergence and empirical processes: with applications to statistics","authors":"Jon Wellner, Wellner","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ye and Barg, 2018] Min Ye and Alexander Barg. Optimal schemes for discrete distribution estimation under locally differential privacy. IEEE Transactions on Information Theory, 64(8):5662-5676, 2018.","title":"Optimal schemes for discrete distribution estimation under locally differential privacy","authors":"Barg Ye, Min Ye, Alexander Barg","year":"2018","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}