{"type":"ScholarlyArticle","title":"Principal Component Analysis in the Local Differential Privacy Model","authors":[{"name":"Di Wang","slug":"di-wang","given":"Di","family":"Wang","affiliation":"Department of Computer Science and Engineering State University of New York at Buffalo , NY , USA."},{"name":"Jinhui Xu","slug":"jinhui-xu","given":"Jinhui","family":"Xu","affiliation":"Department of Computer Science and Engineering State University of New York at Buffalo , NY , USA."}],"abstract":"In this paper, we study the Principal Component Analysis (PCA) problem under the (distributed) non-interactive local differential privacy model.  For the low dimensional case, we show the optimal rate for the private minimax risk of the k-dimensional PCA using the squared subspace distance as the measurement. For the high dimensional row sparse case, we first give a lower bound  on the private minimax risk, . Then we provide an efficient algorithm to achieve a near optimal upper bound. Experiments on both synthetic and real world datasets confirm the theoretical guarantees of our algorithms.","doi":"10.24963/ijcai.2019/666","url":"https://ijcai.org/proceedings/2019/666","pdf":"https://ijcai.org/proceedings/2019/0666.pdf","markdown":"https://ijcai.org/proceedings/2019/666.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/666","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":666,"pages":{"first":4795,"last":4801},"keywords":["Machine Learning: Trusted Machine Learning","Multidisciplinary Topics and Applications: Security and Privacy"],"videos":[],"references":[{"position":1,"raw":"[Balcan et al., 2016] Maria-Florina Balcan, Simon Shaolei Du, Yining Wang, and Adams Wei Yu. An improved gap- dependency analysis of the noisy power method. In Con- ference on Learning Theory, pages 284-309, 2016.","title":"An improved gapdependency analysis of the noisy power method","authors":"Maria-Florina Balcan, Simon Shaolei Du, Yining Wang, Adams Wei Yu","year":"2016","venue":"Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barber et al., 1975] DC Barber, PJ Howlett, and RC Smart. Principal component analysis in medical research. Journal of Applied Statistics, 2(1):39-43, 1975.","title":"Principal component analysis in medical research","authors":"Barber","year":"1975","venue":"Journal of Applied Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhatia, 2013] Rajendra Bhatia. Matrix analysis, volume 169. Springer Science & Business Media, 2013.","title":"Matrix analysis","authors":"Rajendra Bhatia","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blum et al., 2005] Avrim Blum, Cynthia Dwork, Frank Mc- Sherry, and Kobbi Nissim. Practical privacy: the sulq framework. In Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 128-138. ACM, 2005.","title":"Practical privacy: the sulq framework","authors":"Avrim Blum, Cynthia Dwork, Frank Mc-Sherry, Kobbi Nissim","year":"2005","venue":"Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cai et al., 2013] T Tony Cai, Zongming Ma, Yihong Wu, et al. Sparse pca: Optimal rates and adaptive estimation. The Annals of Statistics, 41(6):3074-3110, 2013.","title":"Sparse pca: Optimal rates and adaptive estimation","authors":"Tony Cai, Zongming Ma, Yihong Wu","year":"2013","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chaudhuri et al., 2013] Kamalika Chaudhuri, Anand D Sar- wate, and Kaushik Sinha. A near-optimal algorithm for differentially-private principal components. The Journal of Machine Learning Research, 14(1):2905-2943, 2013.","title":"A near-optimal algorithm for differentially-private principal components","authors":"Kamalika Chaudhuri, Anand D Sarwate, Kaushik Sinha","year":"2013","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Costello and Osborne, 2005] Anna B Costello and Jason W Osborne. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research & evaluation, 10(7):1-9, 2005.","title":"Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis","authors":"B Anna, Jason W Costello, Osborne","year":"2005","venue":"research & evaluation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dheeru and Karra Taniskidou, 2017] Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository, 2017.","title":"Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository","authors":"Karra Dheeru, Taniskidou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Duchi et al., 2013] John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical mini- max rates. In Foundations of Computer Science (FOCS), 2013 IEEE 54th Annual Symposium on, pages 429-438. IEEE, 2013.","title":"Local privacy and statistical minimax rates","authors":"Michael I John C Duchi, Martin J Jordan, Wainwright","year":"2013","venue":"Foundations of Computer Science (FOCS)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dwork et al., 2014] Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, and Li Zhang. Analyze gauss: op- timal bounds for privacy-preserving principal component analysis. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 11-20. ACM, 2014.","title":"Analyze gauss: optimal bounds for privacy-preserving principal component analysis","authors":"Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, Li Zhang","year":"2014","venue":"Proceedings of the forty-sixth annual ACM symposium on Theory of computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Erlingsson et al., 2014] Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. In CCS, pages 1054- 1067. ACM, 2014.","title":"Aleksandra Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response","authors":"Erlingsson","year":"2014","venue":"CCS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ge et al., 2018] Jason Ge, Zhaoran Wang, Mengdi Wang, and Han Liu. Minimax-optimal privacy-preserving sparse pca in distributed systems. In International Conference on Artificial Intelligence and Statistics, pages 1589-1598, 2018.","title":"Minimax-optimal privacy-preserving sparse pca in distributed systems","authors":"Jason Ge, Zhaoran Wang, Mengdi Wang, Han Liu","year":"2018","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gonem and Gilad-Bachrach, 2018] Alon Gonem and Ram Gilad-Bachrach. Smooth sensitivity based approach for differentially private pca. In Algorithmic Learning Theory, pages 438-450, 2018.","title":"Alon Gonem and Ram Gilad-Bachrach. Smooth sensitivity based approach for differentially private pca","authors":"Gilad-Bachrach Gonem","year":"2018","venue":"Algorithmic Learning Theory","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jiang et al., 2016] Wuxuan Jiang, Cong Xie, and Zhihua Zhang. Wishart mechanism for differentially private prin- cipal components analysis. In AAAI, pages 1730-1736, 2016.","title":"Wishart mechanism for differentially private prin-cipal components analysis","authors":"Jiang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kapralov and Talwar, 2013] Michael Kapralov and Kunal Talwar. On differentially private low rank approxima- tion. In Proceedings of the twenty-fourth annual ACM- SIAM symposium on Discrete algorithms, pages 1395- 1414. SIAM, 2013.","title":"On differentially private low rank approximation","authors":"Talwar Kapralov, Michael Kapralov, Kunal Talwar","year":"2013","venue":"Proceedings of the twenty-fourth annual ACM-SIAM symposium on Discrete algorithms","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2017] Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, and Huan Liu. Feature selection: A data perspective. ACM Comput- ing Surveys (CSUR), 50(6):94, 2017.","title":"Feature selection: A data perspective","authors":"Li","year":"2017","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lu and Xu, 2013] Dongsheng Lu and Shuhua Xu. Principal component analysis reveals the 1000 genomes project does not sufficiently cover the human genetic diversity in asia. Frontiers in genetics, 4:127, 2013.","title":"Principal component analysis reveals the 1000 genomes project does not sufficiently cover the human genetic diversity in asia","authors":"Lu, Xu ; ] Dongsheng Lu, Shuhua Xu","year":"2013","venue":"Frontiers in genetics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Near, 2018] Joe Near. Differential privacy at scale: Uber and berkeley collaboration. In Enigma 2018 (Enigma 2018), Santa Clara, CA, 2018. USENIX Association.","title":"Differential privacy at scale: Uber and berkeley collaboration","authors":"Joe Near","year":"2018","venue":"Enigma 2018)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Smith et al., 2017] Adam Smith, Abhradeep Thakurta, and Jalaj Upadhyay. Is interaction necessary for distributed pri- vate learning? In Security and Privacy (SP), 2017 IEEE Symposium on, pages 58-77. IEEE, 2017.","title":"Is interaction necessary for distributed private learning? In Security and Privacy (SP)","authors":"Adam Smith, Abhradeep Thakurta, Jalaj Upadhyay","year":"2017","venue":"IEEE Symposium on","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Stewart, 1990] G. W. Stewart. Matrix perturbation theory, 1990.","title":"Matrix perturbation theory","authors":"G W Stewart","year":"1990","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vu et al., 2013a] Vincent Q Vu, Juhee Cho, Jing Lei, and Karl Rohe. Fantope projection and selection: A near- optimal convex relaxation of sparse pca. In Advances in neural information processing systems, pages 2670-2678, 2013.","title":"Fantope projection and selection: A nearoptimal convex relaxation of sparse pca","authors":"Vu","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vu et al., 2013b] Vincent Q Vu, Jing Lei, et al. Minimax sparse principal subspace estimation in high dimensions. The Annals of Statistics, 41(6):2905-2947, 2013.","title":"Minimax sparse principal subspace estimation in high dimensions","authors":"Vu","year":"2013","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}