{"type":"ScholarlyArticle","title":"GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning","authors":[{"name":"Qiao Zhang","slug":"qiao-zhang","given":"Qiao","family":"Zhang","affiliation":"Old Dominion University , Norfolk , VA 23529 , USA"},{"name":"Cong Wang","slug":"cong-wang","given":"Cong","family":"Wang","affiliation":"Old Dominion University , Norfolk , VA 23529 , USA"},{"name":"Hongyi Wu","slug":"hongyi-wu","given":"Hongyi","family":"Wu","affiliation":"Old Dominion University , Norfolk , VA 23529 , USA"},{"name":"Chunsheng Xin","slug":"chunsheng-xin","given":"Chunsheng","family":"Xin","affiliation":"Old Dominion University , Norfolk , VA 23529 , USA"},{"name":"Tran V. Phuong","slug":"tran-v-phuong","given":"Tran V.","family":"Phuong","affiliation":"Old Dominion University , Norfolk , VA 23529 , USA"}],"abstract":"Privacy is a fundamental challenge for a variety of smart applications that depend on data aggregation and collaborative learning across different entities. In this paper, we propose a novel privacy-preserved architecture where clients can collaboratively train a deep model while preserving the privacy of each client’s data. Our main strategy is to carefully partition a deep neural network to two non-colluding parties. One party performs linear computations on encrypted data utilizing a less complex homomorphic cryptosystem, while the other executes non-polynomial computations in plaintext but in a privacy-preserved manner. We analyze security and compare the communication and computation complexity with the existing approaches. Our extensive experiments on different datasets demonstrate not only stable training without accuracy loss, but also 14 to 35 times speedup compared to the state-of-the-art system.","doi":"10.24963/ijcai.2018/547","url":"https://ijcai.org/proceedings/2018/547","pdf":"https://ijcai.org/proceedings/2018/0547.pdf","markdown":"https://ijcai.org/proceedings/2018/547.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/547","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":547,"pages":{"first":3933,"last":3939},"keywords":["Multidisciplinary Topics and Applications: Security and Privacy","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Boneh et al., 2005] Dan Boneh, Eu-Jin Goh, and Kobbi Nissim. Evaluating 2-dnf formulas on ciphertexts. In Pro- ceedings of Theory of Cryptography Conference, volume 3378, pages 325-341, 2005.","title":"Evaluating 2-dnf formulas on ciphertexts","authors":"Dan Boneh, Eu-Jin, Kobbi Goh, Nissim","year":"2005","venue":"Proceedings of Theory of Cryptography Conference","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Campbell and Meyer, 2009] Stephen L Campbell and Car- l D Meyer. Generalized inverses of linear transformations. SIAM, 2009.","title":"Generalized inverses of linear transformations","authors":"L Stephen, Carl D Campbell, Meyer","year":"2009","venue":"SIAM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Zhong, 2009] Tingting Chen and Sheng Zhong. Privacy-preserving backpropagation neural network learning. IEEE Transactions on Neural Networks, 20(10):1554-1564, 2009.","title":"Privacy-preserving backpropagation neural network learning","authors":"Tingting Chen, Sheng Zhong","year":"2009","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[ElGamal, 1985] Taher ElGamal. A public key cryptosys- tem and a signature scheme based on discrete logarithm- s. IEEE Transactions on Information Theory, 31(4):469- 472, 1985.","title":"A public key cryptosystem and a signature scheme based on discrete logarithms","authors":"Taher Elgamal","year":"1985","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gentry, 2009] Craig Gentry. Fully homomorphic encryption using ideal lattices. In Proceedings of the Forty-first Annu- al ACM Symposium on Theory of Computing, pages 169- 178, 2009.","title":"Fully homomorphic encryption using ideal lattices","authors":"Craig Gentry","year":"2009","venue":"Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gilad-Bachrach et al., 2016] Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In Pro- ceedings of International Conference on Machine Learn- ing, pages 201-210, 2016.","title":"Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy","authors":"Gilad-Bachrach","year":"2016","venue":"Proceedings of International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goldreich, 2009] Oded Goldreich. Foundations of cryptog- raphy: volume 2, basic applications. Cambridge Univer- sity Press, 2009.","title":"Foundations of cryptography: volume 2, basic applications","authors":"Oded Goldreich, Goldreich","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hitaj et al., 2017] Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: Infor- mation leakage from collaborative deep learning. In Pro- ceedings of the 24nd ACM SIGSAC Conference on Com- puter and Communications Security, pages 1310-1321, 2017.","title":"Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: Information leakage from collaborative deep learning","authors":"Hitaj","year":"2017","venue":"Proceedings of the 24nd ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu, 2013] Yin Hu. Improving the efficiency of homomor- phic encryption schemes. PhD thesis, Worcester Polytech- nic Institute, 2013.","title":"Improving the efficiency of homomorphic encryption schemes","authors":"Yin Hu","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Livni et al., 2014] Roi Livni, Shai Shalev-Shwartz, and O- had Shamir. On the computational efficiency of training neural networks. In Proceedings of the 27th Internation- al Conference on Neural Information Processing Systems -Volume 1, pages 855-863, 2014.","title":"On the computational efficiency of training neural networks","authors":"Roi Livni, Shai Shalev-Shwartz, Ohad Shamir","year":"2014","venue":"Proceedings of the 27th International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mahendran and Vedaldi, 2015] Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representa- tions by inverting them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5188-5196, 2015.","title":"Understanding deep image representations by inverting them","authors":"Vedaldi Mahendran, Aravindh Mahendran, Andrea Vedaldi","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Nguyen et al., 2015] Anh Nguyen, Jason Yosinski, and Jef- f Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 427-436, 2015.","title":"Deep neural networks are easily fooled: High confidence predictions for unrecognizable images","authors":"Anh Nguyen, Jason Yosinski, Jeff Clune","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Paillier, 1999] Pascal Paillier. Public-key cryptosystems based on composite degree residuosity classes. In Euro- crypt, volume 99, pages 223-238, 1999.","title":"Pascal Paillier. Public-key cryptosystems based on composite degree residuosity classes","authors":"Paillier","year":"1999","venue":"Eurocrypt","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Papernot et al., 2017] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In Proceedings of the ACM Asia Conference on Computer and Communications Security, pages 506-519, 2017.","title":"Practical black-box attacks against machine learning","authors":"Nicolas Papernot, Patrick Mcdaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, Ananthram Swami","year":"2017","venue":"Proceedings of the ACM Asia Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Schneier, 2007] Bruce Schneier. Applied cryptography: protocols, algorithms, and source code in C. John Wiley & Sons, 2007.","title":"Applied cryptography: protocols, algorithms, and source code in C","authors":"Bruce Schneier","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shokri and Shmatikov, 2015] Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proceed- ings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1310-1321, 2015.","title":"Privacy-preserving deep learning","authors":"Reza Shokri, Vitaly Shmatikov","year":"2015","venue":"Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shokri et al., 2017] Reza Shokri, Marco Stronati, Con- gzheng Song, and Vitaly Shmatikov. Membership infer- ence attacks against machine learning models. In Proceed- ings of IEEE Symposium on Security and Privacy, pages 3-18, 2017.","title":"Membership inference attacks against machine learning models","authors":"Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov","year":"2017","venue":"Proceedings of IEEE Symposium on Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Smart and Vercauteren, 2010] Nigel P Smart and Frederik Vercauteren. Fully homomorphic encryption with relative- ly small key and ciphertext sizes. In Proceedings of In- ternational Workshop on Public Key Cryptography, pages 420-443, 2010.","title":"Fully homomorphic encryption with relatively small key and ciphertext sizes","authors":"P Nigel, Frederik Smart, Vercauteren","year":"2010","venue":"Proceedings of International Workshop on Public Key Cryptography","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tramèr et al., 2016] Florian Tramèr, Fan Zhang, Ari Juel- s, Michael K. Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In Proceed- ings of 25th USENIX Security Symposium, pages 601-618, 2016.","title":"Stealing machine learning models via prediction apis","authors":"Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, Thomas Ristenpart","year":"2016","venue":"Proceedings of 25th USENIX Security Symposium","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yuan and Yu, 2014] Jiawei Yuan and Shucheng Yu. Pri- vacy preserving back-propagation neural network learn- ing made practical with cloud computing. IEEE Transac- tions on Parallel and Distributed Systems, 25(1):212-221, 2014.","title":"Privacy preserving back-propagation neural network learning made practical with cloud computing","authors":"Yu ; ] Jiawei Yuan, Shucheng Yuan, Yu","year":"2014","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2017] Yuan Zhang, Qingjun Chen, and Sheng Zhong. Efficient and privacy-preserving min and k th min computations in mobile sensing systems. IEEE Transac- tions on Dependable and Secure Computing, 14(1):9-21, 2017.","title":"Efficient and privacy-preserving min and k th min computations in mobile sensing systems","authors":"Yuan Zhang, Qingjun Chen, Sheng Zhong","year":"2017","venue":"IEEE Transactions on Dependable and Secure Computing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}