{"type":"ScholarlyArticle","title":"Differentially Private Optimal Transport: Application to Domain Adaptation","authors":[{"name":"Nam LeTien","slug":"nam-letien","given":"Nam","family":"LeTien","affiliation":"Univ Lyon , UJM-Saint-Etienne , CNRS , Institut d'Optique Graduate School , Laboratoire Hubert Curien UMR 5516 , F-42023 , SAINT-ETIENNE , France"},{"name":"Amaury Habrard","slug":"amaury-habrard","given":"Amaury","family":"Habrard","affiliation":"Univ Lyon , UJM-Saint-Etienne , CNRS , Institut d'Optique Graduate School , Laboratoire Hubert Curien UMR 5516 , F-42023 , SAINT-ETIENNE , France"},{"name":"Marc Sebban","slug":"marc-sebban","given":"Marc","family":"Sebban","affiliation":"Univ Lyon , UJM-Saint-Etienne , CNRS , Institut d'Optique Graduate School , Laboratoire Hubert Curien UMR 5516 , F-42023 , SAINT-ETIENNE , France"}],"abstract":"Optimal transport has received much attention during the past few years to deal with domain adaptation tasks. The goal is to transfer knowledge from a source domain to a target domain by finding a transportation of minimal cost moving the source distribution to the target one. In this paper, we address the challenging task of privacy preserving domain adaptation by optimal transport. Using the Johnson-Lindenstrauss transform together with some noise, we present the first differentially private optimal transport model and show how it can be directly applied on both unsupervised and semi-supervised domain adaptation scenarios. Our theoretically grounded method allows the optimization of the transportation plan and the Wasserstein distance between the two distributions while protecting the data of both domains.We perform an extensive series of experiments on various benchmarks (VisDA, Office-Home and Office-Caltech datasets) that demonstrates the efficiency of our method compared to non-private strategies.","doi":"10.24963/ijcai.2019/395","url":"https://ijcai.org/proceedings/2019/395","pdf":"https://ijcai.org/proceedings/2019/0395.pdf","markdown":"https://ijcai.org/proceedings/2019/395.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/395","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":395,"pages":{"first":2852,"last":2858},"keywords":["Multidisciplinary Topics and Applications: Security and Privacy","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In CCS, pages 308-318. ACM, 2016.","title":"Deep learning with differential privacy","authors":"Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan Mcmahan, Ilya Mironov, Kunal Talwar, Li Zhang","year":"2016","venue":"CCS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv:1701.07875, 2017.","title":"Wasserstein gan","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Balle et al., 2018] Borja Balle, Gilles Barthe, and Marco Gaboardi. Privacy amplification by subsampling: Tight analyses via couplings and divergences. In NIPS, pages 6280-6290, 2018.","title":"Privacy amplification by subsampling: Tight analyses via couplings and divergences","authors":"Borja Balle, Gilles Barthe, Marco Gaboardi","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blocki et al., 2012] Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. The johnson-lindenstrauss transform itself pre- serves differential privacy. In FOCS, pages 410-419. IEEE, 2012.","title":"The johnson-lindenstrauss transform itself preserves differential privacy","authors":"Blocki","year":"2012","venue":"FOCS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chaudhuri and Monteleoni, 2009] Kamalika Chaudhuri and Claire Monteleoni. Privacy-preserving logistic regression. In NIPS, pages 289-296, 2009.","title":"Privacy-preserving logistic regression","authors":"Kamalika Chaudhuri, Claire Monteleoni","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chollet, 2015] Franc ¸ois Chollet. Keras. https://keras.io, 2015.","title":"Franc ¸ois Chollet. Keras","authors":"Chollet","year":"2015","venue":null,"doi":null,"url":"https://keras.io","internal_url":null},{"position":7,"raw":"[Courty et al., 2017a] Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal transportation for domain adaptation. In NIPS, pages 3730-3739, 2017.","title":"Joint distribution optimal transportation for domain adaptation","authors":"Courty","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Courty et al., 2017b] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adap- tation. IEEE TPAMI, 39(9):1853-1865, 2017.","title":"Optimal transport for domain adaptation","authors":"Courty","year":"2017","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Lightspeed com- putation of optimal transport. In NIPS, pages 2292-2300, 2013.","title":"Sinkhorn distances: Lightspeed computation of optimal transport","authors":"Marco Cuturi","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dwork et al., 2006] Cynthia Dwork, Frank McSherry, Kobbi Nis- sim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In TCC, pages 265-284. Springer, 2006.","title":"Calibrating noise to sensitivity in private data analysis","authors":"Cynthia Dwork, Frank Mcsherry, Kobbi Nissim, Adam Smith","year":"2006","venue":"TCC","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dwork et al., 2010] Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. Boosting and differential privacy. In FOCS, pages 51- 60. IEEE, 2010.","title":"Boosting and differential privacy","authors":"Cynthia Dwork, Guy N Rothblum, Salil Vadhan","year":"2010","venue":"FOCS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dwork et al., 2014] Cynthia Dwork, Aaron Roth, et al. The al- gorithmic foundations of differential privacy. Foundations and Trends R in Theoretical Computer Science, 9(3-4):211-407, 2014.","title":"The algorithmic foundations of differential privacy","authors":"Cynthia Dwork, Aaron Roth","year":"2014","venue":"Foundations and Trends R in Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Flamary and Courty, 2017] R'emi Flamary and Nicolas Courty. Pot python optimal transport library, 2017.","title":"R'emi Flamary and Nicolas Courty. Pot python optimal transport library","authors":"Courty Flamary","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Frogner et al., 2015] Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso A Poggio. Learning with a wasserstein loss. In NIPS, pages 2053-2061, 2015.","title":"Learning with a wasserstein loss","authors":"Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, Tomaso A Poggio","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR, 17(1):2096-2030, 2016.","title":"Domain-adversarial training of neural networks","authors":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Mario Franc ¸ois Laviolette, Victor Marchand, Lempitsky","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Guo et al., 2018] Xiawei Guo, Quanming Yao, WeiWei Tu, Yuqiang Chen, Wenyuan Dai, and Qiang Yang. Privacy- preserving transfer learning for knowledge sharing. arXiv preprint arXiv:1811.09491, 2018.","title":"Privacypreserving transfer learning for knowledge sharing","authors":"Guo","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hamm et al., 2016] Jihun Hamm, Yingjun Cao, and Mikhail Belkin. Learning privately from multiparty data. In ICML, pages 555-563, 2016.","title":"Learning privately from multiparty data","authors":"Jihun Hamm, Yingjun Cao, Mikhail Belkin","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hardt and Roth, 2013] Moritz Hardt and Aaron Roth. Beyond worst-case analysis in private singular vector computation. In STOC, pages 331-340. ACM, 2013.","title":"Beyond worst-case analysis in private singular vector computation","authors":"Roth ; Hardt, Moritz Hardt, Aaron Roth","year":"2013","venue":"STOC","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Johnson and Lindenstrauss, 1984] William B Johnson and Joram Lindenstrauss. Extensions of lipschitz mappings into a hilbert space. Contemporary mathematics, 26(189-206):1, 1984.","title":"Extensions of lipschitz mappings into a hilbert space","authors":"B William, Joram Johnson, Lindenstrauss","year":"1984","venue":"Contemporary mathematics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kantorovich, 1942] Leonid Vitalievich Kantorovich. On the translocation of masses. In Dokl. Akad. Nauk. USSR (NS), vol- ume 37, pages 199-201, 1942.","title":"Leonid Vitalievich Kantorovich. On the translocation of masses","authors":"Kantorovich","year":"1942","venue":"Dokl. Akad. Nauk. USSR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kenthapadi et al., 2013] Krishnaram Kenthapadi, Aleksandra Ko- rolova, Ilya Mironov, and Nina Mishra. Privacy via the johnson- lindenstrauss transform. Journal of Privacy and Confidentiality, 5(1), 2013.","title":"Privacy via the johnsonlindenstrauss transform","authors":"Kenthapadi","year":"2013","venue":"Journal of Privacy and Confidentiality","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pan and Yang, 2010] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345-1359, 2010.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2010","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Papernot et al., 2017] Nicolas Papernot, Martín , Ulfar Erlings- son, Ian Goodfellow, and Kunal Talwar. Semi-supervised knowl- edge transfer for deep learning from private training data. In ICLR, 2017.","title":"Semi-supervised knowledge transfer for deep learning from private training data","authors":"Nicolas Papernot, Ulfar Martín, Ian Erlingsson, Kunal Goodfellow, Talwar","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pathak et al., 2010] Manas Pathak, Shantanu Rane, and Bhiksha Raj. Multiparty differential privacy via aggregation of locally trained classifiers. In NIPS, pages 1876-1884, 2010.","title":"Multiparty differential privacy via aggregation of locally trained classifiers","authors":"Manas Pathak, Shantanu Rane, Bhiksha Raj","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Peng et al., 2017] Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv:1710.06924, 2017.","title":"Visda: The visual domain adaptation challenge","authors":"Peng","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Perrot et al., 2016] Michaël Perrot, Nicolas Courty, Rémi Flamary, and Amaury Habrard. Mapping estimation for discrete optimal transport. In NIPS, pages 4197-4205, 2016.","title":"Mapping estimation for discrete optimal transport","authors":"Michaël Perrot, Nicolas Courty, Rémi Flamary, Amaury Habrard","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Redko et al., 2017] Ievgen Redko, Amaury Habrard, and Marc Sebban. Theoretical analysis of domain adaptation with optimal transport. In ECML-PKDD, pages 737-753. Springer, 2017.","title":"Ievgen Redko, Amaury Habrard, and Marc Sebban. Theoretical analysis of domain adaptation with optimal transport","authors":"Redko","year":"2017","venue":"ECML-PKDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Reich, 2013] Sebastian Reich. A nonparametric ensemble trans- form method for bayesian inference. SIAM Journal on Scientific Computing, 35(4):A2013-A2024, 2013.","title":"A nonparametric ensemble transform method for bayesian inference","authors":"Sebastian Reich","year":"2013","venue":"SIAM Journal on Scientific Computing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rubinstein et al., 2009] Benjamin IP Rubinstein, Peter L Bartlett, Ling Huang, and Nina Taft. Learning in a large function space: Privacy-preserving mechanisms for svm learning. arXiv preprint arXiv:0911.5708, 2009.","title":"Learning in a large function space: Privacy-preserving mechanisms for svm learning","authors":"Peter L Benjamin Ip Rubinstein, Ling Bartlett, Nina Huang, Taft","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Saenko et al., 2010] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In ECCV, pages 213-226. Springer, 2010.","title":"Adapting visual category models to new domains","authors":"Saenko","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shen et al., 2018] Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. Wasserstein distance guided representation learning for do- main adaptation. In AAAI, 2018.","title":"Wasserstein distance guided representation learning for domain adaptation","authors":"Jian Shen, Yanru Qu, Weinan Zhang, Yong Yu","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Shokri and Shmatikov, 2015] Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In CCS, pages 1310-1321. ACM, 2015.","title":"Privacy-preserving deep learning","authors":"Reza Shokri, Vitaly Shmatikov","year":"2015","venue":"CCS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Tzeng et al., 2017] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, pages 7167-7176, 2017.","title":"Adversarial discriminative domain adaptation","authors":"Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Vempala, 2005] Santosh S Vempala. The random projection method, volume 65. American Mathematical Soc., 2005.","title":"The random projection method","authors":"S Santosh, Vempala","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Venkateswara et al., 2017] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hash- ing network for unsupervised domain adaptation. In CVPR, 2017.","title":"Deep hashing network for unsupervised domain adaptation","authors":"Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, Sethuraman Panchanathan","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Villani, 2008] Cédric Villani. Optimal transport: old and new, vol- ume 338. Springer Science & Business Media, 2008.","title":"Optimal transport: old and new","authors":"Villani","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wang et al., 2018] Yang Wang, Quanquan Gu, and Donald Brown. Differentially private hypothesis transfer learning. In ECML- PKDD, pages 811-826. Springer, 2018.","title":"Differentially private hypothesis transfer learning","authors":"Yang Wang, Quanquan Gu, Donald Brown","year":"2018","venue":"ECML-PKDD","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zoph et al., 2018] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In CVPR, pages 8697-8710, 2018.","title":"Learning transferable architectures for scalable image recognition","authors":"Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V Le","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}