{"type":"ScholarlyArticle","title":"Metric Learning in Optimal Transport for Domain Adaptation","authors":[{"name":"Tanguy Kerdoncuff","slug":"tanguy-kerdoncuff","given":"Tanguy","family":"Kerdoncuff","affiliation":"Univ Lyon , UJM-Saint-Etienne , CNRS , Institut d Optique Graduate School , Laboratoire Hubert Curien UMR 5516 , F-42023 , SAINT-ETIENNE , France"},{"name":"Rémi Emonet","slug":"remi-emonet","given":"Rémi","family":"Emonet","affiliation":"Univ Lyon , UJM-Saint-Etienne , CNRS , Institut d Optique Graduate School , Laboratoire Hubert Curien UMR 5516 , F-42023 , SAINT-ETIENNE , France"},{"name":"Marc Sebban","slug":"marc-sebban","given":"Marc","family":"Sebban","affiliation":"Univ Lyon , UJM-Saint-Etienne , CNRS , Institut d Optique Graduate School , Laboratoire Hubert Curien UMR 5516 , F-42023 , SAINT-ETIENNE , France"}],"abstract":"Domain  Adaptation  aims  at  benefiting  from  a  labeled dataset drawn from a source distribution to learn a model from examples generated from a different  but  related target distribution. Creating  a domain-invariant  representation  between  the  two source and target domains is the most widely technique  used. A  simple  and  robust  way  to  perform this task consists in (i) representing the two domains by  subspaces  described  by  their  respective eigenvectors and (ii) seeking a mapping function  which  aligns  them. In  this  paper,  we  propose to use Optimal Transport (OT) and its associated Wassertein distance to perform this alignment. While the idea of using OT in domain adaptation is not new, the original contribution of this paper is two-fold:  (i) we derive a generalization bound on the  target  error  involving  several  Wassertein  distances.   This  prompts  us  to  optimize  the  ground metric  of  OT  to  reduce  the  target  risk;  (ii)  from this  theoretical  analysis,  we  design  an  algorithm (MLOT) which optimizes a Mahalanobis distance leading to a transportation plan that adapts better. Extensive  experiments  demonstrate  the  effectiveness of this original approach.","doi":"10.24963/ijcai.2020/299","url":"https://ijcai.org/proceedings/2020/299","pdf":"https://ijcai.org/proceedings/2020/0299.pdf","markdown":"https://ijcai.org/proceedings/2020/299.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/299","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":299,"pages":{"first":2162,"last":2168},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24302/file_24302.m3u8","length_sec":275,"provider":"hls"}],"references":[{"position":1,"raw":"[Bay et al., 2006] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. In ECCV, 2006.","title":"Surf: Speeded up robust features","authors":"Herbert Bay, Tinne Tuytelaars, Luc Van Gool","year":"2006","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bellet et al., 2015] Aurélien Bellet, Amaury Habrard, and Marc Sebban. Metric learning. Synthesis Lectures on Ar- tificial Intelligence and Machine Learning, 2015.","title":"Metric learning","authors":"Aurélien Bellet, Amaury Habrard, Marc Sebban","year":"2015","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ben-David et al., 2007] Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In NIPS, 2007.","title":"Analysis of representations for domain adaptation","authors":"Shai Ben-David, John Blitzer, Koby Crammer, Fernando Pereira","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bhushan Damodaran et al., 2018] Bharath Bhushan Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, and Nicolas Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In ECCV, 2018.","title":"Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation","authors":"Benjamin Bharath Bhushan Damodaran, Rémi Kellenberger, Devis Flamary, Nicolas Tuia, Courty","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bolley et al., 2007] Franc ¸ois Bolley, Arnaud Guillin, and Cédric Villani. Quantitative concentration inequalities for empirical measures on non-compact spaces. PTRF, 2007.","title":"Quantitative concentration inequalities for empirical measures on non-compact spaces","authors":"Bolley","year":"2007","venue":"PTRF","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Courty et al., 2017a] Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal transportation f or domain adaptation. In NIPS, 2017.","title":"Joint distribution optimal transportation f or domain adaptation","authors":"Courty","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Courty et al., 2017b] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation. PAMI, 2017.","title":"Optimal transport for domain adaptation","authors":"Courty","year":"2017","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cuturi and Avis, 2014] Marco Cuturi and David Avis. Ground metric learning. JMLR, 2014.","title":"Ground metric learning","authors":"Marco Cuturi, David Avis","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Light- speed computation of optimal transport. In NIPS, 2013.","title":"Sinkhorn distances: Lightspeed computation of optimal transport","authors":"Marco Cuturi","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Davis et al., 2007] Jason V. Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S. Dhillon. Information- theoretic metric learning. In ICML, 2007.","title":"Informationtheoretic metric learning","authors":"Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, Inderjit S Dhillon","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Deshpande et al., 2019] Ishan Deshpande, Yuan-Ting Hu, Ruoyu Sun, Ayis Pyrros, Nasir Siddiqui, Sanmi Koyejo, Zhizhen Zhao, David A. Forsyth, and Alexander G. Schwing. Max-sliced wasserstein distance and its use for gans. CoRR, 2019.","title":"Max-sliced wasserstein distance and its use for gans","authors":"Ishan Deshpande, Yuan-Ting Hu, Ruoyu Sun, Ayis Pyrros, Nasir Siddiqui, Sanmi Koyejo, Zhizhen Zhao, David A Forsyth, Alexander G Schwing","year":"2019","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Donahue et al., 2014] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In ICML, 2014.","title":"Decaf: A deep convolutional activation feature for generic visual recognition","authors":"Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Fernando et al., 2013] Basura Fernando, Amaury Habrard, Marc Sebban, and Tinne Tuytelaars. Unsupervised visual domain adaptation using subspace alignment. In ICCV, 2013.","title":"Unsupervised visual domain adaptation using subspace alignment","authors":"Basura Fernando, Amaury Habrard, Marc Sebban, Tinne Tuytelaars","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Genevay et al., 2018] Aude Genevay, Gabriel Peyre, and Marco Cuturi. Learning generative models with sinkhorn divergences. In AISTATS, 2018.","title":"Learning generative models with sinkhorn divergences","authors":"Aude Genevay, Gabriel Peyre, Marco Cuturi","year":"2018","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gong et al., 2012] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, 2012.","title":"Geodesic flow kernel for unsupervised domain adaptation","authors":"Gong","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Halko et al., 2011] Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate ma- trix decompositions. SIAM review, 2011.","title":"Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions","authors":"Halko","year":"2011","venue":"SIAM review","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kantorovich, 1942] L.V. Kantorovich. On the transfer of masses. Dokl. Acad. Nauk. USSR37, 7-8, 1942.","title":"On the transfer of masses","authors":"L V Kantorovich","year":"1942","venue":"Dokl. Acad. Nauk. USSR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mansour et al., 2009] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv:0902.3430, 2009.","title":"Domain adaptation: Learning bounds and algorithms","authors":"Yishay Mansour, Mehryar Mohri, Afshin Rostamizadeh","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pan et al., 2011] Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Net- works, 2011.","title":"Domain adaptation via transfer component analysis","authors":"Ivor W Sinno Jialin Pan, James T Tsang, Qiang Kwok, Yang","year":"2011","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Paty and Cuturi, 2019] Franc ¸ois-Pierre Paty and Marco Cu- turi. Subspace robust wasserstein distances. arXiv preprint arXiv:1901.08949, 2019.","title":"Franc ¸ois-Pierre Paty and Marco Cuturi. Subspace robust wasserstein distances","authors":"Cuturi Paty","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Redko et al., 2017] Ievgen Redko, Amaury Habrard, and Marc Sebban. Theoretical analysis of domain adaptation with optimal transport. In ECML PKDD, 2017.","title":"Ievgen Redko, Amaury Habrard, and Marc Sebban. Theoretical analysis of domain adaptation with optimal transport","authors":"Redko","year":"2017","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Redko et al., 2019a] Ievgen Redko, Amaury Habrard, and Marc Sebban. On the analysis of adaptability in multi- source domain adaptation. Machine Learning, 2019.","title":"On the analysis of adaptability in multisource domain adaptation","authors":"Redko","year":"2019","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Redko et al., 2019b] Ievgen Redko, Emilie Morvant, Amaury Habrard, Marc Sebban, and Younès Bennani. Advances in Domain Adaptation Theory. Elsevier, 2019.","title":"Advances in Domain Adaptation Theory","authors":"Redko","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Saenko et al., 2010] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In ECCV, 2010.","title":"Adapting visual category models to new domains","authors":"Kate Saenko, Brian Kulis, Mario Fritz, Trevor Darrell","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shen et al., 2017] Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. Wasserstein distance guided represen- tation learning for domain adaptation. arXiv preprint arXiv:1707.01217, 2017.","title":"Wasserstein distance guided representation learning for domain adaptation","authors":"Jian Shen, Yanru Qu, Weinan Zhang, Yong Yu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun and Saenko, 2016] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adap- tation. In ECCV, 2016.","title":"Deep coral: Correlation alignment for deep domain adaptation","authors":"Saenko ; Sun, Baochen Sun, Kate Saenko","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sun et al., 2016] Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In AAAI, 2016.","title":"Return of frustratingly easy domain adaptation","authors":"Sun","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Villani, 2008] Cédric Villani. Optimal transport: old and new. Springer Science & Business Media, 2008.","title":"Optimal transport: old and new","authors":"Cédric Villani","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang and Deng, 2018] Mei Wang and Weihong Deng. Deep visual domain adaptation: A survey. Neurocomput- ing, 2018.","title":"Deep visual domain adaptation: A survey","authors":"Mei Wang, Weihong Deng","year":"2018","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2018] Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, and Philip S Yu. Visual domain adaptation with manifold embedded distribution alignment. In ACM-MM, 2018.","title":"Visual domain adaptation with manifold embedded distribution alignment","authors":"Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, Philip S Yu","year":"2018","venue":"ACM-MM","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Weinberger and Saul, 2009] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. JMLR, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"Q Kilian, Lawrence K Weinberger, Saul","year":"2009","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhong et al., 2010] Erheng Zhong, Wei Fan, Qiang Yang, Olivier Verscheure, and Jiangtao Ren. Cross validation framework to choose amongst models and datasets for transfer learning. In ECML PKDD, 2010.","title":"Cross validation framework to choose amongst models and datasets for transfer learning","authors":"Zhong","year":"2010","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}