{"type":"ScholarlyArticle","title":"Joint Partial Optimal Transport for Open Set Domain Adaptation","authors":[{"name":"Renjun Xu","slug":"renjun-xu","given":"Renjun","family":"Xu","affiliation":"Zhejiang University"},{"name":"Pelen Liu","slug":"pelen-liu","given":"Pelen","family":"Liu","affiliation":"Zhejiang University"},{"name":"Yin Zhang","slug":"yin-zhang","given":"Yin","family":"Zhang","affiliation":"Zhejiang University"},{"name":"Fang Cai","slug":"fang-cai","given":"Fang","family":"Cai","affiliation":""},{"name":"Jindong Wang","slug":"jindong-wang","given":"Jindong","family":"Wang","affiliation":"Microsoft Research; Stanford University"},{"name":"Shuoying Liang","slug":"shuoying-liang","given":"Shuoying","family":"Liang","affiliation":"Zhejiang University"},{"name":"Heting Ying","slug":"heting-ying","given":"Heting","family":"Ying","affiliation":"China Jiliang University"},{"name":"Jianwei Yin","slug":"jianwei-yin","given":"Jianwei","family":"Yin","affiliation":"Zhejiang University"}],"abstract":"Domain adaptation (DA) has achieved a resounding success to learn a good classifier by leveraging labeled data from a source domain to adapt to an unlabeled target domain. However, in a general setting when the target domain contains classes that are never observed in the source domain, namely in Open Set Domain Adaptation (OSDA), existing DA methods failed to work because of the interference of the extra unknown classes. This is a much more challenging problem, since it can easily result in negative transfer due to the mismatch between the unknown and known classes. Existing researches are susceptible to misclassification when target domain unknown samples in the feature space distributed near the decision boundary learned from the labeled source domain. To overcome this, we propose Joint Partial Optimal Transport (JPOT), fully utilizing information of not only the labeled source domain but also the discriminative representation of unknown class in the target domain. The proposed joint discriminative prototypical compactness loss can not only achieve intra-class compactness and inter-class separability, but also estimate the mean and variance of the unknown class through backpropagation, which remains intractable for previous methods due to the blindness about the structure of the unknown classes. To our best knowledge, this is the first optimal transport model for OSDA. Extensive experiments demonstrate that our proposed model can significantly boost the performance of open set domain adaptation on standard DA datasets.","doi":"10.24963/ijcai.2020/352","url":"https://ijcai.org/proceedings/2020/352","pdf":"https://ijcai.org/proceedings/2020/0352.pdf","markdown":"https://ijcai.org/proceedings/2020/352.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/352","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":352,"pages":{"first":2540,"last":2546},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Unsupervised Learning","Machine Learning: Deep Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24814/file_24814.m3u8","length_sec":297,"provider":"hls"}],"references":[{"position":1,"raw":"[Angenent et al., 2003] S. Angenent, S. Haker, and A. Tan- nenbaum. Minimizing flows for the monge-kantorovich problem. SIAM journal on mathematical analysis, 35(1):61-97, 2003.","title":"Minimizing flows for the monge-kantorovich problem","authors":"S Angenent, S Haker, A Tannenbaum","year":"2003","venue":"SIAM journal on mathematical analysis","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bendale and Boult, 2016] A. Bendale and T.E. Boult. To- wards open set deep networks. In Proceedings CVPR, pages 1563-1572, 2016.","title":"Towards open set deep networks","authors":"A Bendale, T E Boult","year":"2016","venue":"Proceedings CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019] C. Chen, Z. Chen, B. Jiang, and X. Jin. Joint domain alignment and discriminative feature learn- ing for unsupervised deep domain adaptation. In Proceed- ings AAAI, volume 33, pages 3296-3303, 2019.","title":"Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation","authors":"C Chen, Z Chen, B Jiang, X Jin","year":"2019","venue":"Proceedings AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Courty et al., 2016] N. Courty, R. Flamary, D. Tuia, and A. Rakotomamonjy. Optimal transport for domain adapta- tion. IEEE transactions on pattern analysis and machine intelligence, 39(9):1853-1865, 2016.","title":"Optimal transport for domain adaptation","authors":"N Courty, R Flamary, D Tuia, A Rakotomamonjy","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Courty et al., 2017] N. Courty, R. Flamary, A. Habrard, and A. Rakotomamonjy. Joint distribution optimal transporta- tion for domain adaptation. In Advances in Neural Infor- mation Processing Systems, pages 3730-3739, 2017.","title":"Joint distribution optimal transportation for domain adaptation","authors":"N Courty, R Flamary, A Habrard, A Rakotomamonjy","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Damodaran et al., 2018] B.B. Damodaran, B. Kellenberger, R. Flamary, D. Tuia, and N. Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised do- main adaptation. In ECCV, pages 467-483, 2018.","title":"Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation","authors":"B B Damodaran, B Kellenberger, R Flamary, D Tuia, N Courty","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Donahue et al., 2014] J. Donahue, Yangqing Jia, O. Vinyals, J. Hoffman, Ning Zhang, E. Tzeng, and T. Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In International conference on machine learning, pages 647-655, 2014.","title":"Decaf: A deep convolutional activation feature for generic visual recognition","authors":"J Donahue, Yangqing Jia, O Vinyals, J Hoffman, Ning Zhang, E Tzeng, T Darrell","year":"2014","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ganin and Lempitsky, 2015] Y. Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, pages 1180-1189, 2015.","title":"Unsupervised domain adaptation by backpropagation","authors":"Y Ganin, V Lempitsky","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"K He, X Zhang, S Ren, J Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Herath et al., 2019] S. Herath, M. Harandi, B. Fernando, and R. Nock. Min-max statistical alignment for transfer learning. In Proceedings CVPR, pages 9288-9297, 2019.","title":"Min-max statistical alignment for transfer learning","authors":"S Herath, M Harandi, B Fernando, R Nock","year":"2019","venue":"Proceedings CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hull, 2002] J. J. Hull. A database for handwritten text recognition research. IEEE Transactions on Pattern Anal- ysis & Machine Intelligence, 16(5):550-554, 2002.","title":"A database for handwritten text recognition research","authors":"J J Hull","year":"2002","venue":"IEEE Transactions on Pattern Analysis & Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jain et al., 2014] L.P. Jain, W.J. Scheirer, and T.E. Boult. Multi-class open set recognition using probability of in- clusion. In ECCV, pages 393-409. Springer, 2014.","title":"Multi-class open set recognition using probability of inclusion","authors":"L P Jain, W J Scheirer, T E Boult","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kang et al., 2019] Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. Contrastive adaptation net- work for unsupervised domain adaptation. In Proceedings CVPR, pages 4893-4902, 2019.","title":"Contrastive adaptation network for unsupervised domain adaptation","authors":"Guoliang Kang, Lu Jiang, Yi Yang, Alexander G Hauptmann","year":"2019","venue":"Proceedings CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lecun et al., 1998] Y. L. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Y L Lecun, L Bottou, Y Bengio, P Haffner","year":"1998","venue":"Proceedings IEEE","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2019] H. Liu, Z. Cao, M. Long, J. Wang, and Q. Yang. Separate to adapt: Open set domain adaptation via progressive separation. In Proceedings CVPR, pages 2927-2936, 2019.","title":"Separate to adapt: Open set domain adaptation via progressive separation","authors":"H Liu, Z Cao, M Long, J Wang, Q Yang","year":"2019","venue":"Proceedings CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Long et al., 2015] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International Confer- ence on Machine Learning, pages 97-105, 2015.","title":"Learning transferable features with deep adaptation networks","authors":"Mingsheng Long, Yue Cao, Jianmin Wang, Michael Jordan","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Long et al., 2016] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Unsupervised domain adap- tation with residual transfer networks. In NeurIPS, pages 136-144, 2016.","title":"Unsupervised domain adaptation with residual transfer networks","authors":"Mingsheng Long, Han Zhu, Jianmin Wang, Michael I Jordan","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Luo et al., 2017] Z. Luo, Y. Zou, J. Hoffman, and F. Li. La- bel efficient learning of transferable representations across domains and tasks. In Advances in Neural Information Processing Systems, pages 165-177, 2017.","title":"Label efficient learning of transferable representations across domains and tasks","authors":"Z Luo, Y Zou, J Hoffman, F Li","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Netzer et al., 2011] Y. Netzer, T. Wang, A. Coates, A. Bis- sacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. Nips Work- shop on Deep Learning & Unsupervised Feature Learn- ing, 2011.","title":"Reading digits in natural images with unsupervised feature learning. Nips Workshop on Deep Learning & Unsupervised Feature Learning","authors":"Y Netzer, T Wang, A Coates, A Bissacco, B Wu, A Y Ng","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pan and Yang, 2010] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowl- edge and data engineering, 22(10):1345-1359, 2010.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2010","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pan et al., 2019] Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong-Wah Ngo, and Tao Mei. Transferrable prototypical networks for unsupervised domain adapta- tion. In Proceedings CVPR, pages 2239-2247, 2019.","title":"Transferrable prototypical networks for unsupervised domain adaptation","authors":"Pan","year":"2019","venue":"Proceedings CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[S. and S., 2016] Baochen S. and Kate S. Deep coral: Cor- relation alignment for deep domain adaptation. In ECCV Workshops, 2016.","title":"Deep coral: Correlation alignment for deep domain adaptation","authors":"S, S Baochen, S, Kate S","year":"2016","venue":"ECCV Workshops","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Saenko et al., 2010] K. Saenko, B. Kulis, M. Fritz, and T. Darrell. Adapting visual category models to new do- mains. In ECCV, 2010.","title":"Adapting visual category models to new domains","authors":"K Saenko, B Kulis, M Fritz, T Darrell","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Saito et al., 2018] K. Saito, S. Yamamoto, Y. Ushiku, and T. Harada. Open set domain adaptation by backpropaga- tion. In Proceedings ECCV, pages 153-168, 2018.","title":"Open set domain adaptation by backpropagation","authors":"K Saito, S Yamamoto, Y Ushiku, T Harada","year":"2018","venue":"Proceedings ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Snell et al., 2017] J. Snell, K. Swersky, and R. Zemel. Pro- totypical networks for few-shot learning. In NeurIPS, pages 4077-4087, 2017.","title":"Prototypical networks for few-shot learning","authors":"J Snell, K Swersky, R Zemel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tzeng et al., 2014] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell. Deep domain confusion: Max- imizing for domain invariance. arXiv:1412.3474, 2014.","title":"Deep domain confusion: Maximizing for domain invariance","authors":"E Tzeng, J Hoffman, N Zhang, K Saenko, T Darrell","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Venkateswara et al., 2017] H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan. Deep hashing net- work for unsupervised domain adaptation. In Proceedings CVPR, pages 5018-5027, 2017.","title":"Deep hashing network for unsupervised domain adaptation","authors":"H Venkateswara, J Eusebio, S Chakraborty, S Panchanathan","year":"2017","venue":"Proceedings CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yan et al., 2018] Y. Yan, W. Li, H. Wu, H. Min, M. Tan, and Q. Wu. Semi-supervised optimal transport for hetero- geneous domain adaptation. In IJCAI, pages 2969-2975, 2018.","title":"Semi-supervised optimal transport for heterogeneous domain adaptation","authors":"Y Yan, W Li, H Wu, H Min, M Tan, Q Wu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/412"},{"position":29,"raw":"[You et al., 2019] K. You, M. Long, Z. Cao, J. Wang, and M. I Jordan. Universal domain adaptation. In Proceedings CVPR, pages 2720-2729, 2019.","title":"Universal domain adaptation","authors":"K You, M Long, Z Cao, J Wang, M I Jordan","year":"2019","venue":"Proceedings CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zellinger et al., 2017] W. Zellinger, T. Grubinger, E. Lughofer, T. Natschläger, and S. Saminger-Platz. Central moment discrepancy (cmd) for domain-invariant representation learning. arXiv:1702.08811, 2017.","title":"Central moment discrepancy (cmd) for domain-invariant representation learning","authors":"W Zellinger, T Grubinger, E Lughofer, T Natschläger, S Saminger-Platz","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}