{"type":"ScholarlyArticle","title":"Semi-supervised Domain Adaptation via Prototype-based Multi-level Learning","authors":[{"name":"Xinyang Huang","slug":"xinyang-huang","given":"Xinyang","family":"Huang","affiliation":"School of Artificial Intelligence , Beijing University of Posts and Telecommunications; School of Artificial Intelligence , Xidian University"},{"name":"Chuang Zhu","slug":"chuang-zhu","given":"Chuang","family":"Zhu","affiliation":"School of Artificial Intelligence , Beijing University of Posts and Telecommunications"},{"name":"Wenkai Chen","slug":"wenkai-chen","given":"Wenkai","family":"Chen","affiliation":"School of Artificial Intelligence , Beijing University of Posts and Telecommunications"}],"abstract":"In semi-supervised domain adaptation (SSDA), a few labeled target samples of each class help the model to transfer knowledge representation from the fully labeled source domain to the target domain. Many existing methods ignore the benefits of making full use of the labeled target samples from multi-level. To make better use of this additional data, we propose a novel Prototype-based Multi-level Learning (ProML) framework to better tap the potential of labeled target samples. To achieve intra-domain adaptation, we first introduce a pseudo-label aggregation based on the intra-domain optimal transport to help the model align the feature distribution of unlabeled target samples and the prototype. At the inter-domain level, we propose a cross-domain alignment loss to help the model use the target prototype for cross-domain knowledge transfer. We further propose a dual consistency based on prototype similarity and linear classifier to promote discriminative learning of compact target feature representation at the batch level. Extensive experiments on three datasets, including DomainNet, VisDA2017, and Office-Home, demonstrate that our proposed method achieves state-of-the-art performance in SSDA. Our code is available at https://github.com/bupt-ai-cz/ProML.","doi":"10.24963/ijcai.2023/98","url":"https://ijcai.org/proceedings/2023/98","pdf":"https://ijcai.org/proceedings/2023/0098.pdf","markdown":"https://ijcai.org/proceedings/2023/98.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/98","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":98,"pages":{"first":884,"last":892},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2019] Chao Chen, Zhihong Chen, Boyuan Jiang, and Xinyu Jin. Joint domain alignment and dis- criminative feature learning for unsupervised deep domain adaptation. In Proceedings of the AAAI conference on ar- tificial intelligence, volume 33, pages 3296-3303, 2019.","title":"Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation","authors":"Chao Chen, Zhihong Chen, Boyuan Jiang, Xinyu Jin","year":"2019","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Courty et al., 2017] Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal transportation for domain adaptation. Advances in Neural Information Processing Systems, 30, 2017.","title":"Joint distribution optimal transportation for domain adaptation","authors":"Courty","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cubuk et al., 2020] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical auto- mated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition workshops, pages 702-703, 2020.","title":"Randaugment: Practical automated data augmentation with a reduced search space","authors":"Barret Ekin D Cubuk, Jonathon Zoph, Quoc V Shlens, Le","year":"2020","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fatras et al., 2021] Kilian Fatras, Thibault Séjourné, Rémi Flamary, and Nicolas Courty. Unbalanced minibatch opti- mal transport; applications to domain adaptation. In Inter- national Conference on Machine Learning, pages 3186- 3197. PMLR, 2021.","title":"Unbalanced minibatch optimal transport; applications to domain adaptation","authors":"Kilian Fatras, Thibault Séjourné, Rémi Flamary, Nicolas Courty","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The jour- nal of machine learning research, 17(1):2096-2030, 2016.","title":"Domain-adversarial training of neural networks","authors":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, Victor Lempitsky","year":"2016","venue":"The journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Grandvalet and Bengio, 2004] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimiza- tion. Advances in neural information processing systems, 17, 2004.","title":"Semi-supervised learning by entropy minimization","authors":"Bengio Grandvalet, Yves Grandvalet, Yoshua Bengio","year":"2004","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jiang et al., 2020] Pin Jiang, Aming Wu, Yahong Han, Yun- feng Shao, Meiyu Qi, and Bingshuai Li. Bidirectional ad- versarial training for semi-supervised domain adaptation. In IJCAI, pages 934-940, 2020.","title":"Bidirectional adversarial training for semi-supervised domain adaptation","authors":"Jiang","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/130"},{"position":9,"raw":"[Kim and Kim, 2020] Taekyung Kim and Changick Kim. Attract, perturb, and explore: Learning a feature align- ment network for semi-supervised domain adaptation. In European conference on computer vision, pages 591-607. Springer, 2020.","title":"Attract, perturb, and explore: Learning a feature alignment network for semi-supervised domain adaptation","authors":"Kim ; Kim, Taekyung Kim, Changick Kim","year":"2020","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2021a] Jichang Li, Guanbin Li, Yemin Shi, and Yizhou Yu. Cross-domain adaptive clustering for semi- supervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2505-2514, 2021.","title":"Cross-domain adaptive clustering for semisupervised domain adaptation","authors":"Li","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2021b] Kai Li, Chang Liu, Handong Zhao, Yulun Zhang, and Yun Fu. Ecacl: A holistic framework for semi-supervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8578-8587, 2021.","title":"Ecacl: A holistic framework for semi-supervised domain adaptation","authors":"Li","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2017] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint adaptation networks. In International conference on machine learning, pages 2208-2217. PMLR, 2017.","title":"Deep transfer learning with joint adaptation networks","authors":"Mingsheng Long, Han Zhu, Jianmin Wang, Michael I Jordan","year":"2017","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Long et al., 2018] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial do- main adaptation. Advances in neural information process- ing systems, 31, 2018.","title":"Conditional adversarial domain adaptation","authors":"Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I Jordan","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pan et al., 2010] Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE transactions on neural net- works, 22(2):199-210, 2010.","title":"Domain adaptation via transfer component analysis","authors":"Ivor W Sinno Jialin Pan, James T Tsang, Qiang Kwok, Yang","year":"2010","venue":"IEEE transactions on neural networks","doi":null,"url":null,"internal_url":"/proceedings/2009/200"},{"position":15,"raw":"[Pei et al., 2018] Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. Multi-adversarial domain adap- tation. In Thirty-second AAAI conference on artificial in- telligence, 2018.","title":"Multi-adversarial domain adaptation","authors":"Zhongyi Pei, Zhangjie Cao, Mingsheng Long, Jianmin Wang","year":"2018","venue":"Thirty-second AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Peng et al., 2017] Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge, 2017.","title":"Visda: The visual domain adaptation challenge","authors":"Peng","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peng et al., 2019] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment match- ing for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pages 1406-1415, 2019.","title":"Moment matching for multi-source domain adaptation","authors":"Peng","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pérez-Carrasco et al., 2022] Manuel Pérez-Carrasco, Pav- los Protopapas, and Guillermo Cabrera-Vives. Con 2 da: Simplifying semi-supervised domain adaptation by learn- ing consistent and contrastive feature representations. arXiv preprint arXiv:2204.01558, 2022.","title":"Con 2 da: Simplifying semi-supervised domain adaptation by learning consistent and contrastive feature representations","authors":"Manuel Pérez-Carrasco, Pavlos Protopapas, Guillermo Cabrera-Vives","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Saito et al., 2019] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi- supervised domain adaptation via minimax entropy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8050-8058, 2019.","title":"Semisupervised domain adaptation via minimax entropy","authors":"Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, Kate Saenko","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Singh, 2021] Ankit Singh. Clda: Contrastive learning for semi-supervised domain adaptation. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Pro- cessing Systems, volume 34, pages 5089-5101.","title":"Clda: Contrastive learning for semi-supervised domain adaptation","authors":"Ankit Singh","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sohn et al., 2020] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with con- sistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.","title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun and Saenko, 2016] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adap- tation. In European conference on computer vision, pages 443-450. Springer, 2016.","title":"Deep coral: Correlation alignment for deep domain adaptation","authors":"Saenko ; Sun, Baochen Sun, Kate Saenko","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Venkateswara et al., 2017] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Pan- chanathan. Deep hashing network for unsupervised do- main adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018- 5027, 2017.","title":"Deep hashing network for unsupervised domain adaptation","authors":"Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, Sethuraman Panchanathan","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2022] Hai-Ming Xu, Lingqiao Liu, Qiuchen Bian, and Zhen Yang. Semi-supervised semantic segmenta- tion with prototype-based consistency regularization. Ad- vances in neural information processing systems, 2022.","title":"Semi-supervised semantic segmentation with prototype-based consistency regularization","authors":"Hai-Ming Xu, Lingqiao Liu, Qiuchen Bian, Zhen Yang","year":"2022","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yan et al., 2022] Zizheng Yan, Yushuang Wu, Guanbin Li, Yipeng Qin, Xiaoguang Han, and Shuguang Cui. Multi- level consistency learning for semi-supervised domain adaptation. arXiv preprint arXiv:2205.04066, 2022.","title":"Multilevel consistency learning for semi-supervised domain adaptation","authors":"Yan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2021] Luyu Yang, Yan Wang, Mingfei Gao, Abhinav Shrivastava, Kilian Q Weinberger, Wei-Lun Chao, and Ser-Nam Lim. Deep co-training with task de- composition for semi-supervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8906-8916, 2021.","title":"Deep co-training with task decomposition for semi-supervised domain adaptation","authors":"Yang","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}