{"type":"ScholarlyArticle","title":"MemREIN: Rein the Domain Shift for Cross-Domain Few-Shot Learning","authors":[{"name":"Yi Xu","slug":"yi-xu","given":"Yi","family":"Xu","affiliation":"Northeastern University"},{"name":"Lichen Wang","slug":"lichen-wang","given":"Lichen","family":"Wang","affiliation":"Northeastern University"},{"name":"Yizhou Wang","slug":"yizhou-wang","given":"Yizhou","family":"Wang","affiliation":"Northeastern University"},{"name":"Can Qin","slug":"can-qin","given":"Can","family":"Qin","affiliation":""},{"name":"Yulun Zhang","slug":"yulun-zhang","given":"Yulun","family":"Zhang","affiliation":"ETH Zürich"},{"name":"Yun Fu","slug":"yun-fu","given":"Yun","family":"Fu","affiliation":"Northeastern University"}],"abstract":"Few-shot learning aims to enable models generalize to new categories (query instances) with only limited labeled samples (support instances) from each category. Metric-based mechanism is a promising direction which compares feature embeddings via different metrics. However, it always fail to generalize to unseen domains due to the considerable domain gap challenge. In this paper, we propose a novel framework, MemREIN, which considers Memorized, Restitution, and Instance Normalization for cross-domain few-shot learning. Specifically, an instance normalization algorithm is explored to alleviate feature dissimilarity, which provides the initial model generalization ability. However, naively normalizing the feature would lose fine-grained discriminative knowledge between different classes. To this end, a memorized module is further proposed to separate the most refined knowledge and remember it. Then, a restitution module is utilized to restitute the discrimination ability from the learned knowledge. A novel reverse contrastive learning strategy is proposed to stabilize the distillation process. Extensive experiments on five popular benchmark datasets demonstrate that MemREIN well addresses the domain shift challenge, and significantly improves the performance up to 16.43% compared with state-of-the-art baselines.","doi":"10.24963/ijcai.2022/505","url":"https://ijcai.org/proceedings/2022/505","pdf":"https://ijcai.org/proceedings/2022/0505.pdf","markdown":"https://ijcai.org/proceedings/2022/505.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/505","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":505,"pages":{"first":3636,"last":3642},"keywords":["Computer Vision: Transfer, low-shot, semi- and un- supervised learning","Machine Learning: Multi-task and Transfer Learning","Machine Learning: Few-shot learning","Knowledge Representation and Reasoning: Applications","Knowledge Representation and Reasoning: Other"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986541","length_sec":73,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984684","length_sec":222,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Blanchard et al., 2011] Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classifi- cation tasks to a new unlabeled sample. In NeurIPS, 2011.","title":"Generalizing from several related classification tasks to a new unlabeled sample","authors":"Blanchard","year":"2011","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019] Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look at few-shot classification. In ICLR, 2019.","title":"A closer look at few-shot classification","authors":"Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank, Wang, Jia-Bin Huang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dumoulin et al., 2016] Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur. A learned representation for artistic style. In ICLR, 2016.","title":"A learned representation for artistic style","authors":"Jonathon Vincent Dumoulin, Manjunath Shlens, Kudlur","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fu et al., 2021] Yuqian Fu, Yanwei Fu, and Yu-Gang Jiang. Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data. In ACM MM, 2021.","title":"Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data","authors":"Fu","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2018] Jie Hu, Li Shen, and Gang Sun. Squeeze- and-excitation networks. In CVPR, 2018. [Islam et al., 2021]","title":"Squeezeand-excitation networks","authors":"Jie Hu, Li Shen, Gang Sun","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Ashraful Islam, Chun-Fu Chen, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, and Richard J Radke. Dynamic distillation network for cross-domain few-shot recognition with unlabeled data. In NeurIPS, 2021.","title":"Dynamic distillation network for cross-domain few-shot recognition with unlabeled data","authors":"Ashraful Islam, Chun-Fu Chen, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Richard J Radke","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jin et al., 2020] Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen, and Li Zhang. Style normalization and restitution for generalizable person re-identification. In CVPR, 2020.","title":"Style normalization and restitution for generalizable person re-identification","authors":"Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen, Li Zhang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Krause et al., 2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In ICCV Workshops, 2013.","title":"3d object representations for finegrained categorization","authors":"Jonathan Krause, Michael Stark, Jia Deng, Li Fei-Fei","year":"2013","venue":"ICCV Workshops","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kumar et al., 2018] Abhishek Kumar, Prasanna Sattigeri, Kahini Wadhawan, Leonid Karlinsky, Rogério Schmidt Feris, Bill Freeman, and Gregory W Wornell. Co- regularized alignment for unsupervised domain adapta- tion. In NeurIPS, 2018.","title":"Coregularized alignment for unsupervised domain adaptation","authors":"Abhishek Kumar, Prasanna Sattigeri, Kahini Wadhawan, Leonid Karlinsky, Rogério Schmidt Feris, Bill Freeman, Gregory W Wornell","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kundu et al., 2019] Jogendra Nath Kundu, Nishank Lakkakula, and R Venkatesh Babu. UM-Adapt: Unsuper- vised multi-task adaptation using adversarial cross-task distillation. In ICCV, 2019.","title":"UM-Adapt: Unsupervised multi-task adaptation using adversarial cross-task distillation","authors":"Kundu","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liang et al., 2021] Hanwen Liang, Qiong Zhang, Peng Dai, and Juwei Lu. Boosting the generalization capability in cross-domain few-shot learning via noise-enhanced super- vised autoencoder. In ICCV, 2021.","title":"Boosting the generalization capability in cross-domain few-shot learning via noise-enhanced supervised autoencoder","authors":"Hanwen Liang, Qiong Zhang, Peng Dai, Juwei Lu","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Long et al., 2015] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In ICML, 2015.","title":"Learning transferable features with deep adaptation networks","authors":"Mingsheng Long, Yue Cao, Jianmin Wang, Michael Jordan","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Muandet et al., 2013] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via in- variant feature representation. In ICML, 2013.","title":"Domain generalization via invariant feature representation","authors":"Muandet","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Phoo and Hariharan, 2020] Cheng Perng Phoo and Bharath Hariharan. Self-training for few-shot transfer across ex- treme task differences. In ICLR, 2020.","title":"Self-training for few-shot transfer across extreme task differences","authors":"Cheng Perng, Phoo, Bharath Hariharan","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ravi and Larochelle, 2017] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In ICLR, 2017. [Satorras and Estrach, 2018] Victor Garcia Satorras and Joan Bruna Estrach. Few-shot learning with graph neural networks. In ICLR, 2018.","title":"Optimization as a model for few-shot learning","authors":"Larochelle ; Ravi, Sachin Ravi, Hugo Larochelle","year":"2017","venue":"Few-shot learning with graph neural networks","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurIPS, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sun et al., 2021] Jiamei Sun, Sebastian Lapuschkin, Wo- jciech Samek, Yunqing Zhao, Ngai-Man Cheung, and Alexander Binder. Explanation-guided training for cross- domain few-shot classification. In ICPR, 2021.","title":"Explanation-guided training for crossdomain few-shot classification","authors":"Jiamei Sun, Sebastian Lapuschkin, Wojciech Samek, Yunqing Zhao, Ngai-Man Cheung, Alexander Binder","year":"2021","venue":"ICPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sung et al., 2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learn- ing. In CVPR, 2018.","title":"Learning to compare: Relation network for few-shot learning","authors":"Sung","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tseng et al., 2020] Hung-Yu Tseng, Hsin-Ying Lee, Jia-Bin Huang, and Ming-Hsuan Yang. Cross-domain few-shot classification via learned feature-wise transformation. In ICLR, 2020.","title":"Cross-domain few-shot classification via learned feature-wise transformation","authors":"Hung-Yu Tseng, Hsin-Ying Lee, Jia-Bin Huang, Ming-Hsuan Yang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tzeng et al., 2017] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, 2017.","title":"Adversarial discriminative domain adaptation","authors":"Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Van Horn et al., 2018] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inatural- ist species classification and detection dataset. In CVPR, 2018.","title":"The inaturalist species classification and detection dataset","authors":"Van Horn","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim- othy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In NeurIPS, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang and Deng, 2021] Haoqing Wang and Zhi-Hong Deng. Cross-domain few-shot classification via adversar- ial task augmentation. In IJCAI, 2021.","title":"Cross-domain few-shot classification via adversarial task augmentation","authors":"Wang, Deng, Haoqing Wang, Zhi-Hong Deng","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/149"},{"position":26,"raw":"[Wang et al., 2020] Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. Generalizing from a few ex- amples: A survey on few-shot learning. ACM Computing Surveys, 2020.","title":"Generalizing from a few examples: A survey on few-shot learning","authors":"Wang","year":"2020","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2022] Yi Xu, Lichen Wang, Yizhou Wang, and Yun Fu. Adaptive trajectory prediction via transferable gnn. arXiv:2203.05046, 2022.","title":"Adaptive trajectory prediction via transferable gnn","authors":"Yi Xu, Lichen Wang, Yizhou Wang, Yun Fu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2018] Baoyao Yang, Andy J. Ma, and Pong C. Yuen. Learning domain-shared group-sparse representa- tion for unsupervised domain adaptation. Pattern Recog- nition, 2018.","title":"Learning domain-shared group-sparse representation for unsupervised domain adaptation","authors":"Baoyao Yang, Andy J Ma, Pong C Yuen","year":"2018","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2017] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE TPAMI, 2017.","title":"Places: A 10 million image database for scene recognition","authors":"Zhou","year":"2017","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zou et al., 2021] Yixiong Zou, Shanghang Zhang, Jianpeng Yu, Yonghong Tian, and José MF Moura. Revisiting mid- level patterns for cross-domain few-shot recognition. In ACM MM, 2021.","title":"Revisiting midlevel patterns for cross-domain few-shot recognition","authors":"Zou","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}