{"type":"ScholarlyArticle","title":"Clustered-patch Element Connection for Few-shot Learning","authors":[{"name":"Jinxiang Lai","slug":"jinxiang-lai","given":"Jinxiang","family":"Lai","affiliation":"Tencent Youtu Lab , China"},{"name":"Siqian Yang","slug":"siqian-yang","given":"Siqian","family":"Yang","affiliation":"Tencent Youtu Lab , China"},{"name":"Junhong Zhou","slug":"junhong-zhou","given":"Junhong","family":"Zhou","affiliation":"Southern University of Science and Technology , China"},{"name":"Wenlong Wu","slug":"wenlong-wu","given":"Wenlong","family":"Wu","affiliation":"Tencent Youtu Lab , China"},{"name":"Xiaochen Chen","slug":"xiaochen-chen","given":"Xiaochen","family":"Chen","affiliation":"Tencent Youtu Lab , China"},{"name":"Jun Liu","slug":"jun-liu","given":"Jun","family":"Liu","affiliation":"Tencent Youtu Lab , China"},{"name":"Bin-Bin Gao","slug":"bin-bin-gao","given":"Bin-Bin","family":"Gao","affiliation":"Tencent Youtu Lab , China"},{"name":"Chengjie Wang","slug":"chengjie-wang","given":"Chengjie","family":"Wang","affiliation":"Tencent Youtu Lab , China; Shanghai Jiao Tong University , China"}],"abstract":"Weak feature representation problem has influenced the performance of few-shot classification task for a long time. To alleviate this problem, recent researchers build connections between support and query instances through embedding patch features to generate discriminative representations. However, we observe that there exists semantic mismatches (foreground/ background) among these local patches, because the location and size of the target object are not fixed. What is worse, these mismatches result in unreliable similarity confidences, and complex dense connection exacerbates the problem. According to this, we propose a novel Clustered-patch Element Connection (CEC) layer to correct the mismatch problem. The CEC layer leverages Patch Cluster and Element Connection operations to collect and establish reliable connections with high similarity patch features, respectively. Moreover, we propose a CECNet, including CEC layer based attention module and distance metric. The former is utilized to generate a more discriminative representation benefiting from the global clustered-patch features, and the latter is introduced to reliably measure the similarity between pair-features. Extensive experiments demonstrate that our CECNet outperforms the state-of-the-art methods on classification benchmark. Furthermore, our CEC approach can be extended into few-shot segmentation and detection tasks, which achieves competitive performances.","doi":"10.24963/ijcai.2023/110","url":"https://ijcai.org/proceedings/2023/110","pdf":"https://ijcai.org/proceedings/2023/0110.pdf","markdown":"https://ijcai.org/proceedings/2023/110.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/110","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":110,"pages":{"first":991,"last":998},"keywords":["Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Bolei et al., 2016] Zhou Bolei, Khosla Aditya, Lapedriza Agata, Oliva Aude, and Torralba Antonio. Learning deep features for discriminative localization. In CVPR, 2016.","title":"Learning deep features for discriminative localization","authors":"Zhou Bolei, Aditya Khosla, Lapedriza Agata, Oliva Aude, Torralba Antonio","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bruna et al., 2013] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and lo- cally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013.","title":"Spectral networks and locally connected networks on graphs","authors":"Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann Lecun","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hou et al., 2019] Ruibing Hou, Hong Chang, MA Bing- peng, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. In NeurIPS, 2019.","title":"Cross attention network for few-shot classification","authors":"Hou","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Jiangtao et al., 2022] Xie Jiangtao, Long Fei, Lv Jiaming, Wang Qilong, and Li Peihua. Joint distribution matters: Deep brownian distance covariance for few-shot classifi- cation. In CVPR, 2022.","title":"Joint distribution matters: Deep brownian distance covariance for few-shot classification","authors":"Xie Jiangtao, Long Fei, Lv Jiaming, Wang Qilong, Li Peihua","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jinxiang and Siqian, 2022] Lai Jinxiang and Yang Siqian. Adaptive multi distance metrics for few-shot classification. In arXiv, 2022.","title":"Adaptive multi distance metrics for few-shot classification","authors":"Lai Jinxiang, Yang Siqian","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kang et al., 2019] Bingyi Kang, Zhuang Liu, Xin Wang, Fisher Yu, Jiashi Feng, and Trevor Darrell. Few-shot ob- ject detection via feature reweighting. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 8420-8429, 2019.","title":"Few-shot object detection via feature reweighting","authors":"Bingyi Kang, Zhuang Liu, Xin Wang, Fisher Yu, Jiashi Feng, Trevor Darrell","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2020] Yongfei Liu, Xiangyi Zhang, Songyang Zhang, and Xuming He. Part-aware prototype network for few-shot semantic segmentation. In ECCV, 2020.","title":"Part-aware prototype network for few-shot semantic segmentation","authors":"Liu","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Malik et al., 2021] Boudiaf Malik, Kervadec Hoel, Im- tiaz Masud Ziko, Piantanida Pablo, Ben Ayed Ismail, and Dolz Jose. Few-shot segmentation without meta-learning: A good transductive inference is all you need? In CVPR, 2021.","title":"Few-shot segmentation without meta-learning: A good transductive inference is all you need?","authors":"Boudiaf Malik, Kervadec Hoel, Imtiaz Masud Ziko, Piantanida Pablo, Ben Ayed Ismail, Dolz Jose","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Qiao et al., 2021] Limeng Qiao, Yuxuan Zhao, Zhiyuan Li, Xi Qiu, Jianan Wu, and Chi Zhang. Defrcn: Decoupled faster r-cnn for few-shot object detection. In ICCV, 2021.","title":"Defrcn: Decoupled faster r-cnn for few-shot object detection","authors":"Qiao","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rizve et al., 2021] Mamshad Nayeem Rizve, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Exploring com- plementary strengths of invariant and equivariant represen- tations for few-shot learning. In CVPR, 2021.","title":"Exploring complementary strengths of invariant and equivariant representations for few-shot learning","authors":"Rizve","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Siam et al., 2019] Mennatullah Siam, Boris N Oreshkin, and Martin Jagersand. AMP: Adaptive masked proxies for few-shot segmentation. In ICCV, 2019.","title":"AMP: Adaptive masked proxies for few-shot segmentation","authors":"Siam","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurIPS, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tian et al., 2020] Yonglong Tian, Yue Wang, Dilip Krish- nan, Joshua B Tenenbaum, and Phillip Isola. Rethinking few-shot image classification: a good embedding is all you need? In ECCV, 2020.","title":"Rethinking few-shot image classification: a good embedding is all you need?","authors":"Tian","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim- othy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In NeurIPS, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2020] Xin Wang, Thomas E Huang, Trevor Darrell, Joseph E Gonzalez, and Fisher Yu. Frustrat- ingly simple few-shot object detection. arXiv preprint arXiv:2003.06957, 2020.","title":"Frustratingly simple few-shot object detection","authors":"Xin Wang, Thomas E Huang, Trevor Darrell, Joseph E Gonzalez, Fisher Yu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2022] Shuang Wu, Wenjie Pei, Dianwen Mei, Fanglin Chen, Jiandong Tian, and Guangming Lu. Multi- faceted distillation of base-novel commonality for few- shot object detection. In ECCV, 2022.","title":"Multifaceted distillation of base-novel commonality for fewshot object detection","authors":"Shuang Wu, Wenjie Pei, Dianwen Mei, Fanglin Chen, Jiandong Tian, Guangming Lu","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2021a] Chengming Xu, Yanwei Fu, Chen Liu, Chengjie Wang, Jilin Li, Feiyue Huang, Li Zhang, and Xi- angyang Xue. Learning dynamic alignment via meta-filter for few-shot learning. In CVPR, 2021.","title":"Learning dynamic alignment via meta-filter for few-shot learning","authors":"Xu","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2021b] Luo Xu, Wei Longhui, Wen Liangjian, Yang Jinrong, Xie Lingxi, Xu Zenglin, and Tian Qi. Rec- tifying the shortcut learning of background for few-shot learning. NeurIPS, 2021.","title":"Rectifying the shortcut learning of background for few-shot learning","authors":"Xu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2022] Liu Yang, Zhang Weifeng, Xiang Chao, Zheng Tu, Cai Deng, and He Xiaofei. Learning to affiliate: Mutual centralized learning for few-shot classification. In CVPR, 2022.","title":"Learning to affiliate: Mutual centralized learning for few-shot classification","authors":"Liu Yang, Zhang Weifeng, Xiang Chao, Zheng Tu, Cai Deng, He Xiaofei","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2020a] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classifica- tion with differentiable earth mover's distance and struc- tured classifiers. In CVPR, 2020.","title":"Deepemd: Few-shot image classification with differentiable earth mover's distance and structured classifiers","authors":"Zhang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2020b] Xiaolin Zhang, Yunchao Wei, Yi Yang, and Thomas S Huang. SG-one: Similarity guidance net- work for one-shot semantic segmentation. IEEE Transac- tions on Cybernetics, 2020.","title":"SG-one: Similarity guidance network for one-shot semantic segmentation","authors":"Zhang","year":"2020","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhengyu et al., 2021] Chen Zhengyu, Ge Jixie, Zhan Hes- hen, Huang Siteng, and Wang Donglin. Pareto self- supervised training for few-shot learning. In CVPR, 2021.","title":"Pareto selfsupervised training for few-shot learning","authors":"Chen Zhengyu, Ge Jixie, Zhan Heshen, Huang Siteng, Wang Donglin","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}