{"type":"ScholarlyArticle","title":"Boosting Few-Shot Open-Set Recognition with Multi-Relation Margin Loss","authors":[{"name":"Yongjuan Che","slug":"yongjuan-che","given":"Yongjuan","family":"Che","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"},{"name":"Yuexuan An","slug":"yuexuan-an","given":"Yuexuan","family":"An","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"},{"name":"Hui Xue","slug":"hui-xue","given":"Hui","family":"Xue","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"}],"abstract":"Few-shot open-set recognition (FSOSR) has become a great challenge, which requires classifying known classes and rejecting the unknown ones with only limited samples. Existing FSOSR methods mainly construct an ambiguous distribution of known classes from scarce known samples without considering the latent distribution information of unknowns, which degrades the performance of open-set recognition. To address this issue, we propose a novel loss function called multi-relation margin (MRM) loss that can plug in few-shot methods to boost the performance of FSOSR. MRM enlarges the margin between different classes by extracting the multi-relationship of paired samples to dynamically refine the decision boundary for known classes and implicitly delineate the distribution of unknowns. Specifically, MRM separates the classes by enforcing a margin while concentrating samples of the same class on a hypersphere with a learnable radius. In order to better capture the distribution information of each class, MRM extracts the similarity and correlations among paired samples, ameliorating the optimization of the margin and radius. Experiments on public benchmarks reveal that methods with MRM loss can improve the unknown detection of AUROC by a significant margin while correctly classifying the known classes.","doi":"10.24963/ijcai.2023/390","url":"https://ijcai.org/proceedings/2023/390","pdf":"https://ijcai.org/proceedings/2023/0390.pdf","markdown":"https://ijcai.org/proceedings/2023/390.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/390","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":390,"pages":{"first":3505,"last":3513},"keywords":["Machine Learning: ML: Meta-learning","Machine Learning: ML: Few-shot learning"],"videos":[],"references":[{"position":1,"raw":"[An et al., 2021] Yuexuan An, Hui Xue, Xingyu Zhao, and Lu Zhang. Conditional Self-Supervised Learning for Few- Shot Classification. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2021.","title":"Conditional Self-Supervised Learning for Few-Shot Classification","authors":"An","year":"2021","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2021/295"},{"position":2,"raw":"[An et al., 2023] Yuexuan An, Hui Xue, Xingyu Zhao, and Jing Wang. From Instance to Metric Calibration: A Unified Framework for Open-World Few-Shot Learning. IEEE Transactions on Pattern Analysis and Machine In- telligence, 2023.","title":"From Instance to Metric Calibration: A Unified Framework for Open-World Few-Shot Learning","authors":"An","year":"2023","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bendale and Boult, 2016] Abhijit Bendale and Terrance E. Boult. Towards Open Set Deep Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2016.","title":"Towards Open Set Deep Networks","authors":"Abhijit Bendale, Terrance E Boult","year":"2016","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2019] Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A Closer Look at Few-shot Classification. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.","title":"A Closer Look at Few-shot Classification","authors":"Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank, Wang, Jia-Bin Huang","year":"2019","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Finn et al., 2017] Chelsea Finn, P. Abbeel, and S. Levine. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In Proceedings of International Confer- ence on Machine Learning (ICML), 2017.","title":"Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks","authors":"Chelsea Finn, P Abbeel, S Levine","year":"2017","venue":"Proceedings of International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Geng et al., 2020] Chuanxing Geng, Sheng-jun Huang, and Songcan Chen. Recent Advances in Open Set Recogni- tion: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.","title":"Recent Advances in Open Set Recognition: A Survey","authors":"Geng","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.","title":"Deep Residual Learning for Image Recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hsu et al., 2020] Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized ODIN: Detecting Out- of-Distribution Image Without Learning From Out-of- Distribution Data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.","title":"Generalized ODIN: Detecting Outof-Distribution Image Without Learning From Out-of-Distribution Data","authors":"Yen-Chang Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2022] Shiyuan Huang, Jiawei Ma, Guangxing Han, and Shih-Fu Chang. Task-Adaptive Negative Envi- sion for Few-Shot Open-Set Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2022.","title":"Task-Adaptive Negative Envision for Few-Shot Open-Set Recognition","authors":"Shiyuan Huang, Jiawei Ma, Guangxing Han, Shih-Fu Chang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jeong et al., 2021] Minki Jeong, Seokeon Choi, and Chang- ick Kim. Few-shot Open-set Recognition by Transfor- mation Consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.","title":"Few-shot Open-set Recognition by Transformation Consistency","authors":"Minki Jeong, Seokeon Choi, Changick Kim","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2020] Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and Liwei Wang. Boosting Few-Shot Learning With Adaptive Margin Loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2020.","title":"Boosting Few-Shot Learning With Adaptive Margin Loss","authors":"Li","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2020a] Bin Liu, Yue Cao, Yutong Lin, Qi Li, Zheng Zhang, Mingsheng Long, and Han Hu. Negative Margin Matters: Understanding Margin in Few-Shot Clas- sification. In Proceedings of the IEEE/CVF European Conference on Computer Vision (ECCV), 2020.","title":"Negative Margin Matters: Understanding Margin in Few-Shot Classification","authors":"Liu","year":"2020","venue":"Proceedings of the IEEE/CVF European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2020b] Bo Liu, Hao Kang, Haoxiang Li, Gang Hua, and Nuno Vasconcelos. Few-Shot Open-Set Recog- nition Using Meta-Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.","title":"Few-Shot Open-Set Recognition Using Meta-Learning","authors":"Liu","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2022a] Rongkai Ma, Pengfei Fang, Gil Avraham, Yan Zuo, Tianyu Zhu, Tom Drummond, and Mehrtash Ha- randi. Learning instance and task-aware dynamic kernels for few-shot learning. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.","title":"Learning instance and task-aware dynamic kernels for few-shot learning","authors":"Ma","year":"2022","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2022b] Rongkai Ma, Pengfei Fang, Tom Drum- mond, and Mehrtash Harandi. Adaptive poincaré point to set distance for few-shot classification. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2022.","title":"Adaptive poincaré point to set distance for few-shot classification","authors":"Ma","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mendes Júnior et al., 2017] Pedro R. Mendes Júnior, Roberto M. de Souza, Rafael de O. Werneck, Bernardo V. Stein, Daniel V. Pazinato, Waldir R. de Almeida, Otávio A. B. Penatti, Ricardo da S. Torres, and Anderson Rocha. Nearest neighbors distance ratio open-set classifier. Machine Learning, 2017.","title":"Nearest neighbors distance ratio open-set classifier","authors":"Pedro R Mendes Júnior, Roberto M De Souza, Rafael De, O Werneck, Bernardo V Stein, Daniel V Pazinato, R Waldir, De Almeida, A B Otávio, Ricardo Penatti, S Da, Anderson Torres, Rocha","year":"2017","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Neal et al., 2018] Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li. Open Set Learning with Counterfactual Images. In Proceedings of the Euro- pean Conference on Computer Vision (ECCV), 2018.","title":"Open Set Learning with Counterfactual Images","authors":"Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, Fuxin Li","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Oza and Patel, 2019] Poojan Oza and Vishal M. Patel. C2AE: Class Conditioned Auto-Encoder for Open-Set Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.","title":"C2AE: Class Conditioned Auto-Encoder for Open-Set Recognition","authors":"Poojan Oza, M Vishal, Patel","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pal et al., 2022] Debabrata Pal, Valay Bundele, Renuka Sharma, Biplab Banerjee, and Yogananda Jeppu. Few- Shot Open-Set Recognition of Hyperspectral Images With Outlier Calibration Network. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision (WACV), 2022.","title":"Few-Shot Open-Set Recognition of Hyperspectral Images With Outlier Calibration Network","authors":"Debabrata Pal, Valay Bundele, Renuka Sharma, Biplab Banerjee, Yogananda Jeppu","year":"2022","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pal et al., 2023] Debabrata Pal, Shirsha Bose, Biplab Banerjee, and Yogananda Jeppu. MORGAN: Meta- Learning-Based Few-Shot Open-Set Recognition via Gen- erative Adversarial Network. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision (WACV), 2023.","title":"MORGAN: Meta-Learning-Based Few-Shot Open-Set Recognition via Generative Adversarial Network","authors":"Debabrata Pal, Shirsha Bose, Biplab Banerjee, Yogananda Jeppu","year":"2023","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Perera et al., 2020] Pramuditha Perera, Vlad I. Morariu, Ra- jiv Jain, Varun Manjunatha, Curtis Wigington, Vicente Or- donez, and Vishal M. Patel. Generative-Discriminative Feature Representations for Open-Set Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.","title":"Generative-Discriminative Feature Representations for Open-Set Recognition","authors":"Perera","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ren et al., 2018] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel. Meta-Learning for Semi-Supervised Few-Shot Classification. In Proceed- ings of the International Conference on Learning Repre- sentations (ICLR), 2018.","title":"Meta-Learning for Semi-Supervised Few-Shot Classification","authors":"Ren","year":"2018","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ruff et al., 2018] Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexan- der Binder, Emmanuel Müller, and Marius Kloft. Deep One-Class Classification. In Proceedings of the Inter- national Conference on Machine Learning (ICML), July 2018.","title":"Deep One-Class Classification","authors":"Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Ahmed Shoaib, Alexander Siddiqui, Emmanuel Binder, Marius Müller, Kloft","year":"2018","venue":"Proceedings of the International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. FaceNet: A Unified Embedding for Face Recognition and Clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2015.","title":"FaceNet: A Unified Embedding for Face Recognition and Clustering","authors":"Schroff","year":"2015","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical Networks for Few- shot Learning. In Advancesof Neural Information Processing Systems (NeurIPS), 2017.","title":"Prototypical Networks for Fewshot Learning","authors":"Jake Snell, Kevin Swersky, Richard S Zemel","year":"2017","venue":"Advancesof Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sohn, 2016] Kihyuk Sohn. Improved Deep Metric Learning with Multi-class N-pair Loss Objective. In Advances in Neural Information Processing Systems (NeurIPS), 2016.","title":"Improved Deep Metric Learning with Multi-class N-pair Loss Objective","authors":"Kihyuk Sohn, Sohn","year":"2016","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Song et al., 2016] Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep Metric Learning via Lifted Structured Feature Embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2016.","title":"Deep Metric Learning via Lifted Structured Feature Embedding","authors":"Hyun Oh Song, Yu Xiang, Stefanie Jegelka, Silvio Savarese","year":"2016","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Song et al., 2022] Nan Song, Chi Zhang, and Guosheng Lin. Few-shot Open-set Recognition Using Background as Unknowns. In Proceedings of the ACM International Conference on Multimedia (ACM MM), 2022.","title":"Few-shot Open-set Recognition Using Background as Unknowns","authors":"Nan Song, Chi Zhang, Guosheng Lin","year":"2022","venue":"Proceedings of the ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sun et al., 2020] Xin Sun, Zhenning Yang, Chi Zhang, Keck-Voon Ling, and Guohao Peng. Conditional Gaus- sian Distribution Learning for Open Set Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.","title":"Conditional Gaussian Distribution Learning for Open Set Recognition","authors":"Xin Sun, Zhenning Yang, Chi Zhang, Keck-Voon, Guohao Ling, Peng","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[van der Maaten and Hinton, 2008] Laurens van der Maaten and Geoffrey Hinton. Visualizing Data using t-SNE. Jour- nal of Machine Learning Research, 2008.","title":"Visualizing Data using t-SNE","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Vinyals et al., 2016] Oriol Vinyals, C. Blundell, T. Lilli- crap, K. Kavukcuoglu, and Daan Wierstra. Matching Net- works for One Shot Learning. In Advances of Neural In- formation Processing Systems (NeurIPS), 2016.","title":"Matching Networks for One Shot Learning","authors":"Oriol, C Vinyals, T Blundell, K Lillicrap, Daan Kavukcuoglu, Wierstra","year":"2016","venue":"Advances of Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. Technical Report CNS-TR- 2010-001, California Institute of Technology, 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2019a] Xinshao Wang, Yang Hua, Elyor Kodirov, Guosheng Hu, Romain Garnier, and Neil M. Robertson. Ranked List Loss for Deep Metric Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.","title":"Ranked List Loss for Deep Metric Learning","authors":"Wang","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2019b] Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, and Matthew R. Scott. Multi- Similarity Loss With General Pair Weighting for Deep Metric Learning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), 2019.","title":"Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning","authors":"Wang","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wen et al., 2016] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A Discriminative Feature Learning Ap- proach for Deep Face Recognition. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Proceedings of the European Conference on Computer Vision (ECCV), 2016.","title":"A Discriminative Feature Learning Approach for Deep Face Recognition","authors":"Yandong Wen, Kaipeng Zhang, Zhifeng Li, Yu Qiao","year":"2016","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ye et al., 2020] Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-Shot Learning via Embedding Adapta- tion With Set-to-Set Functions. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.","title":"Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions","authors":"Han-Jia Ye, Hexiang Hu, Fei De-Chuan Zhan, Sha","year":"2020","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}