{"type":"ScholarlyArticle","title":"Revealing the Two Sides of Data Augmentation: An Asymmetric Distillation-based Win-Win Solution for Open-Set Recognition","authors":[{"name":"Yunbing Jia","slug":"yunbing-jia","given":"Yunbing","family":"Jia","affiliation":"School of Artificial Intelligence , Jilin University , China; Institute of Computing Technology , Chinese Academy of Sciences , China"},{"name":"Xiaoyu Kong","slug":"xiaoyu-kong","given":"Xiaoyu","family":"Kong","affiliation":"Harbin Institute of Technology (Shenzhen) , China"},{"name":"Fan Tang","slug":"fan-tang","given":"Fan","family":"Tang","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , China"},{"name":"Yixing Gao","slug":"yixing-gao","given":"Yixing","family":"Gao","affiliation":"School of Artificial Intelligence , Jilin University , China; Engineering Research Center of Knowledge-Driven Human-Machine Intelligence , Ministry of Education , China"},{"name":"Weiming Dong","slug":"weiming-dong","given":"Weiming","family":"Dong","affiliation":"MAIS , Institute of Automation , Chinese Academy of Sciences , China"},{"name":"Yi Yang","slug":"yi-yang","given":"Yi","family":"Yang","affiliation":"Didi Chuxing , China"}],"abstract":"In this paper, we reveal the two sides of data augmentation: enhancements in closed-set recognition correlate with a significant decrease in open-set recognition. Through empirical investigation, we find that multi-sample-based augmentations would contribute to reducing feature discrimination, thereby diminishing the open-set criteria. Although knowledge distillation could impair the feature via imitation, the mixed feature with ambiguous semantics hinders the distillation. To this end, we propose an asymmetric distillation framework by feeding teacher model extra raw data to enlarge the benefit of teacher. Moreover, a joint mutual information loss and a selective relabel strategy are utilized to alleviate the influence of hard mixed samples. Our method successfully mitigates the decline in open-set and outperforms SOTAs by 2%~3% AUROC on the Tiny-ImageNet dataset and experiments on large-scale dataset ImageNet-21K demonstrate the generalization of our method.","doi":"10.24963/ijcai.2024/101","url":"https://ijcai.org/proceedings/2024/101","pdf":"https://ijcai.org/proceedings/2024/0101.pdf","markdown":"https://ijcai.org/proceedings/2024/101.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/101","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":101,"pages":{"first":911,"last":919},"keywords":["Computer Vision: CV: Representation learning","Computer Vision: CV: Structural and model-based approaches, knowledge representation and reasoning"],"videos":[],"references":[{"position":1,"raw":"[Balestriero et al., 2022] Randall Balestriero, Leon Bottou, and Yann LeCun. The effects of regularization and data augmentation are class dependent. Advances in Neural Information Processing Systems, 35:37878-37891, 2022.","title":"The effects of regularization and data augmentation are class dependent","authors":"Randall Balestriero, Leon Bottou, Yann Lecun","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2021a] Dong Chen, Fan Tang, Weiming Dong, Hanxing Yao, and Changsheng Xu. Siamcpn: Visual tracking with the siamese center-prediction network. Com- putational Visual Media, 7:253-265, 2021.","title":"Siamcpn: Visual tracking with the siamese center-prediction network","authors":"Chen","year":"2021","venue":"Computational Visual Media","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021b] Guangyao Chen, Peixi Peng, Xiangqian Wang, and Yonghong Tian. Adversarial reciprocal points learning for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8065- 8081, 2021.","title":"Adversarial reciprocal points learning for open set recognition","authors":"Chen","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2023] Dong Chen, Xingjia Pan, Fan Tang, Weiming Dong, and Changsheng Xu. Spa 2 net: Structure- preserved attention activated network for weakly super- vised object localization. IEEE Transactions on Image Processing, 2023.","title":"Spa 2 net: Structurepreserved attention activated network for weakly supervised object localization","authors":"Dong Chen, Xingjia Pan, Fan Tang, Weiming Dong, Changsheng Xu","year":"2023","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choi et al., 2023] Hongjun Choi, Eun Som Jeon, Ankita Shukla, and Pavan Turaga. Understanding the role of mixup in knowledge distillation: An empirical study. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2319-2328, 2023.","title":"Understanding the role of mixup in knowledge distillation: An empirical study","authors":"Hongjun Choi, Eun Som Jeon, Ankita Shukla, Pavan Turaga","year":"2023","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cubuk et al., 2018] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018.","title":"Autoaugment: Learning augmentation policies from data","authors":"Barret Ekin D Cubuk, Dandelion Zoph, Vijay Mane, Quoc V Vasudevan, Le","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Davis and Goadrich, 2006] Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pages 233-240, 2006.","title":"The relationship between precision-recall and roc curves","authors":"Goadrich Davis, Jesse Davis, Mark Goadrich","year":"2006","venue":"Proceedings of the 23rd international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[DeVries and Taylor, 2017] Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017.","title":"Improved regularization of convolutional neural networks with cutout","authors":"Terrance Devries, Graham W Taylor","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Feng et al., 2023] Runyang Feng, Yixing Gao, Xueqing Ma, Tze Ho Elden Tse, and Hyung Jin Chang. Mutual information-based temporal difference learning for human pose estimation in video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17131-17141, 2023.","title":"Mutual information-based temporal difference learning for human pose estimation in video","authors":"Feng","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hendrycks et al., 2020] Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. AugMix: A simple data processing method to improve robustness and uncertainty. Proceedings of the International Conference on Learning Representations (ICLR), 2020.","title":"AugMix: A simple data processing method to improve robustness and uncertainty","authors":"Hendrycks","year":"2020","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network, 2015. Accessed: 2018-12-06.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hou et al., 2021] Xuege Hou, Yali Li, and Shengjin Wang. Disentangled representation for age-invariant face recog- nition: A mutual information minimization perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3692-3701, 2021.","title":"Disentangled representation for age-invariant face recognition: A mutual information minimization perspective","authors":"Hou","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hou et al., 2024] Liang Hou, Qi Cao, Yige Yuan, Song- tao Zhao, Chongyang Ma, Siyuan Pan, Pengfei Wan, Zhongyuan Wang, Huawei Shen, and Xueqi Cheng. Augmentation-aware self-supervision for data-efficient gan training. Advances in Neural Information Processing Systems, 36, 2024.","title":"Augmentation-aware self-supervision for data-efficient gan training","authors":"Liang Hou, Qi Cao, Yige Yuan, Songtao Zhao, Chongyang Ma, Siyuan Pan, Pengfei Wan, Zhongyuan Wang, Huawei Shen, Xueqi Cheng","year":"2024","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2022a] Hongzhi Huang, Yu Wang, Qinghua Hu, and Ming-Ming Cheng. Class-specific semantic reconstruction for open set recognition. IEEE transactions on pattern analysis and machine intelligence, 45(4):4214- 4228, 2022.","title":"Class-specific semantic reconstruction for open set recognition","authors":"Huang","year":"2022","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang et al., 2022b] Tao Huang, Shan You, Fei Wang, Chen Qian, and Chang Xu. Knowledge distillation from a stronger teacher. arXiv preprint arXiv:2205.10536, 2022.","title":"Knowledge distillation from a stronger teacher","authors":"Huang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Krause et al., 2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops, June 2013.","title":"3d object representations for fine-grained categorization","authors":"Jonathan Krause, Michael Stark, Jia Deng, Li Fei-Fei","year":"2013","venue":"Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Le and Yang, 2015] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.","title":"Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge","authors":"Yang Le","year":"2015","venue":"CS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Maji et al., 2013] Subhransu Maji, Esa Rahtu, Juho Kan- nala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013.","title":"Finegrained visual classification of aircraft","authors":"Maji","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Read- ing digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":"NIPS Workshop on Deep Learning and Unsupervised Feature Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark","year":"2021","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ren et al., 2023] Hairui Ren, Fan Tang, Xingjia Pan, Juan Cao, Weiming Dong, Zhiwen Lin, Ke Yan, and Chang- sheng Xu. A 2 pt: Anti-associative prompt tuning for open set visual recognition. IEEE Transactions on Multimedia, 2023.","title":"A 2 pt: Anti-associative prompt tuning for open set visual recognition","authors":"Fan Hairui Ren, Xingjia Tang, Juan Pan, Weiming Cao, Zhiwen Dong, Ke Lin, Changsheng Yan, Xu","year":"2023","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Roady et al., 2020] Ryne Roady, Tyler L Hayes, and Christopher Kanan. Improved robustness to open set inputs via tempered mixup. In Computer Vision-ECCV 2020 Workshops: Glasgow, UK, August 23-28, 2020, Proceedings, Part I 16, pages 186-201. Springer, 2020.","title":"Improved robustness to open set inputs via tempered mixup","authors":"Ryne Roady, Tyler L Hayes, Christopher Kanan","year":"2020","venue":"Computer Vision-ECCV 2020 Workshops","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sandler et al., 2018] Mark Sandler, Andrew Howard, Men- glong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510-4520, 2018.","title":"Mobilenetv2: Inverted residuals and linear bottlenecks","authors":"Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vaze et al., 2022] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: a good closed-set classifier is all you need? In International Conference on Learning Representations, 2022.","title":"Open-set recognition: a good closed-set classifier is all you need?","authors":"Sagar Vaze, Kai Han, Andrea Vedaldi, Andrew Zisserman","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2022] Huan Wang, Suhas Lohit, Michael Jones, and Yun Fu. What makes a \"good\" data augmenta- tion in knowledge distillation -a statistical perspective. In NeurIPS, 2022.","title":"What makes a \"good\" data augmentation in knowledge distillation -a statistical perspective","authors":"Huan Wang, Suhas Lohit, Michael Jones, Yun Fu","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2024] Yue Wang, Yuke Li, James H Elder, Runmin Wu, and Huchuan Lu. Class-conditional domain adaptation for semantic segmentation. Computational Visual Media, pages 1-18, 2024.","title":"Class-conditional domain adaptation for semantic segmentation","authors":"Yue Wang, Yuke Li, James H Elder, Runmin Wu, Huchuan Lu","year":"2024","venue":"Computational Visual Media","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2023] Huimin Wu, Chenyang Lei, Xiao Sun, Peng-Shuai Wang, Qifeng Chen, Kwang-Ting Cheng, Stephen Lin, and Zhirong Wu. Randomized quantization: A generic augmentation for data agnostic self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16305-16316, 2023.","title":"Randomized quantization: A generic augmentation for data agnostic self-supervised learning","authors":"Huimin Wu, Chenyang Lei, Xiao Sun, Peng-Shuai Wang, Qifeng Chen, Kwang-Ting Cheng, Stephen Lin, Zhirong Wu","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xu et al., 2022] Yifan Xu, Huapeng Wei, Minxuan Lin, Yingying Deng, Kekai Sheng, Mengdan Zhang, Fan Tang, Weiming Dong, Feiyue Huang, and Changsheng Xu. Transformers in computational visual media: A survey. Computational Visual Media, 8:33-62, 2022.","title":"Transformers in computational visual media: A survey","authors":"Yifan Xu, Huapeng Wei, Minxuan Lin, Yingying Deng, Kekai Sheng, Mengdan Zhang, Fan Tang, Weiming Dong, Feiyue Huang, Changsheng Xu","year":"2022","venue":"Computational Visual Media","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xu et al., 2023] Baile Xu, Furao Shen, and Jian Zhao. Con- trastive open set recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 10546-10556, 2023.","title":"Contrastive open set recognition","authors":"Baile Xu, Furao Shen, Jian Zhao","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2022] Suorong Yang, Weikang Xiao, Mengcheng Zhang, Suhan Guo, Jian Zhao, and Furao Shen. Image data augmentation for deep learning: A survey. arXiv preprint arXiv:2204.08610, 2022.","title":"Image data augmentation for deep learning: A survey","authors":"Suorong Yang, Weikang Xiao, Mengcheng Zhang, Suhan Guo, Jian Zhao, Furao Shen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang et al., 2023] Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.","title":"Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification","authors":"Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, Bingbing Ni","year":"2023","venue":"Scientific Data","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yun et al., 2019] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023- 6032, 2019.","title":"Cutmix: Regularization strategy to train strong classifiers with localizable features","authors":"Yun","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2017] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.","title":"mixup: Beyond empirical risk minimization","authors":"Hongyi Zhang, Moustapha Cisse, David Yann N Dauphin, Lopez-Paz","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhou et al., 2022] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337-2348, 2022.","title":"Learning to prompt for vision-language models","authors":"Zhou","year":"2022","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}