{"type":"ScholarlyArticle","title":"Let’s Start Over: Retraining with Selective Samples for Generalized Category Discovery","authors":[{"name":"Zhimao Peng","slug":"zhimao-peng","given":"Zhimao","family":"Peng","affiliation":"VCIP , College of Computer Science , Nankai University , Tianjin , China."},{"name":"Enguang Wang","slug":"enguang-wang","given":"Enguang","family":"Wang","affiliation":"VCIP , College of Computer Science , Nankai University , Tianjin , China."},{"name":"Xialei Liu","slug":"xialei-liu","given":"Xialei","family":"Liu","affiliation":"VCIP , College of Computer Science , Nankai University , Tianjin , China."},{"name":"Ming-Ming Cheng","slug":"ming-ming-cheng","given":"Ming-Ming","family":"Cheng","affiliation":"VCIP , College of Computer Science , Nankai University , Tianjin , China."}],"abstract":"Generalized Category Discovery (GCD) presents a realistic\r\nand challenging problem in open-world learning. Given a par-\r\ntially labeled dataset, GCD aims to categorize unlabeled data\r\nby leveraging visual knowledge from the labeled data, where\r\nthe unlabeled data includes both known and unknown classes.\r\nExisting methods based on parametric/non-parametric classi-\r\nfiers attempt to generate pseudo-labels/relationships for the\r\nunlabeled data to enhance representation learning. However,\r\nthe lack of ground-truth labels for novel classes often leads\r\nto noisy pseudo-labels/relationships, resulting in suboptimal\r\nrepresentation learning. This paper introduces a novel method\r\nusing Nearest Neighbor Distance-aware Label Consistency\r\nsample selection. It creates class-consistent subsets for novel\r\nclass sample clusters from the current GCD method, acting\r\nas “pseudo-labeled sets” to mitigate representation bias. We\r\npropose progressive supervised representation learning with\r\nselected samples to optimize the trade-off between quantity\r\nand purity in each subset. Our method is versatile and appli-\r\ncable to various GCD methods, whether parametric or non-\r\nparametric. We conducted extensive experiments on multiple\r\ngeneric and fine-grained image classification datasets to eval-\r\nuate the effectiveness of our approach. The results demon-\r\nstrate the superiority of our method in achieving improved\r\nperformance in generalized category discovery tasks.","doi":"10.24963/ijcai.2024/532","url":"https://ijcai.org/proceedings/2024/532","pdf":"https://ijcai.org/proceedings/2024/0532.pdf","markdown":"https://ijcai.org/proceedings/2024/532.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/532","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":532,"pages":{"first":4815,"last":4823},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)","Machine Learning: ML: Classification","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning","Machine Learning: ML: Clustering"],"videos":[],"references":[{"position":1,"raw":"[Arazo et al., 2020] Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo- labeling and confirmation bias in deep semi-supervised learning. In 2020 International Joint Conference on Neu- ral Networks (IJCNN), pages 1-8. IEEE, 2020.","title":"Pseudolabeling and confirmation bias in deep semi-supervised learning","authors":"Eric Arazo, Diego Ortego, Paul Albert, E O' Noel, Kevin Connor, Mcguinness","year":"2020","venue":"2020 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arthur and Vassilvitskii, 2007] David Arthur and Sergei Vassilvitskii. K-means++ the advantages of careful seed- ing. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pages 1027-1035, 2007.","title":"K-means++ the advantages of careful seeding","authors":"David Arthur, Sergei Vassilvitskii","year":"2007","venue":"Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2021] Kaidi Cao, Maria Brbic, and Jure Leskovec. Open-world semi-supervised learning. arXiv preprint arXiv:2102.03526, 2021.","title":"Open-world semi-supervised learning","authors":"Kaidi Cao, Maria Brbic, Jure Leskovec","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Caron et al., 2018] Mathilde Caron, Piotr Bojanowski, Ar- mand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European conference on computer vision (ECCV), pages 132-149, 2018.","title":"Deep clustering for unsupervised learning of visual features","authors":"Mathilde Caron, Piotr Bojanowski, Armand Joulin, Matthijs Douze","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Caron et al., 2020] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by con- trasting cluster assignments. Advances in neural informa- tion processing systems, 33:9912-9924, 2020.","title":"Unsupervised learning of visual features by contrasting cluster assignments","authors":"Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, Armand Joulin","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Caron et al., 2021] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vi- sion transformers. In Proceedings of the IEEE/CVF in- ternational conference on computer vision, pages 9650- 9660, 2021.","title":"Emerging properties in self-supervised vision transformers","authors":"Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, Armand Joulin","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fei et al., 2022] Yixin Fei, Zhongkai Zhao, Siwei Yang, and Bingchen Zhao. Xcon: Learning with experts for fine-grained category discovery. arXiv preprint arXiv:2208.01898, 2022.","title":"Xcon: Learning with experts for fine-grained category discovery","authors":"Yixin Fei, Zhongkai Zhao, Siwei Yang, Bingchen Zhao","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fini et al., 2021] Enrico Fini, Enver Sangineto, Stéphane Lathuilière, Zhun Zhong, Moin Nabi, and Elisa Ricci. A unified objective for novel class discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9284-9292, 2021.","title":"A unified objective for novel class discovery","authors":"Enrico Fini, Enver Sangineto, Stéphane Lathuilière, Zhun Zhong, Moin Nabi, Elisa Ricci","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fini et al., 2023] Enrico Fini, Pietro Astolfi, Karteek Ala- hari, Xavier Alameda-Pineda, Julien Mairal, Moin Nabi, and Elisa Ricci. Semi-supervised learning made simple with self-supervised clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3187-3197, 2023.","title":"Semi-supervised learning made simple with self-supervised clustering","authors":"Enrico Fini, Pietro Astolfi, Karteek Alahari, Xavier Alameda-Pineda, Julien Mairal, Moin Nabi, Elisa Ricci","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Han et al., 2019] Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8401-8409, 2019.","title":"Learning to discover novel visual categories via deep transfer clustering","authors":"Han","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Han et al., 2020] Kai Han, Sylvestre-Alvise Rebuffi, Se- bastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Automatically discovering and learning new visual cate- gories with ranking statistics. In International Conference on Learning Representations (ICLR), 2020.","title":"Automatically discovering and learning new visual categories with ranking statistics","authors":"Kai Han, Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, Andrew Zisserman","year":"2020","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hao et al., 2023] Shaozhe Hao, Kai Han, and Kwan-Yee K Wong. Cipr: An efficient framework with cross-instance positive relations for generalized category discovery. arXiv preprint arXiv:2304.06928, 2023.","title":"Cipr: An efficient framework with cross-instance positive relations for generalized category discovery","authors":"Hao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ji et al., 2019] Xu Ji, Joao F Henriques, and Andrea Vedaldi. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 9865-9874, 2019.","title":"Invariant information clustering for unsupervised image classification and segmentation","authors":"Xu Ji, Joao F Henriques, Andrea Vedaldi","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krause et al., 2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the IEEE inter- national conference on computer vision workshops, pages 554-561, 2013.","title":"3d object representations for finegrained categorization","authors":"Jonathan Krause, Michael Stark, Jia Deng, Li Fei-Fei","year":"2013","venue":"Proceedings of the IEEE international conference on computer vision workshops","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural infor- mation processing systems, 25, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Technical report, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kuhn, 1955] Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83-97, 1955.","title":"The hungarian method for the assignment problem","authors":"Harold W Kuhn","year":"1955","venue":"Naval research logistics quarterly","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Laine and Aila, 2016] Samuli Laine and Timo Aila. Tem- poral ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242, 2016.","title":"Temporal ensembling for semi-supervised learning","authors":"Laine, Aila ;, Samuli Laine, Timo Aila","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lee and others, 2013] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, page 896, 2013.","title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","authors":"Dong-Hyun Lee","year":"2013","venue":"Workshop on challenges in representation learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Maji et al., 2013] Subhransu Maji, Esa Rahtu, Juho Kan- nala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013.","title":"Finegrained visual classification of aircraft","authors":"Maji","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ortego et al., 2021] Diego Ortego, Eric Arazo, Paul Al- bert, Noel E O'Connor, and Kevin McGuinness. Multi- objective interpolation training for robustness to label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6606- 6615, 2021.","title":"Multiobjective interpolation training for robustness to label noise","authors":"Ortego","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Parkhi et al., 2012] Omkar M Parkhi, Andrea Vedaldi, An- drew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recogni- tion, pages 3498-3505. IEEE, 2012.","title":"Cats and dogs","authors":"Andrea Omkar M Parkhi, Andrew Vedaldi, Zisserman, Jawahar","year":"2012","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pu et al., 2023] Nan Pu, Zhun Zhong, and Nicu Sebe. Dy- namic conceptional contrastive learning for generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7579-7588, 2023.","title":"Dynamic conceptional contrastive learning for generalized category discovery","authors":"Nan Pu, Zhun Zhong, Nicu Sebe","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rizve et al., 2021] Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. In defense of pseudo-labeling: An uncertainty-aware pseudo-label se- lection framework for semi-supervised learning. arXiv preprint arXiv:2101.06329, 2021.","title":"In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning","authors":"Rizve","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rizve et al., 2022] Mamshad Nayeem Rizve, Navid Kar- dan, and Mubarak Shah. Towards realistic semi- supervised learning. In European Conference on Com- puter Vision, pages 437-455. Springer, 2022.","title":"Towards realistic semisupervised learning","authors":"Rizve","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Scheirer et al., 2012] Walter J Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E Boult. Toward open set recognition. IEEE transac- tions on pattern analysis and machine intelligence, 35(7):1757-1772, 2012.","title":"Toward open set recognition","authors":"J Walter, Anderson Scheirer, De Rezende, Archana Rocha, Terrance E Sapkota, Boult","year":"2012","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. Advances in neural information pro- cessing systems, 30, 2017.","title":"Mean teachers are better role models: Weightaveraged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vaze et al., 2022a] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7492-7501, 2022.","title":"Generalized category discovery","authors":"Vaze","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Vaze et al., 2022b] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: a good closed-set classifier is all you need? In International Con- ference on Learning Representations, 2022.","title":"Open-set recognition: a good closed-set classifier is all you need?","authors":"Vaze","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wen et al., 2023] Xin Wen, Bingchen Zhao, and Xiaojuan Qi. Parametric classification for generalized category dis- covery: A baseline study. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 16590-16600, 2023.","title":"Parametric classification for generalized category discovery: A baseline study","authors":"Xin Wen, Bingchen Zhao, Xiaojuan Qi","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xie et al., 2020] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmen- tation for consistency training. Advances in neural infor- mation processing systems, 33:6256-6268, 2020.","title":"Unsupervised data augmentation for consistency training","authors":"Xie","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[YM. et al., 2020] Asano YM., Rupprecht C., and Vedaldi A. Self-labelling via simultaneous clustering and represen- tation learning. In International Conference on Learning Representations (ICLR), 2020.","title":"Self-labelling via simultaneous clustering and representation learning","authors":"Y M Asano, C Rupprecht, A Vedaldi","year":"2020","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2023] Sheng Zhang, Salman Khan, Zhiqiang Shen, Muzammal Naseer, Guangyi Chen, and Fahad Shah- baz Khan. Promptcal: Contrastive affinity learning via auxiliary prompts for generalized novel category discov- ery. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 3479-3488, 2023.","title":"Promptcal: Contrastive affinity learning via auxiliary prompts for generalized novel category discovery","authors":"Sheng Zhang, Salman Khan, Zhiqiang Shen, Muzammal Naseer, Guangyi Chen, Fahad Shahbaz Khan","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhao et al., 2023] Bingchen Zhao, Xin Wen, and Kai Han. Learning semi-supervised gaussian mixture mod- els for generalized category discovery. arXiv preprint arXiv:2305.06144, 2023.","title":"Learning semi-supervised gaussian mixture models for generalized category discovery","authors":"Zhao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhong et al., 2021] Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, and Nicu Sebe. Neighbor- hood contrastive learning for novel class discovery. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10867-10875, 2021.","title":"Neighborhood contrastive learning for novel class discovery","authors":"Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, Nicu Sebe","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}