{"type":"ScholarlyArticle","title":"Open-world Semi-supervised Novel Class Discovery","authors":[{"name":"Jiaming Liu","slug":"jiaming-liu","given":"Jiaming","family":"Liu","affiliation":"University of Electronic Science and Technology of China"},{"name":"Yangqiming Wang","slug":"yangqiming-wang","given":"Yangqiming","family":"Wang","affiliation":"University of Electronic Science and Technology of China"},{"name":"Tongze Zhang","slug":"tongze-zhang","given":"Tongze","family":"Zhang","affiliation":"University of Electronic Science and Technology of China"},{"name":"Yulu Fan","slug":"yulu-fan","given":"Yulu","family":"Fan","affiliation":"University of Electronic Science and Technology of China"},{"name":"Qinli Yang","slug":"qinli-yang","given":"Qinli","family":"Yang","affiliation":"University of Electronic Science and Technology of China"},{"name":"Junming Shao","slug":"junming-shao","given":"Junming","family":"Shao","affiliation":"University of Electronic Science and Technology of China"}],"abstract":"Traditional semi-supervised learning tasks assume that both labeled and unlabeled data follow the same class distribution, but the realistic open-world scenarios are of more complexity with unknown novel classes mixed in the unlabeled set. Therefore, it is of great challenge to not only recognize samples from known classes but also discover the unknown number of novel classes within the unlabeled data. In this paper, we introduce a new open-world semi-supervised novel class discovery approach named OpenNCD, a progressive bi-level contrastive learning method over multiple prototypes. The proposed method is composed of two reciprocally enhanced parts. First, a bi-level contrastive learning method is introduced, which maintains the pair-wise similarity of the prototypes and the prototype group levels for better representation learning. Then, a reliable prototype similarity metric is proposed based on the common representing instances. Prototypes with high similarities will be grouped progressively for known class recognition and novel class discovery. Extensive experiments on three image datasets are conducted and the results show the effectiveness of the proposed method in open-world scenarios, especially with scarce known classes and labels.","doi":"10.24963/ijcai.2023/445","url":"https://ijcai.org/proceedings/2023/445","pdf":"https://ijcai.org/proceedings/2023/0445.pdf","markdown":"https://ijcai.org/proceedings/2023/445.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/445","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":445,"pages":{"first":4002,"last":4010},"keywords":["Machine Learning: ML: Self-supervised Learning","Machine Learning: ML: Semi-supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi- supervised learning. Advances in neural information pro- cessing systems, 32, 2019.","title":"Mixmatch: A holistic approach to semisupervised learning","authors":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin A Raffel","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blondel et al., 2008] Vincent D Blondel, Jean-Loup Guil- laume, Renaud Lambiotte, and Etienne Lefebvre. Fast unfolding of communities in large networks. Jour- nal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008.","title":"Fast unfolding of communities in large networks","authors":"Jean-Loup Vincent D Blondel, Renaud Guillaume, Etienne Lambiotte, Lefebvre","year":"2008","venue":"Journal of statistical mechanics: theory and experiment","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brbić et al., 2020] Maria Brbić, Marinka Zitnik, Sheng Wang, Angela O Pisco, Russ B Altman, Spyros Darma- nis, and Jure Leskovec. Mars: discovering novel cell types across heterogeneous single-cell experiments. Na- ture methods, 17(12):1200-1206, 2020.","title":"Mars: discovering novel cell types across heterogeneous single-cell experiments","authors":"Maria Brbić, Marinka Zitnik, Sheng Wang, Angela O Pisco, Spyros Russ B Altman, Jure Darmanis, Leskovec","year":"2020","venue":"Nature methods","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2022] Kaidi Cao, Maria Brbic, and Jure Leskovec. Open-world semi-supervised learning. In Inter- national Conference on Learning Representations, 2022.","title":"Open-world semi-supervised learning","authors":"Kaidi Cao, Maria Brbic, Jure Leskovec","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020a] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In In- ternational conference on machine learning, pages 1597- 1607. PMLR, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Chen","year":"2020","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2020b] Yanbei Chen, Xiatian Zhu, Wei Li, and Shaogang Gong. Semi-supervised learning under class distribution mismatch. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 34, pages 3569- 3576, 2020.","title":"Semi-supervised learning under class distribution mismatch","authors":"Chen","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Frey and Dueck, 2007] Brendan J Frey and Delbert Dueck. Clustering by passing messages between data points. sci- ence, 315(5814):972-976, 2007.","title":"Clustering by passing messages between data points","authors":"J Brendan, Delbert Frey, Dueck","year":"2007","venue":"science","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2020] Lan-Zhe Guo, Zhen-Yu Zhang, Yuan Jiang, Yu-Feng Li, and Zhi-Hua Zhou. Safe deep semi- supervised learning for unseen-class unlabeled data. In International Conference on Machine Learning, pages 3897-3906. PMLR, 2020.","title":"Safe deep semisupervised learning for unseen-class unlabeled data","authors":"Lan-Zhe Guo, Zhen-Yu Zhang, Yuan Jiang, Yu-Feng Li, Zhi-Hua Zhou","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hamerly and Elkan, 2003] Greg Hamerly and Charles Elkan. Learning the k in k-means. Advances in neural information processing systems, 16, 2003.","title":"Learning the k in k-means","authors":"Elkan Hamerly, Greg Hamerly, Charles Elkan","year":"2003","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Han et al., 2019a] Kai Han, Sylvestre-Alvise Rebuffi, Se- bastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Automatically discovering and learning new visual cate- gories with ranking statistics. In International Conference on Learning Representations, 2019.","title":"Automatically discovering and learning new visual categories with ranking statistics","authors":"Han","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Han et al., 2019b] Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8401-8409, 2019.","title":"Learning to discover novel visual categories via deep transfer clustering","authors":"Han","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2021] Junkai Huang, Chaowei Fang, Weikai Chen, Zhenhua Chai, Xiaolin Wei, Pengxu Wei, Liang Lin, and Guanbin Li. Trash to treasure: Harvesting ood data with cross-modal matching for open-set semi-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8310-8319, 2021.","title":"Trash to treasure: Harvesting ood data with cross-modal matching for open-set semi-supervised learning","authors":"Huang","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kalogeratos and Likas, 2012] Argyris Kalogeratos and Aristidis Likas. Dip-means: an incremental clustering method for estimating the number of clusters. Advances in neural information processing systems, 25, 2012.","title":"Dip-means: an incremental clustering method for estimating the number of clusters","authors":"Likas Kalogeratos, Argyris Kalogeratos, Aristidis Likas","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Master's thesis, Univer- sity of Toronto, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":"Master's thesis","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kuhn, 1955] Kuhn. The hungarian method for the assign- ment problem. Naval research logistics quarterly, 2(1- 2):83-97, 1955.","title":"The hungarian method for the assignment problem","authors":"Kuhn ; Kuhn","year":"1955","venue":"Naval research logistics quarterly","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee and others, 2013] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896, 2013.","title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","authors":"Dong-Hyun Lee","year":"2013","venue":"Workshop on challenges in representation learning, ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Leiber et al., 2021] Collin Leiber, Lena GM Bauer, Ben- jamin Schelling, Christian Böhm, and Claudia Plant. Dip- based deep embedded clustering with k-estimation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 903-913, 2021.","title":"Dipbased deep embedded clustering with k-estimation","authors":"Collin Leiber, G M Lena, Benjamin Bauer, Christian Schelling, Claudia Böhm, Plant","year":"2021","venue":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2021] Junnan Li, Caiming Xiong, and Steven CH Hoi. Comatch: Semi-supervised learning with contrastive graph regularization. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 9475- 9484, 2021.","title":"Comatch: Semi-supervised learning with contrastive graph regularization","authors":"Li","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pelleg et al., 2000] Dan Pelleg, Andrew W Moore, et al. X- means: Extending k-means with efficient estimation of the number of clusters. In Icml, volume 1, pages 727-734, 2000.","title":"Xmeans: Extending k-means with efficient estimation of the number of clusters","authors":"Dan Pelleg, Andrew W Moore","year":"2000","venue":"Icml","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ronen et al., 2022] Meitar Ronen, Shahaf E Finder, and Oren Freifeld. Deepdpm: Deep clustering with an unknown number of clusters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9861-9870, 2022.","title":"Deepdpm: Deep clustering with an unknown number of clusters","authors":"Shahaf E Meitar Ronen, Oren Finder, Freifeld","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sohn et al., 2020] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with con- sistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.","title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vaze et al., 2022] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discov- ery. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.","title":"Generalized category discovery","authors":"Sagar Vaze, Kai Han, Andrea Vedaldi, Andrew Zisserman","year":"2022","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2020] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmen- tation for consistency training. Advances in Neural Infor- mation Processing Systems, 33:6256-6268, 2020.","title":"Unsupervised data augmentation for consistency training","authors":"Xie","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2020] Qing Yu, Daiki Ikami, Go Irie, and Kiy- oharu Aizawa. Multi-task curriculum framework for open- set semi-supervised learning. In European Conference on Computer Vision, pages 438-454. Springer, 2020.","title":"Multi-task curriculum framework for openset semi-supervised learning","authors":"Qing Yu, Daiki Ikami, Go Irie, Kiyoharu Aizawa","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhong et al., 2021] Zhun Zhong, Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang, and Nicu Sebe. Openmix: Re- viving known knowledge for discovering novel visual cat- egories in an open world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9462-9470, 2021.","title":"Openmix: Reviving known knowledge for discovering novel visual categories in an open world","authors":"Zhun Zhong, Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang, Nicu Sebe","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}