{"type":"ScholarlyArticle","title":"Bridging the Gap: Learning Pace Synchronization for Open-World Semi-Supervised Learning","authors":[{"name":"Bo Ye","slug":"bo-ye","given":"Bo","family":"Ye","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Lab. of Computer Network and Information Integration ( Southeast University) , MoE , China"},{"name":"Kai Gan","slug":"kai-gan","given":"Kai","family":"Gan","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Lab. of Computer Network and Information Integration ( Southeast University) , MoE , China"},{"name":"Tong Wei","slug":"tong-wei","given":"Tong","family":"Wei","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Lab. of Computer Network and Information Integration ( Southeast University) , MoE , China"},{"name":"Min-Ling Zhang","slug":"min-ling-zhang","given":"Min-Ling","family":"Zhang","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Lab. of Computer Network and Information Integration ( Southeast University) , MoE , China"}],"abstract":"In open-world semi-supervised learning, a machine learning model is tasked with uncovering novel categories from unlabeled data while maintaining performance on seen categories from labeled data. The central challenge is the substantial learning gap between seen and novel categories, as the model learns the former faster due to accurate supervisory information. Moreover, capturing the semantics of unlabeled novel category samples is also challenging due to the missing label information. To address the above issues, we introduce 1) the adaptive synchronizing marginal loss which imposes class-specific negative margins to alleviate the model bias towards seen classes, and 2) the pseudo-label contrastive clustering which exploits pseudo-labels predicted by the model to group unlabeled data from the same category together in the output space. Extensive experiments on benchmark datasets demonstrate that previous approaches may significantly hinder novel class learning, whereas our method strikingly balances the learning pace between seen and novel classes, achieving a remarkable 3% average accuracy increase on the ImageNet dataset. Importantly, we find that fine-tuning the self-supervised pre-trained model significantly boosts the performance, which is overlooked in prior literature. Our code is available at https://github.com/yebo0216best/LPS-main.","doi":"10.24963/ijcai.2024/593","url":"https://ijcai.org/proceedings/2024/593","pdf":"https://ijcai.org/proceedings/2024/0593.pdf","markdown":"https://ijcai.org/proceedings/2024/593.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/593","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":593,"pages":{"first":5362,"last":5370},"keywords":["Machine Learning: ML: Weakly supervised learning","Machine Learning: ML: Semi-supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Arazo et al., 2020] Eric Arazo, Diego Ortego, Paul Albert, Noel E. O'Connor, and Kevin McGuinness. Pseudo- labeling and confirmation bias in deep semi-supervised learning. In International Joint Conference on Neural Net- works, 2020.","title":"Pseudolabeling and confirmation bias in deep semi-supervised learning","authors":"Eric Arazo, Diego Ortego, Paul Albert, Noel E O'connor, Kevin Mcguinness","year":"2020","venue":"International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bendale and Boult, 2015] Abhijit Bendale and Terrance E Boult. Towards open world recognition. In IEEE Confer- ence on Computer Vision and Pattern Recognition, 2015.","title":"Towards open world recognition","authors":"Abhijit Bendale, Terrance E Boult","year":"2015","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. MixMatch: A holistic approach to semi- supervised learning. Neural Information Processing Sys- tems, 2019.","title":"MixMatch: A holistic approach to semisupervised learning","authors":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin A Raffel","year":"2019","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boult et al., 2019] Terrance E Boult, Steve Cruz, Ak- shay Raj Dhamija, M Gunther, James Henrydoss, and Wal- ter J Scheirer. Learning and the unknown: Surveying steps toward open world recognition. In AAAI Conference on Artificial Intelligence, 2019.","title":"Learning and the unknown: Surveying steps toward open world recognition","authors":"Terrance E Boult, Steve Cruz, Raj Akshay, M Dhamija, James Gunther, Walter J Henrydoss, Scheirer","year":"2019","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2020] Kaidi Cao, Colin Wei, Adrien Gaidon, Thang Luong, and Quoc Le. Learning imbalanced datasets with label-distribution-aware margin loss. In Neural Infor- mation Processing Systems, 2020.","title":"Learning imbalanced datasets with label-distribution-aware margin loss","authors":"Kaidi Cao, Colin Wei, Adrien Gaidon, Thang Luong, Quoc Le","year":"2020","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cao et al., 2022] Kaidi Cao, Maria Brbic, and Jure Leskovec. Open-world semi-supervised learning. In Inter- national Conference on Learning Representations, 2022.","title":"Open-world semi-supervised learning","authors":"Kaidi Cao, Maria Brbic, Jure Leskovec","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2020a] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Inter- national Conference on Machine Learning, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Chen","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2020b] Yanbei Chen, Xiatian Zhu, Wei Li, and Shaogang Gong. Semi-supervised learning under class distribution mismatch. In AAAI Conference on Artificial Intelligence, 2020.","title":"Semi-supervised learning under class distribution mismatch","authors":"Chen","year":"2020","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cubuk et al., 2019] Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. Randaugment: Practical au- tomated data augmentation with a reduced search space. arXiv preprint arXiv:1909.13719, 2019.","title":"Randaugment: Practical automated data augmentation with a reduced search space","authors":"D Ekin, Barret Cubuk, Jonathon Zoph, Shlens, V Quoc, Le","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2020] Lan-Zhe Guo, Zhen-Yu Zhang, Yuan Jiang, Yu-Feng Li, and Zhi-Hua Zhou. Safe deep semi- supervised learning for unseen-class unlabeled data. In International Conference on Machine Learning, 2020.","title":"Safe deep semisupervised learning for unseen-class unlabeled data","authors":"Lan-Zhe Guo, Zhen-Yu Zhang, Yuan Jiang, Yu-Feng Li, Zhi-Hua Zhou","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2022] Lan-Zhe Guo, Yi-Ge Zhang, Zhi-Fan Wu, Jie-Jing Shao, and Yu-Feng Li. Robust semi-supervised learning when not all classes have labels. In Neural Infor- mation Processing Systems, 2022.","title":"Robust semi-supervised learning when not all classes have labels","authors":"Lan-Zhe Guo, Yi-Ge Zhang, Zhi-Fan Wu, Jie-Jing Shao, Yu-Feng Li","year":"2022","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ha and Blanz, 2021] Mai Lan Ha and Volker Blanz. Deep ranking with adaptive margin triplet loss. arXiv preprint arXiv:2107.06187, 2021.","title":"Deep ranking with adaptive margin triplet loss","authors":"Blanz ; Ha, Lan Mai, Volker Ha, Blanz","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Han et al., 2019] Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustering. In IEEE International Con- ference on Computer Vision, 2019.","title":"Learning to discover novel visual categories via deep transfer clustering","authors":"Han","year":"2019","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Han et al., 2020] Kai Han, Sylvestre-Alvise Rebuffi, Se- bastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Automatically discovering and learning new visual cate- gories with ranking statistics. In International Conference on Learning Representations, 2020.","title":"Automatically discovering and learning new visual categories with ranking statistics","authors":"Kai Han, Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, Andrew Zisserman","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2023] Bin-Bin Jia, Jun-Ying Liu, Jun-Yi Hang, and Min-Ling Zhang. Learning label-specific features for decomposition-based multi-class classification. Frontiers of Computer Science, 17(6):176348, 2023.","title":"Learning label-specific features for decomposition-based multi-class classification","authors":"Jia","year":"2023","venue":"Frontiers of Computer Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Master's thesis, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kuhn, 1955] Harold W Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics Quar- terly, 1955.","title":"The Hungarian method for the assignment problem","authors":"Harold W Kuhn","year":"1955","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Laine and Aila, 2016] Samuli Laine and Timo Aila. Tem- poral Ensembling for Semi-Supervised Learning. arXiv: Neural and Evolutionary Computing, 2016.","title":"Temporal Ensembling for Semi-Supervised Learning","authors":"Laine, Aila ;, Samuli Laine, Timo Aila","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lee, 2013] Dong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, 2013.","title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","authors":"Dong-Hyun Lee","year":"2013","venue":"Workshop on challenges in representation learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2020] Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and Liwei Wang. Boosting few-shot learning with adaptive margin loss. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.","title":"Boosting few-shot learning with adaptive margin loss","authors":"Li","year":"2020","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2023] Jiaming Liu, Yangqiming Wang, Tongze Zhang, Yulu Fan, Qinli Yang, and Junming Shao. Open- world semi-supervised novel class discovery. In Interna- tional Joint Conference on Artificial Intelligence, 2023.","title":"Openworld semi-supervised novel class discovery","authors":"Jiaming Liu, Yangqiming Wang, Tongze Zhang, Yulu Fan, Qinli Yang, Junming Shao","year":"2023","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Oliver et al., 2018] Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. In Neural Information Processing Systems, 2018.","title":"Realistic evaluation of deep semi-supervised learning algorithms","authors":"Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, Ian Goodfellow","year":"2018","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. ImageNet large scale visual recogni- tion challenge. International Journal of Computer Vision, 2015.","title":"ImageNet large scale visual recognition challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein","year":"2015","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Saito et al., 2021] Kuniaki Saito, Donghyun Kim, and Kate Saenko. Openmatch: Open-set semi-supervised learning with open-set consistency regularization. Neural Informa- tion Processing Systems, 2021.","title":"Openmatch: Open-set semi-supervised learning with open-set consistency regularization","authors":"Kuniaki Saito, Donghyun Kim, Kate Saenko","year":"2021","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sajjadi et al., 2016] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic trans- formations and perturbations for deep semi-supervised learning. In Neural Information Processing Systems, 2016.","title":"Regularization with stochastic transformations and perturbations for deep semi-supervised learning","authors":"Mehdi Sajjadi, Mehran Javanmardi, Tolga Tasdizen","year":"2016","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Singh et al., 2021] Ankit Singh, Omprakash Chakraborty, Ashutosh Varshney, Rameswar Panda, Rogerio Feris, Kate Saenko, and Abir Das. Semi-supervised action recognition with temporal contrastive learning. In IEEE Conference on Computer Vision and Pattern Recognition, 2021.","title":"Semi-supervised action recognition with temporal contrastive learning","authors":"Ankit Singh, Omprakash Chakraborty, Ashutosh Varshney, Rameswar Panda, Rogerio Feris, Kate Saenko, Abir Das","year":"2021","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sohn et al., 2020] Kihyuk Sohn, David Berthelot, Chun- Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. FixMatch: Simplifying semi-supervised learning with consistency and confidence. In Neural Information Processing Sys- tems, 2020.","title":"FixMatch: Simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vaze et al., 2022] Sagar Vaze, Kai Hant, Andrea Vedaldi, and Andrew Zisserman. Generalized category discov- ery. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.","title":"Generalized category discovery","authors":"Sagar Vaze, Kai Hant, Andrea Vedaldi, Andrew Zisserman","year":"2022","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang and Isola, 2020] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Interna- tional Conference on Machine Learning, 2020.","title":"Understanding contrastive representation learning through alignment and uniformity on the hypersphere","authors":"Wang, Isola ;, Tongzhou Wang, Phillip Isola","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wei and Gan, 2023] Tong Wei and Kai Gan. Towards re- alistic long-tailed semi-supervised learning: Consistency is all you need. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.","title":"Towards realistic long-tailed semi-supervised learning: Consistency is all you need","authors":"Tong Wei, Kai Gan","year":"2023","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wei et al., 2022a] Tong Wei, Qian-Yu Liu, Jiang-Xin Shi, Wei-Wei Tu, and Lan-Zhe Guo. Transfer and share: Semi- supervised learning from long-tailed data. Machine Learn- ing, 2022.","title":"Transfer and share: Semisupervised learning from long-tailed data","authors":"Wei","year":"2022","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wei et al., 2022b] Tong Wei, Hai Wang, Weiwei Tu, and Yufeng Li. Robust model selection for positive and un- labeled learning with constraints. Science China Informa- tion Sciences, 65(11):212101, 2022.","title":"Robust model selection for positive and unlabeled learning with constraints","authors":"Wei","year":"2022","venue":"Science China Information Sciences","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xu et al., 2021] Yi Xu, Lei Shang, Jinxing Ye, Qi Qian, Yu- Feng Li, Baigui Sun, Hao Li, and Rong Jin. Dash: Semi- supervised learning with dynamic thresholding. In Inter- national Conference on Machine Learning, 2021.","title":"Dash: Semisupervised learning with dynamic thresholding","authors":"Yi Xu, Lei Shang, Jinxing Ye, Qi Qian, Yu-Feng Li, Baigui Sun, Hao Li, Rong Jin","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhong et al., 2021a] Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, and Nicu Sebe. Neigh- borhood contrastive learning for novel class discovery. In IEEE conference on Computer Vision and Pattern Recog- nition, 2021.","title":"Neighborhood contrastive learning for novel class discovery","authors":"Zhong","year":"2021","venue":"IEEE conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhong et al., 2021b] Zhun Zhong, Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang, and Nicu Sebe. Openmix: Re- viving known knowledge for discovering novel visual cat- egories in an open world. In IEEE conference on Computer Vision and Pattern Recognition, 2021.","title":"Openmix: Reviving known knowledge for discovering novel visual categories in an open world","authors":"Zhong","year":"2021","venue":"IEEE conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhu and Goldberg, 2009] Xiaojin Zhu and Andrew B Gold- berg. Introduction to semi-supervised learning. 2009.","title":"Introduction to semi-supervised learning","authors":"Xiaojin Zhu, Andrew B Goldberg","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}