{"type":"ScholarlyArticle","title":"AllMatch: Exploiting All Unlabeled Data for Semi-Supervised Learning","authors":[{"name":"Zhiyu Wu","slug":"zhiyu-wu","given":"Zhiyu","family":"Wu","affiliation":"National Key Laboratory of General Artificial Intelligence , School of Intelligence Science and Technology , Peking University"},{"name":"Jinshi Cui","slug":"jinshi-cui","given":"Jinshi","family":"Cui","affiliation":"National Key Laboratory of General Artificial Intelligence , School of Intelligence Science and Technology , Peking University"}],"abstract":"Existing semi-supervised learning algorithms adopt pseudo-labeling and consistency regulation techniques to introduce supervision signals for unlabeled samples. To overcome the inherent limitation of threshold-based pseudo-labeling, prior studies have attempted to align the confidence threshold with the evolving learning status of the model, which is estimated through the predictions made on the unlabeled data. In this paper, we further reveal that classifier weights can reflect the differentiated learning status across categories and consequently propose a class-specific adaptive threshold mechanism. Additionally, considering that even the optimal threshold scheme cannot resolve the problem of discarding unlabeled samples, a binary classification consistency regulation approach is designed to distinguish candidate classes from negative options for all unlabeled samples. By combining the above strategies, we present a novel SSL algorithm named AllMatch, which achieves improved pseudo-label accuracy and a 100% utilization ratio for the unlabeled data. We extensively evaluate our approach on multiple benchmarks, encompassing both balanced and imbalanced settings. The results demonstrate that AllMatch consistently outperforms existing state-of-the-art methods.","doi":"10.24963/ijcai.2024/580","url":"https://ijcai.org/proceedings/2024/580","pdf":"https://ijcai.org/proceedings/2024/0580.pdf","markdown":"https://ijcai.org/proceedings/2024/580.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/580","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":580,"pages":{"first":5245,"last":5253},"keywords":["Machine Learning: ML: Semi-supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Arazo et al., 2020] Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo- labeling and confirmation bias in deep semi-supervised learning. In 2020 International Joint Conference on Neu- ral Networks (IJCNN), pages 1-8. IEEE, 2020.","title":"Pseudolabeling and confirmation bias in deep semi-supervised learning","authors":"Eric Arazo, Diego Ortego, Paul Albert, E O' Noel, Kevin Connor, Mcguinness","year":"2020","venue":"2020 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2009] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learn- ing. In Proceedings of the 26th annual international con- ference on machine learning, pages 41-48, 2009.","title":"Curriculum learning","authors":"Yoshua Bengio, Jérôme Louradour, Ronan Collobert, Jason Weston","year":"2009","venue":"Proceedings of the 26th annual international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berthelot et al., 2019a] David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. arXiv preprint arXiv:1911.09785, 2019.","title":"Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring","authors":"Berthelot","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Berthelot et al., 2019b] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi- supervised learning. Advances in neural information pro- cessing systems, 32, 2019.","title":"Mixmatch: A holistic approach to semisupervised learning","authors":"Berthelot","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2023] Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang, Bernt Schiele, Xing Xie, Bhik- sha Raj, and Marios Savvides. Softmatch: Addressing the quantity-quality trade-off in semi-supervised learning. arXiv preprint arXiv:2301.10921, 2023.","title":"Softmatch: Addressing the quantity-quality trade-off in semi-supervised learning","authors":"Ran Hao Chen, Yue Tao, Yidong Fan, Jindong Wang, Bernt Wang, Xing Schiele, Bhiksha Xie, Marios Raj, Savvides","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Coates et al., 2011] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelli- gence and statistics, pages 215-223. JMLR Workshop and Conference Proceedings, 2011.","title":"An analysis of single-layer networks in unsupervised feature learning","authors":"Adam Coates, Andrew Ng, Honglak Lee","year":"2011","venue":"Proceedings of the fourteenth international conference on artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fan et al., 2022] Yue Fan, Dengxin Dai, Anna Kukleva, and Bernt Schiele. Cossl: Co-learning of representation and classifier for imbalanced semi-supervised learning. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14574-14584, 2022.","title":"Cossl: Co-learning of representation and classifier for imbalanced semi-supervised learning","authors":"Dengxin Yue Fan, Anna Dai, Bernt Kukleva, Schiele","year":"2022","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Grandvalet and Bengio, 2004] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimiza- tion. Advances in neural information processing systems, 17, 2004.","title":"Semi-supervised learning by entropy minimization","authors":"Bengio Grandvalet, Yves Grandvalet, Yoshua Bengio","year":"2004","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kang et al., 2019] Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1910.09217, 2019.","title":"Decoupling representation and classifier for long-tailed recognition","authors":"Kang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krause et al., 2010] Andreas Krause, Pietro Perona, and Ryan Gomes. Discriminative clustering by regularized in- formation maximization. Advances in neural information processing systems, 23, 2010.","title":"Discriminative clustering by regularized information maximization","authors":"Andreas Krause, Pietro Perona, Ryan Gomes","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lai et al., 2022] Zhengfeng Lai, Chao Wang, Henrry Gu- nawan, Sen-Ching S Cheung, and Chen-Nee Chuah. Smoothed adaptive weighting for imbalanced semi- supervised learning: Improve reliability against unknown distribution data. In International Conference on Machine Learning, pages 11828-11843. PMLR, 2022.","title":"Smoothed adaptive weighting for imbalanced semisupervised learning: Improve reliability against unknown distribution data","authors":"Lai","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Laine and Aila, 2016] Samuli Laine and Timo Aila. Tem- poral ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242, 2016.","title":"Temporal ensembling for semi-supervised learning","authors":"Laine, Aila ;, Samuli Laine, Timo Aila","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lee and others, 2013] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896, 2013.","title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","authors":"Dong-Hyun Lee","year":"2013","venue":"Workshop on challenges in representation learning, ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee et al., 2021] Hyuck Lee, Seungjae Shin, and Heey- oung Kim. Abc: Auxiliary balanced classifier for class- imbalanced semi-supervised learning. Advances in Neural Information Processing Systems, 34:7082-7094, 2021.","title":"Abc: Auxiliary balanced classifier for classimbalanced semi-supervised learning","authors":"Hyuck Lee, Seungjae Shin, Heeyoung Kim","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2021] Junnan Li, Caiming Xiong, and Steven CH Hoi. Comatch: Semi-supervised learning with contrastive graph regularization. In Proceedings of the IEEE/CVF in- ternational conference on computer vision, pages 9475- 9484, 2021.","title":"Comatch: Semi-supervised learning with contrastive graph regularization","authors":"Li","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learn- ing. 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Oh et al., 2022] Youngtaek Oh, Dong-Jin Kim, and In So Kweon. Daso: Distribution-aware semantics-oriented pseudo-label for imbalanced semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9786-9796, 2022.","title":"Daso: Distribution-aware semantics-oriented pseudo-label for imbalanced semi-supervised learning","authors":"Oh","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pham et al., 2021] Hieu Pham, Zihang Dai, Qizhe Xie, and Quoc V Le. Meta pseudo labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11557-11568, 2021.","title":"Meta pseudo labels","authors":"Hieu Pham, Zihang Dai, Qizhe Xie, Quoc V Le","year":"2021","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rosenberg et al., 2005] Chuck Rosenberg, Martial Hebert, and Henry Schneiderman. Semi-supervised self-training of object detection models. 2005.","title":"Semi-supervised self-training of object detection models","authors":"Rosenberg","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sajjadi et al., 2016] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic trans- formations and perturbations for deep semi-supervised learning. Advances in neural information processing sys- tems, 29, 2016.","title":"Regularization with stochastic transformations and perturbations for deep semi-supervised learning","authors":"Mehdi Sajjadi, Mehran Javanmardi, Tolga Tasdizen","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sohn et al., 2020] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with con- sistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.","title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. Advances in neural information pro- cessing systems, 30, 2017.","title":"Mean teachers are better role models: Weightaveraged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2022] Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Marios Savvides, Takahiro Shinozaki, Bhik- sha Raj, Zhen Wu, and Jindong Wang. Freematch: Self- adaptive thresholding for semi-supervised learning. arXiv preprint arXiv:2205.07246, 2022.","title":"Freematch: Selfadaptive thresholding for semi-supervised learning","authors":"Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj, Zhen Wu, Jindong Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wei et al., 2021] Chen Wei, Kihyuk Sohn, Clayton Mellina, Alan Yuille, and Fan Yang. Crest: A class-rebalancing self-training framework for imbalanced semi-supervised learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10857- 10866, 2021.","title":"Crest: A class-rebalancing self-training framework for imbalanced semi-supervised learning","authors":"Chen Wei, Kihyuk Sohn, Clayton Mellina, Alan Yuille, Fan Yang","year":"2021","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2020] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmen- tation for consistency training. Advances in neural infor- mation processing systems, 33:6256-6268, 2020.","title":"Unsupervised data augmentation for consistency training","authors":"Xie","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2021] Yi Xu, Lei Shang, Jinxing Ye, Qi Qian, Yu- Feng Li, Baigui Sun, Hao Li, and Rong Jin. Dash: Semi- supervised learning with dynamic thresholding. In Inter- national Conference on Machine Learning, pages 11525- 11536. PMLR, 2021.","title":"Dash: Semisupervised learning with dynamic thresholding","authors":"Yi Xu, Lei Shang, Jinxing Ye, Qi Qian, Yu-Feng Li, Baigui Sun, Hao Li, Rong Jin","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zagoruyko and Komodakis, 2016] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.","title":"Wide residual networks","authors":"Sergey Zagoruyko, Nikos Komodakis","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2017] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Be- yond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.","title":"mixup: Beyond empirical risk minimization","authors":"Hongyi Zhang, Moustapha Cisse, David Yann N Dauphin, Lopez-Paz","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2021] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi- supervised learning with curriculum pseudo labeling. Advances in Neural Information Processing Systems, 34:18408-18419, 2021.","title":"Flexmatch: Boosting semisupervised learning with curriculum pseudo labeling","authors":"Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, Takahiro Shinozaki","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhao et al., 2022] Zhen Zhao, Luping Zhou, Yue Duan, Lei Wang, Lei Qi, and Yinghuan Shi. Dc-ssl: Addressing mis- matched class distribution in semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9757-9765, 2022.","title":"Dc-ssl: Addressing mismatched class distribution in semi-supervised learning","authors":"Zhen Zhao, Luping Zhou, Yue Duan, Lei Wang, Lei Qi, Yinghuan Shi","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zheng et al., 2022] Mingkai Zheng, Shan You, Lang Huang, Fei Wang, Chen Qian, and Chang Xu. Sim- match: Semi-supervised learning with similarity match- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14471- 14481, 2022.","title":"Simmatch: Semi-supervised learning with similarity matching","authors":"Zheng","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhou et al., 2020] Tianyi Zhou, Shengjie Wang, and Jeff Bilmes. Time-consistent self-supervision for semi- supervised learning. In International Conference on Ma- chine Learning, pages 11523-11533. PMLR, 2020.","title":"Time-consistent self-supervision for semi-supervised learning","authors":"Tianyi Zhou, Shengjie Wang, Jeff Bilmes","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhu, 2005] Xiaojin Jerry Zhu. Semi-supervised learning lit- erature survey. 2005.","title":"Semi-supervised learning literature survey","authors":"Jerry Xiaojin, Zhu","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}