{"type":"ScholarlyArticle","title":"Low-Confidence Samples Mining for Semi-supervised Object Detection","authors":[{"name":"Guandu Liu","slug":"guandu-liu","given":"Guandu","family":"Liu","affiliation":"School of Software , Tsinghua University , China; Beijing National Research Center for Information Science and Technology (BNRist) , China"},{"name":"Fangyuan Zhang","slug":"fangyuan-zhang","given":"Fangyuan","family":"Zhang","affiliation":"School of Software , Tsinghua University , China; Beijing National Research Center for Information Science and Technology (BNRist) , China"},{"name":"Tianxiang Pan","slug":"tianxiang-pan","given":"Tianxiang","family":"Pan","affiliation":"School of Software , Tsinghua University , China; Beijing National Research Center for Information Science and Technology (BNRist) , China"},{"name":"Jun-Hai Yong","slug":"jun-hai-yong","given":"Jun-Hai","family":"Yong","affiliation":"School of Software , Tsinghua University , China; Beijing National Research Center for Information Science and Technology (BNRist) , China"},{"name":"Bin Wang","slug":"bin-wang","given":"Bin","family":"Wang","affiliation":"School of Software , Tsinghua University , China; Beijing National Research Center for Information Science and Technology (BNRist) , China"}],"abstract":"Reliable pseudo labels from unlabeled data play a key role in semi-supervised object detection (SSOD). However, the state-of-the-art SSOD methods all rely on pseudo labels with high confidence, which ignore valuable pseudo labels with lower confidence. Additionally, the insufficient excavation for unlabeled data results in an excessively low recall rate thus hurting the network training. In this paper, we propose a novel Low-confidence Samples Mining (LSM) method to utilize low confidence pseudo labels efficiently. Specifically, we develop an additional pseudo information mining (PIM) branch on account of low-resolution feature maps to extract reliable large area instances, the IoUs of which are higher than small area ones. Owing to the complementary predictions between PIM and the main branch, we further design self-distillation (SD) to compensate for both in a mutually learning manner. Meanwhile, the extensibility of the above approaches enables our LSM to apply to Faster-RCNN and Deformable-DETR respectively. On the MS-COCO benchmark, our method achieves 3.54% mAP improvement over state-of-the-art methods under 5% labeling ratios.","doi":"10.24963/ijcai.2023/132","url":"https://ijcai.org/proceedings/2023/132","pdf":"https://ijcai.org/proceedings/2023/0132.pdf","markdown":"https://ijcai.org/proceedings/2023/132.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/132","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":132,"pages":{"first":1186,"last":1194},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)","Data Mining: DM: Applications","Data Mining: DM: Exploratory data mining"],"videos":[],"references":[{"position":1,"raw":"[Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: a holistic approach to semi- supervised learning. In NeurIPS, volume 32, 2019.","title":"Mixmatch: a holistic approach to semisupervised learning","authors":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin A Raffel","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213-229, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2022a] Binbin Chen, Weijie Chen, Shicai Yang, Yunyi Xuan, Jie Song, Di Xie, Shiliang Pu, Mingli Song, and Yueting Zhuang. Label matching semi-supervised ob- ject detection. In CVPR, pages 14381-14390, 2022.","title":"Label matching semi-supervised object detection","authors":"Chen","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2022b] Binghui Chen, Pengyu Li, Xiang Chen, Biao Wang, Lei Zhang, and Xian-Sheng Hua. Dense learn- ing based semi-supervised object detection. In CVPR, pages 4815-4824, 2022.","title":"Dense learning based semi-supervised object detection","authors":"Chen","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cheng et al., 2020] Tianheng Cheng, Xinggang Wang, Lichao Huang, and Wenyu Liu. Boundary-preserving mask r-cnn. In ECCV, pages 660-676, 2020.","title":"Boundary-preserving mask r-cnn","authors":"Cheng","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. {ImageNet}: a large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"{ImageNet}: a large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. pages 303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2022] Qiushan Guo, Yao Mu, Jianyu Chen, Tianqi Wang, Yizhou Yu, and Ping Luo. Scale-equivalent distillation for semi-supervised object detection. In CVPR, pages 14522-14531, 2022.","title":"Scale-equivalent distillation for semi-supervised object detection","authors":"Qiushan Guo, Yao Mu, Jianyu Chen, Tianqi Wang, Yizhou Yu, Ping Luo","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, pages 2961-2969, 2017.","title":"Kaiming He, Georgia Gkioxari","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Izmailov et al., 2018] Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. In arXiv preprint arXiv:1803.05407, 2018.","title":"Averaging weights leads to wider optima and better generalization","authors":"Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, Andrew Gordon, Wilson","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jeong et al., 2019] Jisoo Jeong, Seungeui Lee, Jeesoo Kim, and Nojun Kwak. Consistency-based semi-supervised learning for object detection. In NeurIPS, volume 32, 2019.","title":"Consistency-based semi-supervised learning for object detection","authors":"Jisoo Jeong, Seungeui Lee, Jeesoo Kim, Nojun Kwak","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim and Lee, 2020] Kang Kim and Hee Seok Lee. Prob- abilistic anchor assignment with iou prediction for object detection. In ECCV, pages 355-371, 2020.","title":"Probabilistic anchor assignment with iou prediction for object detection","authors":"Kang Kim, Hee Seok, Lee","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kim et al., 2022] JongMok Kim, Jooyoung Jang, Se- unghyeon Seo, Jisoo Jeong, Jongkeun Na, and Nojun Kwak. Mum: mix image tiles and unmix feature tiles for semi-supervised object detection. In CVPR, pages 14512- 14521, 2022.","title":"Mum: mix image tiles and unmix feature tiles for semi-supervised object detection","authors":"Kim","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2019] Yanghao Li, Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. Scale-aware trident networks for object detection. In ICCV, pages 6054-6063, 2019.","title":"Scale-aware trident networks for object detection","authors":"Li","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2022a] Gang Li, Xiang Li, Yujie Wang, Shanshan Zhang, Yichao Wu, and Ding Liang. Pseco: pseudo la- beling and consistency training for semi-supervised object detection. In ECCV, pages 1-17, 2022.","title":"Pseco: pseudo labeling and consistency training for semi-supervised object detection","authors":"Li","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2022b] Hengduo Li, Zuxuan Wu, Abhinav Shri- vastava, and Larry S Davis. Rethinking pseudo labels for semi-supervised object detection. In AAAI, pages 1314- 1322, 2022.","title":"Rethinking pseudo labels for semi-supervised object detection","authors":"Li","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C Lawrence Zitnick. Microsoft {COCO}: common objects in context. In ECCV, pages 740-755, 2014.","title":"Microsoft {COCO}: common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2017] Weibo Liu, Zidong Wang, Xiaohui Liu, Ni- anyin Zeng, Yurong Liu, and Fuad E Alsaadi. A survey of deep neural network architectures and their applications. In Neurocomputing, pages 11-26, 2017.","title":"A survey of deep neural network architectures and their applications","authors":"Weibo Liu, Zidong Wang, Xiaohui Liu, Nianyin Zeng, Yurong Liu, Fuad E Alsaadi","year":"2017","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2021] Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi- supervised object detection. In ICLR, pages 1-17, 2021.","title":"Unbiased teacher for semisupervised object detection","authors":"Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, Peter Vajda","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: towards real-time ob- ject detection with region proposal networks. In NeurIPS, volume 28, 2015.","title":"Faster r-cnn: towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sajjadi et al., 2016] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic trans- formations and perturbations for deep semi-supervised learning. In NeurIPS, volume 29, 2016.","title":"Regularization with stochastic transformations and perturbations for deep semi-supervised learning","authors":"Mehdi Sajjadi, Mehran Javanmardi, Tolga Tasdizen","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shao et al., 2019] Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: a large-scale, high-quality dataset for object detection. In ICCV, pages 8430-8439, 2019.","title":"Objects365: a large-scale, high-quality dataset for object detection","authors":"Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, Jian Sun","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Singh et al., 2018] Bharat Singh, Mahyar Najibi, and Larry S Davis. Sniper: efficient multi-scale training. In NeurIPS, volume 31, 2018.","title":"Sniper: efficient multi-scale training","authors":"Bharat Singh, Mahyar Najibi, Larry S Davis","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sohn et al., 2020a] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raf- fel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: simplifying semi-supervised learning with consistency and confidence. In NeurIPS, pages 596-608, 2020.","title":"Fixmatch: simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sohn et al., 2020b] Kihyuk Sohn, Zizhao Zhang, Chun- Liang Li, Han Zhang, Chen-Yu Lee, and Tomas Pfister. A simple semi-supervised learning framework for object detection. In arXiv preprint arXiv:2005.04757, 2020.","title":"A simple semi-supervised learning framework for object detection","authors":"Sohn","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tang et al., 2021] Yihe Tang, Weifeng Chen, Yijun Luo, and Yuting Zhang. Humble teachers teach better students for semi-supervised object detection. In CVPR, pages 3132-3141, 2021.","title":"Humble teachers teach better students for semi-supervised object detection","authors":"Yihe Tang, Weifeng Chen, Yijun Luo, Yuting Zhang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: weight- averaged consistency targets improve semi-supervised deep learning results. In NeurIPS, volume 30, 2017.","title":"Mean teachers are better role models: weightaveraged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2022] Kuo Wang, Yuxiang Nie, Chaowei Fang, Chengzhi Han, Xuewen Wu, Xiaohui Wang, Liang Lin, Fan Zhou, and Guanbin Li. Double-check soft teacher for semi-supervised object detection. In IJCAI, pages 1430- 1436, 2022.","title":"Double-check soft teacher for semi-supervised object detection","authors":"Kuo Wang, Yuxiang Nie, Chaowei Fang, Chengzhi Han, Xuewen Wu, Xiaohui Wang, Liang Lin, Fan Zhou, Guanbin Li","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/199"},{"position":29,"raw":"[Xie et al., 2020a] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmen- tation for consistency training. In NeurIPS, pages 6256- 6268, 2020.","title":"Unsupervised data augmentation for consistency training","authors":"Xie","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xie et al., 2020b] Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. In CVPR, pages 10687- 10698, 2020.","title":"Self-training with noisy student improves imagenet classification","authors":"Xie","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2021] Mengde Xu, Zheng Zhang, Han Hu, Jian- feng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. In ICCV, pages 3060-3069, 2021.","title":"End-to-end semi-supervised object detection with soft teacher","authors":"Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, Zicheng Liu","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2021] Qize Yang, Xihan Wei, Biao Wang, Xian- Sheng Hua, and Lei Zhang. Interactive self-training with mean teachers for semi-supervised object detection. In CVPR, pages 5941-5950, 2021.","title":"Interactive self-training with mean teachers for semi-supervised object detection","authors":"Yang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2022] Fangyuan Zhang, Tianxiang Pan, and Bin Wang. Semi-supervised object detection with adap- tive class-rebalancing self-training. In AAAI, pages 3252- 3261, 2022.","title":"Semi-supervised object detection with adaptive class-rebalancing self-training","authors":"Fangyuan Zhang, Tianxiang Pan, Bin Wang","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zheng et al., 2022] Shida Zheng, Chenshu Chen, Xiaowei Cai, Tingqun Ye, and Wenming Tan. Dual decoupling training for semi-supervised object detection with noise- bypass head. In AAAI, pages 3526-3534, 2022.","title":"Dual decoupling training for semi-supervised object detection with noisebypass head","authors":"Shida Zheng, Chenshu Chen, Xiaowei Cai, Tingqun Ye, Wenming Tan","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhou et al., 2021] Qiang Zhou, Chaohui Yu, Zhibin Wang, Qi Qian, and Hao Li. Instant-teaching: an end-to-end semi-supervised object detection framework. In CVPR, pages 4081-4090, 2021.","title":"Instant-teaching: an end-to-end semi-supervised object detection framework","authors":"Qiang Zhou, Chaohui Yu, Zhibin Wang, Qi Qian, Hao Li","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhu et al., 2021] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: de- formable transformers for end-to-end object detection. In ICLR, pages 1-16, 2021.","title":"Deformable detr: deformable transformers for end-to-end object detection","authors":"Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}