{"type":"ScholarlyArticle","title":"Double-Check Soft Teacher for Semi-Supervised Object Detection","authors":[{"name":"Kuo Wang","slug":"kuo-wang","given":"Kuo","family":"Wang","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China"},{"name":"Yuxiang Nie","slug":"yuxiang-nie","given":"Yuxiang","family":"Nie","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China"},{"name":"Chaowei Fang","slug":"chaowei-fang","given":"Chaowei","family":"Fang","affiliation":"School of Artificial Intelligence , Xidian University , Xi'an , China"},{"name":"Chengzhi Han","slug":"chengzhi-han","given":"Chengzhi","family":"Han","affiliation":"Huawei Technologies Co . China"},{"name":"Xuewen Wu","slug":"xuewen-wu","given":"Xuewen","family":"Wu","affiliation":"Huawei Technologies Co . China"},{"name":"Xiaohui Wang Wang","slug":"xiaohui-wang-wang","given":"Xiaohui Wang","family":"Wang","affiliation":"Huawei Technologies Co . China"},{"name":"Liang Lin","slug":"liang-lin","given":"Liang","family":"Lin","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China"},{"name":"Fan Zhou","slug":"fan-zhou","given":"Fan","family":"Zhou","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China"},{"name":"Guanbin Li","slug":"guanbin-li","given":"Guanbin","family":"Li","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China"}],"abstract":"In the semi-supervised object detection task, due to the scarcity of labeled data and the diversity and complexity of objects to be detected, the quality of pseudo-labels generated by existing methods for unlabeled data is relatively low, which severely restricts the performance of semi-supervised object detection. In this paper, we revisit the pseudo-labeling based Teacher-Student mutual learning framework for semi-supervised object detection and identify that the inconsistency of the location and feature of the candidate object proposals between the Teacher and the Student branches are the fatal cause of the low quality of the pseudo labels. To address this issue, we propose a simple yet effective technique within the mainstream teacher-student framework, called Double Check Soft Teacher, to overcome the harm caused by insufficient quality of pseudo labels. Specifically, our proposed method leverages teacher model to generate pseudo labels for the student model. Especially, the candidate boxes generated by the student model based on the pseudo label will be sent to the teacher model for \"double check\", and then the teacher model will output probabilistic soft label with background class for those candidate boxes, which will be used to train the student model. Together with a pseudo labeling mechanism based on the sum of the TOP-K prediction score, which improves the recall rate of pseudo labels, Double Check Soft Teacher consistently surpasses state-of-the-art methods by significant margins on the MS-COCO benchmark, pushing the new state-of-the-art.  Source codes are available at https://github.com/wkfdb/DCST.","doi":"10.24963/ijcai.2022/199","url":"https://ijcai.org/proceedings/2022/199","pdf":"https://ijcai.org/proceedings/2022/0199.pdf","markdown":"https://ijcai.org/proceedings/2022/199.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/199","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":199,"pages":{"first":1430,"last":1436},"keywords":["Machine Learning: Semi-Supervised Learning","Computer Vision: Recognition (object detection, categorization)"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986563","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984706","length_sec":336,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi- supervised learning. Advances in Neural Information Pro- cessing Systems, page 5049-5059, 2019.","title":"Mixmatch: A holistic approach to semisupervised learning","authors":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin Raffel","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155, 2019.","title":"MMDetection: Open mmlab detection toolbox and benchmark","authors":"Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, Dahua Lin","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Jeong et al., 2019] Jisoo Jeong, Seungeui Lee, Jeesoo Kim, and Nojun Kwak. Consistency-based semi-supervised learning for object detection. Advances in Neural Infor- mation Processing Systems, 32:10759-10768, 2019.","title":"Consistency-based semi-supervised learning for object detection","authors":"Jisoo Jeong, Seungeui Lee, Jeesoo Kim, Nojun Kwak","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Laine and Aila, 2017] Samuli Laine and Timo Aila. Tempo- ral ensembling for semi-supervised learning. Proceedings of the International Conference on Learning Representa- tions (ICLR), 2017.","title":"Temporal ensembling for semi-supervised learning","authors":"Laine, Aila ;, Samuli Laine, Timo Aila","year":"2017","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2019] Xinzhe Li, Qianru Sun, Yaoyao Liu, Qin Zhou, Shibao Zheng, Tat-Seng Chua, and Bernt Schiele. Learning to self-train for semi-supervised few-shot classi- fication. Advances in Neural Information Processing Sys- tems, 32:10276-10286, 2019.","title":"Learning to self-train for semi-supervised few-shot classification","authors":"Li","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In European conference on com- puter vision, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Fea- ture pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117-2125, 2017.","title":"Feature pyramid networks for object detection","authors":"Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, Serge Belongie","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2021] Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi- supervised object detection. In ICLR, 2021.","title":"Unbiased teacher for semisupervised object detection","authors":"Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, Peter Vajda","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Miyato et al., 2018] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 41(8):1979-1993, 2018.","title":"Virtual adversarial training: a regularization method for supervised and semi-supervised learning","authors":"Miyato","year":"2018","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. Advances in neural information processing systems, 28:91-99, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Sohn et al., 2020a] Kihyuk Sohn, David Berthelot, Chun- Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Process- ing Systems, 2020.","title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sohn et al., 2020b] Kihyuk Sohn, Zizhao Zhang, Chun- Liang Li, Han Zhang, Chen-Yu Lee, and Tomas Pfister. A simple semi-supervised learning framework for object detection. In arXiv:2005.04757, 2020.","title":"A simple semi-supervised learning framework for object detection","authors":"Sohn","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tang et al., 2021a] Peng Tang, Chetan Ramaiah, Yan Wang, Ran Xu, and Caiming Xiong. Proposal learning for semi-supervised object detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 2291-2301, 2021.","title":"Proposal learning for semi-supervised object detection","authors":"Tang","year":"2021","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tang et al., 2021b] Yihe Tang, Weifeng Chen, Yijun Luo, and Yuting Zhang. Humble teachers teach better students for semi-supervised object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 3132-3141, 2021.","title":"Humble teachers teach better students for semi-supervised object detection","authors":"Tang","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. Advances in neural information pro- cessing systems, pages 1195-1204, 2017.","title":"Mean teachers are better role models: Weightaveraged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2021] Zhenyu Wang, Yali Li, Ye Guo, Lu Fang, and Shengjin Wang. Data-uncertainty guided multi-phase learning for semi-supervised object detection. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4568-4577, 2021.","title":"Data-uncertainty guided multi-phase learning for semi-supervised object detection","authors":"Zhenyu Wang, Yali Li, Ye Guo, Lu Fang, Shengjin Wang","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Xie et al., 2020a] Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le. Unsupervised data augmentation for consistency training. Advances in Neural Information Processing Systems, 2020.","title":"Unsupervised data augmentation for consistency training","authors":"Xie","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xie et al., 2020b] Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10687-10698, 2020.","title":"Self-training with noisy student improves imagenet classification","authors":"Xie","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2021] Mengde Xu, Zheng Zhang, Han Hu, Jian- feng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. ICCV, 2021.","title":"End-to-end semi-supervised object detection with soft teacher","authors":"Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, Zicheng Liu","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yang et al., 2021] Qize Yang, Xihan Wei, Biao Wang, Xian- Sheng Hua, and Lei Zhang. Interactive self-training with mean teachers for semi-supervised object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5941-5950, 2021.","title":"Interactive self-training with mean teachers for semi-supervised object detection","authors":"Yang","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou et al., 2021] Qiang Zhou, Chaohui Yu, Zhibin Wang, Qi Qian, and Hao Li. Instant-teaching: An end-to-end semi-supervised object detection framework. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4081-4090, 2021.","title":"Instant-teaching: An end-to-end semi-supervised object detection framework","authors":"Qiang Zhou, Chaohui Yu, Zhibin Wang, Qi Qian, Hao Li","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}