{"type":"ScholarlyArticle","title":"Region-level Contrastive and Consistency Learning for Semi-Supervised Semantic Segmentation","authors":[{"name":"Jianrong Zhang","slug":"jianrong-zhang","given":"Jianrong","family":"Zhang","affiliation":"College of Computer Science and Technology , Jilin University , Changchun , China"},{"name":"Tianyi Wu","slug":"tianyi-wu","given":"Tianyi","family":"Wu","affiliation":"Institute of Deep Learning , Baidu Research , Beijing , China; National Engineering Laboratory for Deep Learning Technology and Application , Beijing , China"},{"name":"Chuanghao Ding","slug":"chuanghao-ding","given":"Chuanghao","family":"Ding","affiliation":"College of Software , Jilin University , Changchun , China"},{"name":"Hongwei Zhao","slug":"hongwei-zhao","given":"Hongwei","family":"Zhao","affiliation":"College of Computer Science and Technology , Jilin University , Changchun , China"},{"name":"Guodong Guo","slug":"guodong-guo","given":"Guodong","family":"Guo","affiliation":"Institute of Deep Learning , Baidu Research , Beijing , China; National Engineering Laboratory for Deep Learning Technology and Application , Beijing , China"}],"abstract":"Current semi-supervised semantic segmentation methods mainly focus on designing pixel-level consistency and contrastive regularization. However, pixel-level regularization is sensitive to noise from pixels with incorrect predictions, and pixel-level contrastive regularization has a large memory and computational cost. To address the issues, we propose a novel region-level contrastive and consistency learning framework (RC^2L) for semi-supervised semantic segmentation. Specifically, we first propose a Region Mask Contrastive (RMC) loss and a Region Feature Contrastive (RFC) loss to accomplish region-level contrastive property. Furthermore, Region Class Consistency (RCC) loss and Semantic Mask Consistency (SMC) loss are proposed for achieving region-level consistency. Based on the proposed region-level contrastive and consistency regularization, we develop a region-level contrastive and consistency learning framework (RC^2L) for semi-supervised semantic segmentation, and evaluate our RC^2L on two challenging benchmarks (PASCAL VOC 2012 and Cityscapes), outperforming the state-of-the-art.","doi":"10.24963/ijcai.2022/226","url":"https://ijcai.org/proceedings/2022/226","pdf":"https://ijcai.org/proceedings/2022/0226.pdf","markdown":"https://ijcai.org/proceedings/2022/226.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/226","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":226,"pages":{"first":1622,"last":1628},"keywords":["Computer Vision: Segmentation","Computer Vision: Transfer, low-shot, semi- and un- supervised learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986965","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985108","length_sec":537,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Alonso et al., 2021] Iñigo Alonso, Alberto Sabater, David Ferstl, Luis Montesano, and Ana C Murillo. Semi-supervised semantic segmentation with pixel-level contrastive learning from a class- wise memory bank. In ICCV, 2021.","title":"Semi-supervised semantic segmentation with pixel-level contrastive learning from a classwise memory bank","authors":"Iñigo Alonso, Alberto Sabater, David Ferstl, Luis Montesano, Ana C Murillo","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213-229. Springer, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Liang-Chieh Chen, George Papandreou, Ia- sonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Se- mantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2018.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020a] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020b] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020.","title":"Improved baselines with momentum contrastive learning","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2021] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic segmentation with cross pseudo supervision. In CVPR, 2021.","title":"Semi-supervised semantic segmentation with cross pseudo supervision","authors":"Xiaokang Chen, Yuhui Yuan, Gang Zeng, Jingdong Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2021] Bowen Cheng, Alexander G. Schwing, and Alexander Kirillov. Per-pixel classification is not all you need for semantic segmentation. In NeurIPS, 2021.","title":"Per-pixel classification is not all you need for semantic segmentation","authors":"Bowen Cheng, Alexander G Schwing, Alexander Kirillov","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Everingham et al., 2015] Mark Everingham, SM Ali Eslami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes challenge: A retrospec- tive. International Journal of Computer Vision, 111(1):98-136, 2015.","title":"The pascal visual object classes challenge: A retrospective","authors":"Mark Everingham, Ali Sm, Luc Eslami, Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2015","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[French et al., 2020] Geoff French, Timo Aila, Samuli Laine, Michal Mackiewicz, and Graham Finlayson. Semi-supervised semantic segmentation needs strong, high-dimensional perturba- tions. In BMVC, 2020.","title":"Semi-supervised semantic segmentation needs strong, high-dimensional perturbations","authors":"Geoff French, Timo Aila, Samuli Laine, Michal Mackiewicz, Graham Finlayson","year":"2020","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2021] Ruifei He, Jihan Yang, and Xiaojuan Qi. Re- distributing biased pseudo labels for semi-supervised semantic segmentation: A baseline investigation. In ICCV, pages 6930- 6940, 2021.","title":"Redistributing biased pseudo labels for semi-supervised semantic segmentation: A baseline investigation","authors":"He","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hung et al., 2018] Wei-Chih Hung, Yi-Hsuan Tsai, Yan-Ting Liou, Yen-Yu Lin, and Ming-Hsuan Yang. Adversarial learning for semi-supervised semantic segmentation. In BMVC, 2018.","title":"Adversarial learning for semi-supervised semantic segmentation","authors":"Wei-Chih Hung, Yi-Hsuan Tsai, Yan-Ting Liou, Yen-Yu Lin, Ming-Hsuan Yang","year":"2018","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ke et al., 2020] Zhanghan Ke, Di Qiu, Kaican Li, Qiong Yan, and Rynson WH Lau. Guided collaborative training for pixel-wise semi-supervised learning. In ECCV, 2020.","title":"Guided collaborative training for pixel-wise semi-supervised learning","authors":"Zhanghan Ke, Di Qiu, Kaican Li, Qiong Yan, Rynson Wh Lau","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lai et al., 2021] Xin Lai, Zhuotao Tian, Li Jiang, Shu Liu, Heng- shuang Zhao, Liwei Wang, and Jiaya Jia. Semi-supervised se- mantic segmentation with directional context-aware consistency. In CVPR, 2021.","title":"Semi-supervised semantic segmentation with directional context-aware consistency","authors":"Xin Lai, Zhuotao Tian, Li Jiang, Shu Liu, Hengshuang Zhao, Liwei Wang, Jiaya Jia","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, James Hays Serge Belongie, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in con- text. In ECCV, 2014.","title":"Microsoft coco: Common objects in context","authors":"Lin","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaim- ing He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Milletari et al., 2016] F Milletari, N Navab, SA Ahmadi, and V- net. Fully convolutional neural networks for volumetric medical image segmentation. In 3DV, 2016.","title":"Fully convolutional neural networks for volumetric medical image segmentation","authors":"Milletari, Navab, Ahmadi, Vnet","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Miyato et al., 2018] Takeru Miyato, Shin-ichi Maeda, Shin Ishii, and Masanori Koyama. Virtual adversarial training: a regu- larization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 2018.","title":"Virtual adversarial training: a regularization method for supervised and semi-supervised learning","authors":"Miyato","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ouali et al., 2020] Yassine Ouali, Céline Hudelot, and Myriam Tami. Semi-supervised semantic segmentation with cross- consistency training. In CVPR, 2020.","title":"Semi-supervised semantic segmentation with crossconsistency training","authors":"Yassine Ouali, Céline Hudelot, Myriam Tami","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consis- tency targets improve semi-supervised deep learning results. In NeurIPS, 2017.","title":"Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2021] Wenguan Wang, Tianfei Zhou, Fisher Yu, Jifeng Dai, Ender Konukoglu, and Luc Van Gool. Exploring cross-image pixel contrast for semantic segmentation. In ICCV, pages 7303-7313, 2021.","title":"Exploring cross-image pixel contrast for semantic segmentation","authors":"Wang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2020a] Tianyi Wu, Yu Lu, Yu Zhu, Chuang Zhang, Ming Wu, Zhanyu Ma, and Guodong Guo. Ginet: Graph in- teraction network for scene parsing. In European Conference on Computer Vision, pages 34-51. Springer, 2020.","title":"Ginet: Graph interaction network for scene parsing","authors":"Wu","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2020b] Tianyi Wu, Sheng Tang, Rui Zhang, Juan Cao, and Yongdong Zhang. Cgnet: A light-weight context guided net- work for semantic segmentation. IEEE Transactions on Image Processing, 30:1169-1179, 2020.","title":"Cgnet: A light-weight context guided network for semantic segmentation","authors":"Wu","year":"2020","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2021] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. arXiv preprint arXiv:2105.15203, 2021.","title":"Segformer: Simple and efficient design for semantic segmentation with transformers","authors":"Xie","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yun et al., 2019] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ICCV, pages 6023-6032, 2019.","title":"Cutmix: Regularization strategy to train strong classifiers with localizable features","authors":"Yun","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhong et al., 2021] Yuanyi Zhong, Bodi Yuan, Hong Wu, Zhiqiang Yuan, Jian Peng, and Yu-Xiong Wang. Pixel contrastive-consistent semi-supervised semantic segmentation. In ICCV, pages 7273-7282, 2021.","title":"Pixel contrastive-consistent semi-supervised semantic segmentation","authors":"Yuanyi Zhong, Bodi Yuan, Hong Wu, Zhiqiang Yuan, Jian Peng, Yu-Xiong Wang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zoph et al., 2020] Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin D. Cubuk, and Quoc V. Le. Rethinking pre-training and self-training. CoRR, abs/2006.06882, 2020.","title":"Rethinking pre-training and self-training","authors":"Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin D Cubuk, Quoc V Le","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zou et al., 2021] Yuliang Zou, Zizhao Zhang, Han Zhang, Chun- Liang Li, Xiao Bian, Jia-Bin Huang, and Tomas Pfister. Pseu- doseg: Designing pseudo labels for semantic segmentation. In ICLR, 2021.","title":"Pseudoseg: Designing pseudo labels for semantic segmentation","authors":"Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, Tomas Pfister","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}