{"type":"ScholarlyArticle","title":"CARD: Semi-supervised Semantic Segmentation via Class-agnostic Relation based Denoising","authors":[{"name":"Xiaoyang Wang","slug":"xiaoyang-wang","given":"Xiaoyang","family":"Wang","affiliation":"Xi'an Jiaotong Liverpool University; University of Liverpool; Dinnar Automation Technology"},{"name":"Jimin Xiao","slug":"jimin-xiao","given":"Jimin","family":"Xiao","affiliation":"Xi'an Jiaotong Liverpool University"},{"name":"Bingfeng Zhang","slug":"bingfeng-zhang","given":"Bingfeng","family":"Zhang","affiliation":"Xi'an Jiaotong Liverpool University; University of Liverpool"},{"name":"Limin Yu","slug":"limin-yu","given":"Limin","family":"Yu","affiliation":"Xi'an Jiaotong Liverpool University"}],"abstract":"Recent semi-supervised semantic segmentation methods focus on mining extra supervision from unlabeled data by generating pseudo labels. However, noisy labels are inevitable in this process which prevent effective self-supervision. This paper proposes that noisy labels can be corrected based on semantic connections among features. Since a segmentation classifier produces both high and low-quality predictions, we can trace back to feature encoder to investigate how a feature in a noisy group is related to those in the confident groups. Discarding the weak predictions from the classifier, rectified predictions are assigned to the wrongly predicted features through the feature relations. The key to such an idea lies in mining reliable feature connections. With this goal, we propose a class-agnostic relation network to precisely capture semantic connections among features while ignoring their semantic categories. The feature relations enable us to perform effective noisy label corrections to boost self-training performance. Extensive experiments on PASCAL VOC and Cityscapes demonstrate the state-of-the-art performances of the proposed methods under various semi-supervised settings.","doi":"10.24963/ijcai.2022/202","url":"https://ijcai.org/proceedings/2022/202","pdf":"https://ijcai.org/proceedings/2022/0202.pdf","markdown":"https://ijcai.org/proceedings/2022/202.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/202","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":202,"pages":{"first":1451,"last":1457},"keywords":["Computer Vision: Segmentation","Machine Learning: Semi-Supervised Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986709","length_sec":79,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984852","length_sec":324,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Alonso et al., 2021] Iñigo Alonso, Alberto Sabater, David Ferstl, Luis Montesano, and Ana C Murillo. Semi- supervised semantic segmentation with pixel-level con- trastive learning from a class-wise memory bank. In ICCV, 2021.","title":"Semisupervised semantic segmentation with pixel-level contrastive learning from a class-wise memory bank","authors":"Iñigo Alonso, Alberto Sabater, David Ferstl, Luis Montesano, Ana C Murillo","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arazo et al., 2020] Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo- labeling and confirmation bias in deep semi-supervised learning. In IJCNN, 2020.","title":"Pseudolabeling and confirmation bias in deep semi-supervised learning","authors":"Eric Arazo, Diego Ortego, Paul Albert, E O' Noel, Kevin Connor, Mcguinness","year":"2020","venue":"IJCNN","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi- supervised learning. In NeurlPS, 2019.","title":"Mixmatch: A holistic approach to semisupervised learning","authors":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin Raffel","year":"2019","venue":"NeurlPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convo- lutional nets, atrous convolution, and fully connected crfs. TPAMI, 2017.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Liang-Chieh Chen, Raphael Gontijo Lopes, Bowen Cheng, Maxwell D Collins, Ekin D Cubuk, Barret Zoph, Hartwig Adam, and Jonathon Shlens. Naive- student: Leveraging semi-supervised learning in video se- quences for urban scene segmentation. In ECCV, 2020.","title":"Naivestudent: Leveraging semi-supervised learning in video sequences for urban scene segmentation","authors":"Liang-Chieh Chen, Raphael Gontijo Lopes, Bowen Cheng, Maxwell D Collins, Ekin D Cubuk, Barret Zoph, Hartwig Adam, Jonathon Shlens","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2021] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic seg- mentation with cross pseudo supervision. In CVPR, 2021.","title":"Semi-supervised semantic segmentation with cross pseudo supervision","authors":"Xiaokang Chen, Yuhui Yuan, Gang Zeng, Jingdong Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene under- standing. In CVPR, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. IJCV, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Feng et al., 2020] Zhengyang Feng, Qianyu Zhou, Qiqi Gu, Xin Tan, Guangliang Cheng, Xuequan Lu, Jian- ping Shi, and Lizhuang Ma. Dmt: Dynamic mutual training for semi-supervised learning. arXiv preprint arXiv:2004.08514, 2020.","title":"Dmt: Dynamic mutual training for semi-supervised learning","authors":"Feng","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[French et al., 2020] Geoff French, Samuli Laine, Timo Aila, Michal Mackiewicz, and Graham Finlayson. Semi- supervised semantic segmentation needs strong, varied perturbations. In BMVC, 2020.","title":"Semisupervised semantic segmentation needs strong, varied perturbations","authors":"Geoff French, Samuli Laine, Timo Aila, Michal Mackiewicz, Graham Finlayson","year":"2020","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Han et al., 2018] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurlPS, 2018.","title":"Co-teaching: Robust training of deep neural networks with extremely noisy labels","authors":"Han","year":"2018","venue":"NeurlPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hariharan et al., 2011] Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In ICCV, 2011.","title":"Semantic contours from inverse detectors","authors":"Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, Jitendra Malik","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2021] Ruifei He, Jihan Yang, and Xiaojuan Qi. Re-distributing biased pseudo labels for semi-supervised semantic segmentation: A baseline investigation. In ICCV, 2021.","title":"Re-distributing biased pseudo labels for semi-supervised semantic segmentation: A baseline investigation","authors":"He","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hung et al., 2018] Wei-Chih Hung, Yi-Hsuan Tsai, Yan- Ting Liou, Yen-Yu Lin, and Ming-Hsuan Yang. Adversar- ial learning for semi-supervised semantic segmentation. In BMVC, 2018.","title":"Adversarial learning for semi-supervised semantic segmentation","authors":"Wei-Chih Hung, Yi-Hsuan Tsai, Yan-Ting Liou, Yen-Yu Lin, Ming-Hsuan Yang","year":"2018","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ke et al., 2019] Zhanghan Ke, Daoye Wang, Qiong Yan, Jimmy Ren, and Rynson WH Lau. Dual student: Break- ing the limits of the teacher in semi-supervised learning. In ICCV, 2019.","title":"Dual student: Breaking the limits of the teacher in semi-supervised learning","authors":"Ke","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ke et al., 2020] Zhanghan Ke, Di Qiu, Kaican Li, Qiong Yan, and Rynson WH Lau. Guided collaborative training for pixel-wise semi-supervised learning. In ECCV, 2020.","title":"Guided collaborative training for pixel-wise semi-supervised learning","authors":"Zhanghan Ke, Di Qiu, Kaican Li, Qiong Yan, Rynson Wh Lau","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In ECCV, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Luo et al., 2018] Yucen Luo, Jun Zhu, Mengxi Li, Yong Ren, and Bo Zhang. Smooth neighbors on teacher graphs for semi-supervised learning. In CVPR, 2018.","title":"Smooth neighbors on teacher graphs for semi-supervised learning","authors":"Luo","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mendel et al., 2020] Robert Mendel, Luis Antonio De Souza, David Rauber, João Paulo Papa, and Christoph Palm. Semi-supervised segmentation based on error- correcting supervision. In ECCV, 2020.","title":"Semi-supervised segmentation based on errorcorrecting supervision","authors":"Robert Mendel, Luis Antonio De, David Souza, João Rauber, Paulo Papa, Christoph Palm","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mittal et al., 2019] Sudhanshu Mittal, Maxim Tatarchenko, and Thomas Brox. Semi-supervised semantic segmenta- tion with high-and low-level consistency. TPAMI, 2019.","title":"Semi-supervised semantic segmentation with high-and low-level consistency","authors":"Sudhanshu Mittal, Maxim Tatarchenko, Thomas Brox","year":"2019","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Olsson et al., 2021] Viktor Olsson, Wilhelm Tranheden, Juliano Pinto, and Lennart Svensson. Classmix: Segmentation-based data augmentation for semi- supervised learning. In WACV, 2021.","title":"Classmix: Segmentation-based data augmentation for semisupervised learning","authors":"Olsson","year":"2021","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sohn et al., 2020] Kihyuk Sohn, David Berthelot, Chun- Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurlPS, 2020.","title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"NeurlPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2021] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi- supervised learning with curriculum pseudo labeling. In NeurlPS, 2021.","title":"Flexmatch: Boosting semisupervised learning with curriculum pseudo labeling","authors":"Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, Takahiro Shinozaki","year":"2021","venue":"NeurlPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2021] Yanning Zhou, Hang Xu, Wei Zhang, Bin Gao, and Pheng-Ann Heng. C3-semiseg: Contrastive semi-supervised segmentation via cross-set learning and dynamic class-balancing. In ICCV, 2021.","title":"C3-semiseg: Contrastive semi-supervised segmentation via cross-set learning and dynamic class-balancing","authors":"Yanning Zhou, Hang Xu, Wei Zhang, Bin Gao, Pheng-Ann Heng","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}