{"type":"ScholarlyArticle","title":"Distilling Inter-Class Distance for Semantic Segmentation","authors":[{"name":"Zhengbo Zhang","slug":"zhengbo-zhang","given":"Zhengbo","family":"Zhang","affiliation":"Wuhan University"},{"name":"Chunluan Zhou","slug":"chunluan-zhou","given":"Chunluan","family":"Zhou","affiliation":"Wormpex AI Research"},{"name":"Zhigang Tu","slug":"zhigang-tu","given":"Zhigang","family":"Tu","affiliation":"Wuhan University"}],"abstract":"Knowledge distillation is widely adopted in semantic segmentation to reduce the computation cost. The previous knowledge distillation methods for semantic segmentation focus on pixel-wise feature alignment and intra-class feature variation distillation, neglecting to transfer the knowledge of the inter-class distance in the feature space, which is important for semantic segmentation such a pixel-wise classification task. To address this issue, we propose an Inter-class Distance Distillation (IDD) method to transfer the inter-class distance in the feature space from the teacher network to the student network. Furthermore, semantic segmentation is a position-dependent task, thus we exploit a position information distillation module to help the student network encode more position information. Extensive experiments on three popular datasets: Cityscapes, Pascal VOC and ADE20K show that our method is helpful to improve the accuracy of semantic segmentation models and achieves the state-of-the-art performance. E.g. it boosts the benchmark model (``PSPNet+ResNet18\") by 7.50% in accuracy on the Cityscapes dataset.","doi":"10.24963/ijcai.2022/235","url":"https://ijcai.org/proceedings/2022/235","pdf":"https://ijcai.org/proceedings/2022/0235.pdf","markdown":"https://ijcai.org/proceedings/2022/235.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/235","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":235,"pages":{"first":1686,"last":1692},"keywords":["Computer Vision: Scene analysis and understanding","Computer Vision: Segmentation","Computer Vision: Recognition (object detection, categorization)","Computer Vision: Transfer, low-shot, semi- and un- supervised learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986998","length_sec":73,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985141","length_sec":465,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chen et al., 2017] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convo- lutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intel- ligence, 40(4):834-848, 2017.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene under- standing. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 3213-3223, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dong et al., 2020] Genshun Dong, Yan Yan, Chunhua Shen, and Hanzi Wang. Real-time high-performance semantic image segmentation of urban street scenes. IEEE Transac- tions on Intelligent Transportation Systems, 22(6):3258- 3274, 2020.","title":"Real-time high-performance semantic image segmentation of urban street scenes","authors":"Dong","year":"2020","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Everingham et al., 2015] Mark Everingham, SM Ali Es- lami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. International journal of com- puter vision, 111(1):98-136, 2015.","title":"The pascal visual object classes challenge: A retrospective","authors":"Mark Everingham, Ali Sm, Luc Eslami, Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2015","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2021] Mingyuan Fan, Shenqi Lai, Junshi Huang, Xiaoming Wei, Zhenhua Chai, Junfeng Luo, and Xiaolin Wei. Rethinking bisenet for real-time semantic segmen- tation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9716- 9725, 2021.","title":"Rethinking bisenet for real-time semantic segmentation","authors":"Mingyuan Fan, Shenqi Lai, Junshi Huang, Xiaoming Wei, Zhenhua Chai, Junfeng Luo, Xiaolin Wei","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2019] Tong He, Chunhua Shen, Zhi Tian, Dong Gong, Changming Sun, and Youliang Yan. Knowledge adaptation for efficient semantic segmentation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 578-587, 2019.","title":"Knowledge adaptation for efficient semantic segmentation","authors":"Tong He, Chunhua Shen, Zhi Tian, Dong Gong, Changming Sun, Youliang Yan","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Islam et al., 2020] Md Amirul Islam, Sen Jia, and Neil DB Bruce. How much position information do convo- lutional neural networks encode? arXiv preprint arXiv:2001.08248, 2020.","title":"How much position information do convolutional neural networks encode","authors":"Amirul Md, Sen Islam, Neil Db Jia, Bruce","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kemker et al., 2018] Ronald Kemker, Carl Salvaggio, and Christopher Kanan. Algorithms for semantic segmenta- tion of multispectral remote sensing imagery using deep learning. ISPRS journal of photogrammetry and remote sensing, 145:60-77, 2018.","title":"Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning","authors":"Ronald Kemker, Carl Salvaggio, Christopher Kanan","year":"2018","venue":"ISPRS journal of photogrammetry and remote sensing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2019] Yifan Liu, Ke Chen, Chris Liu, Zengchang Qin, Zhenbo Luo, and Jingdong Wang. Structured knowl- edge distillation for semantic segmentation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2604-2613, 2019.","title":"Structured knowledge distillation for semantic segmentation","authors":"Yifan Liu, Ke Chen, Chris Liu, Zengchang Qin, Zhenbo Luo, Jingdong Wang","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431- 3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mehta et al., 2018] Sachin Mehta, Mohammad Rastegari, Anat Caspi, Linda Shapiro, and Hannaneh Hajishirzi. Es- pnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation. In Proceedings of the european conference on computer vision (ECCV), pages 552-568, 2018.","title":"Espnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation","authors":"Sachin Mehta, Mohammad Rastegari, Anat Caspi, Linda Shapiro, Hannaneh Hajishirzi","year":"2018","venue":"Proceedings of the european conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Paszke et al., 2016] Adam Paszke, Abhishek Chaurasia, Sangpil Kim, and Eugenio Culurciello. Enet: A deep neu- ral network architecture for real-time semantic segmenta- tion. arXiv preprint arXiv:1606.02147, 2016.","title":"Enet: A deep neural network architecture for real-time semantic segmentation","authors":"Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shu et al., 2021] Changyong Shu, Yifan Liu, Jianfei Gao, Zheng Yan, and Chunhua Shen. Channel-wise knowledge distillation for dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5311-5320, 2021.","title":"Channel-wise knowledge distillation for dense prediction","authors":"Changyong Shu, Yifan Liu, Jianfei Gao, Zheng Yan, Chunhua Shen","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tu et al., 2019] Zhigang Tu, Hongyan Li, Dejun Zhang, Justin Dauwels, Baoxin Li, and Junsong Yuan. Action- stage emphasized spatiotemporal vlad for video action recognition. IEEE Transactions on Image Processing, 28(6):2799-2812, 2019.","title":"Actionstage emphasized spatiotemporal vlad for video action recognition","authors":"Zhigang Tu, Hongyan Li, Dejun Zhang, Justin Dauwels, Baoxin Li, Junsong Yuan","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2020] Yukang Wang, Wei Zhou, Tao Jiang, Xi- ang Bai, and Yongchao Xu. Intra-class feature variation distillation for semantic segmentation. In European Con- ference on Computer Vision, pages 346-362. Springer, 2020.","title":"Intra-class feature variation distillation for semantic segmentation","authors":"Yukang Wang, Wei Zhou, Tao Jiang, Xiang Bai, Yongchao Xu","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yuan et al., 2018] Yuhui Yuan, Lang Huang, Jianyuan Guo, Chao Zhang, Xilin Chen, and Jingdong Wang. Ocnet: Object context network for scene parsing. arXiv preprint arXiv:1809.00916, 2018.","title":"Ocnet: Object context network for scene parsing","authors":"Yuhui Yuan, Lang Huang, Jianyuan Guo, Chao Zhang, Xilin Chen, Jingdong Wang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yuan et al., 2020] Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object-contextual representations for semantic seg- mentation. In Computer Vision-ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23-28, 2020, Pro- ceedings, Part VI 16, pages 173-190. Springer, 2020.","title":"Object-contextual representations for semantic segmentation","authors":"Yuhui Yuan, Xilin Chen, Jingdong Wang","year":"2020","venue":"Computer Vision-ECCV 2020: 16th European Conference","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhao et al., 2017] Hengshuang Zhao, Jianping Shi, Xiao- juan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881- 2890, 2017.","title":"Pyramid scene parsing network","authors":"Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhao et al., 2018] Hengshuang Zhao, Xiaojuan Qi, Xiaoy- ong Shen, Jianping Shi, and Jiaya Jia. Icnet for real-time semantic segmentation on high-resolution images. In Pro- ceedings of the European conference on computer vision (ECCV), pages 405-420, 2018.","title":"Icnet for real-time semantic segmentation on high-resolution images","authors":"Zhao","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou et al., 2017] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 633-641, 2017.","title":"Scene parsing through ade20k dataset","authors":"Zhou","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhu et al., 2019] Zhen Zhu, Mengde Xu, Song Bai, Tengteng Huang, and Xiang Bai. Asymmetric non-local neural networks for semantic segmentation. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 593-602, 2019.","title":"Asymmetric non-local neural networks for semantic segmentation","authors":"Zhen Zhu, Mengde Xu, Song Bai, Tengteng Huang, Xiang Bai","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}