{"type":"ScholarlyArticle","title":"RFENet: Towards Reciprocal Feature Evolution for Glass Segmentation","authors":[{"name":"Ke Fan","slug":"ke-fan","given":"Ke","family":"Fan","affiliation":"Shanghai Jiao Tong University"},{"name":"Changan Wang","slug":"changan-wang","given":"Changan","family":"Wang","affiliation":"Tencent Youtu Lab"},{"name":"Yabiao Wang","slug":"yabiao-wang","given":"Yabiao","family":"Wang","affiliation":"Tencent Youtu Lab"},{"name":"Chengjie Wang","slug":"chengjie-wang","given":"Chengjie","family":"Wang","affiliation":"Shanghai Jiao Tong University; Tencent Youtu Lab"},{"name":"Ran Yi","slug":"ran-yi","given":"Ran","family":"Yi","affiliation":"Shanghai Jiao Tong University"},{"name":"Lizhuang Ma","slug":"lizhuang-ma","given":"Lizhuang","family":"Ma","affiliation":"Shanghai Jiao Tong University"}],"abstract":"Glass-like objects are widespread in daily life but remain intractable to be segmented for most existing methods. The transparent property makes it difficult to be distinguished from background, while the tiny separation boundary further impedes the acquisition of their exact contour. In this paper, by revealing the key co-evolution demand of semantic and boundary learning, we propose a Selective Mutual Evolution (SME) module to enable the reciprocal feature learning between them. Then to exploit the global shape context, we propose a Structurally Attentive Refinement (SAR) module to conduct a fine-grained feature refinement for those ambiguous points around the boundary. Finally, to further utilize the multi-scale representation, we integrate the above two modules into a cascaded structure and then introduce a Reciprocal Feature Evolution Network (RFENet) for effective glass-like object segmentation. Extensive experiments demonstrate that our RFENet achieves state-of-the-art performance on three popular public datasets. Code is available at https://github.com/VankouF/RFENet.","doi":"10.24963/ijcai.2023/80","url":"https://ijcai.org/proceedings/2023/80","pdf":"https://ijcai.org/proceedings/2023/0080.pdf","markdown":"https://ijcai.org/proceedings/2023/80.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/80","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":80,"pages":{"first":717,"last":725},"keywords":["Computer Vision: CV: Scene analysis and understanding","Computer Vision: CV: Segmentation"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2018a] Guanying Chen, Kai Han, and Kwan- Yee K Wong. Tom-net: Learning transparent object mat- ting from a single image. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 9233-9241, 2018.","title":"Tom-net: Learning transparent object matting from a single image","authors":"Chen","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2018b] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In Proceedings of the Euro- pean conference on computer vision (ECCV), pages 801- 818, 2018.","title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","authors":"Chen","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ding et al., 2019] Henghui Ding, Xudong Jiang, Ai Qun Liu, Nadia Magnenat Thalmann, and Gang Wang. Boundary-aware feature propagation for scene segmenta- tion. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 6819-6829, 2019.","title":"Boundary-aware feature propagation for scene segmentation","authors":"Ding","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guan et al., 2022] Huankang Guan, Jiaying Lin, and Ryn- son WH Lau. Learning semantic associations for mirror detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5941- 5950, 2022.","title":"Learning semantic associations for mirror detection","authors":"Guan","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2010] Kaiming He, Jian Sun, and Xiaoou Tang. Guided image filtering. In European conference on com- puter vision, pages 1-14. Springer, 2010.","title":"Guided image filtering","authors":"He","year":"2010","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2021] Hao He, Xiangtai Li, Guangliang Cheng, Jianping Shi, Yunhai Tong, Gaofeng Meng, Véronique Prinet, and LuBin Weng. Enhanced boundary learning for glass-like object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15859-15868, 2021.","title":"Enhanced boundary learning for glass-like object segmentation","authors":"Xiangtai Hao He, Guangliang Li, Jianping Cheng, Yunhai Shi, Gaofeng Tong, Véronique Meng, Lubin Prinet, Weng","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huo et al., 2022] Dong Huo, Jian Wang, Yiming Qian, and Yee-Hong Yang. Glass segmentation with rgb-thermal im- age pairs. arXiv preprint arXiv:2204.05453, 2022.","title":"Glass segmentation with rgb-thermal image pairs","authors":"Dong Huo, Jian Wang, Yiming Qian, Yee-Hong Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huynh et al., 2021] Chuong Huynh, Anh Tuan Tran, Khoa Luu, and Minh Hoai. Progressive semantic segmenta- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16755- 16764, 2021.","title":"Progressive semantic segmentation","authors":"Chuong Huynh, Anh Tuan Tran, Khoa Luu, Minh Hoai","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ji et al., 2023] Ge-Peng Ji, Deng-Ping Fan, Yu-Cheng Chou, Dengxin Dai, Alexander Liniger, and Luc Van Gool. Deep gradient learning for efficient camouflaged object detection. Machine Intelligence Research, 20(1):92-108, 2023.","title":"Deep gradient learning for efficient camouflaged object detection","authors":"Ge-Peng Ji, Deng-Ping Fan, Yu-Cheng Chou, Dengxin Dai, Alexander Liniger, Luc Van Gool","year":"2023","venue":"Machine Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kirillov et al., 2020] Alexander Kirillov, Yuxin Wu, Kaim- ing He, and Ross Girshick. Pointrend: Image segmenta- tion as rendering. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 9799-9808, 2020.","title":"Pointrend: Image segmentation as rendering","authors":"Alexander Kirillov, Yuxin Wu, Kaiming He, Ross Girshick","year":"2020","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krähenbühl and Koltun, 2011] Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. Advances in neural information processing systems, 24, 2011.","title":"Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials","authors":"Koltun Krähenbühl","year":"2011","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018] Xin Li, Fan Yang, Hong Cheng, Wei Liu, and Dinggang Shen. Contour knowledge transfer for salient object detection. In Proceedings of the european conference on computer vision (ECCV), pages 355-370, 2018.","title":"Contour knowledge transfer for salient object detection","authors":"Xin Li, Fan Yang, Hong Cheng, Wei Liu, Dinggang Shen","year":"2018","venue":"Proceedings of the european conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2020] Jiaying Lin, Guodong Wang, and Ryn- son WH Lau. Progressive mirror detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 3697-3705, 2020.","title":"Progressive mirror detection","authors":"Jiaying Lin, Guodong Wang, Rynson Wh Lau","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2021] Jiaying Lin, Zebang He, and Rynson WH Lau. Rich context aggregation with reflection prior for glass surface detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13415-13424, 2021.","title":"Rich context aggregation with reflection prior for glass surface detection","authors":"Jiaying Lin, Zebang He, Rynson Wh Lau","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2022] Jiang-Jiang Liu, Qibin Hou, Zhi-Ang Liu, and Ming-Ming Cheng. Poolnet+: Exploring the potential of pooling for salient object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):887- 904, 2022.","title":"Poolnet+: Exploring the potential of pooling for salient object detection","authors":"Jiang-Jiang Liu, Qibin Hou, Zhi-Ang Liu, Ming-Ming Cheng","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431- 3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mei et al., 2020] Haiyang Mei, Xin Yang, Yang Wang, Yuanyuan Liu, Shengfeng He, Qiang Zhang, Xiaopeng Wei, and Rynson WH Lau. Don't hit me! glass detec- tion in real-world scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3687-3696, 2020.","title":"Don't hit me! glass detection in real-world scenes","authors":"Haiyang Mei, Xin Yang, Yang Wang, Yuanyuan Liu, Shengfeng He, Qiang Zhang, Xiaopeng Wei, Rynson Wh Lau","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Milletari et al., 2016] Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565-571. IEEE, 2016.","title":"V-net: Fully convolutional neural networks for volumetric medical image segmentation","authors":"Fausto Milletari, Nassir Navab, Seyed-Ahmad Ahmadi","year":"2016","venue":"fourth international conference on 3D vision (3DV)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pang et al., 2020] Youwei Pang, Xiaoqi Zhao, Lihe Zhang, and Huchuan Lu. Multi-scale interactive network for salient object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9413-9422, 2020.","title":"Multi-scale interactive network for salient object detection","authors":"Youwei Pang, Xiaoqi Zhao, Lihe Zhang, Huchuan Lu","year":"2020","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formation processing systems, 32, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Qin et al., 2019] Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jager- sand. Basnet: Boundary-aware salient object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7479-7489, 2019.","title":"Basnet: Boundary-aware salient object detection","authors":"Qin","year":"2019","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Takikawa et al., 2019] Towaki Takikawa, David Acuna, Varun Jampani, and Sanja Fidler. Gated-scnn: Gated shape cnns for semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5229-5238, 2019.","title":"Gated-scnn: Gated shape cnns for semantic segmentation","authors":"Takikawa","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tan et al., 2022] Xin Tan, Jiaying Lin, Ke Xu, Pan Chen, Lizhuang Ma, and Rynson WH Lau. Mirror detection with the visual chirality cue. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2022.","title":"Mirror detection with the visual chirality cue","authors":"Xin Tan, Jiaying Lin, Ke Xu, Pan Chen, Lizhuang Ma, Rynson Wh Lau","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2019] Wenguan Wang, Shuyang Zhao, Jian- bing Shen, Steven CH Hoi, and Ali Borji. Salient object detection with pyramid attention and salient edges. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1448-1457, 2019.","title":"Salient object detection with pyramid attention and salient edges","authors":"Wang","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2020] Enze Xie, Wenjia Wang, Wenhai Wang, Mingyu Ding, Chunhua Shen, and Ping Luo. Segmenting transparent objects in the wild. In European conference on computer vision, pages 696-711. Springer, 2020.","title":"Segmenting transparent objects in the wild","authors":"Xie","year":"2020","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2015] Yichao Xu, Hajime Nagahara, Atsushi Shi- mada, and Rin-ichiro Taniguchi. Transcut: Transparent object segmentation from a light-field image. In Proceed- ings of the IEEE International Conference on Computer Vision, pages 3442-3450, 2015.","title":"Transcut: Transparent object segmentation from a light-field image","authors":"Xu","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2018] Maoke Yang, Kun Yu, Chi Zhang, Zhi- wei Li, and Kuiyuan Yang. Denseaspp for semantic seg- mentation in street scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3684-3692, 2018.","title":"Denseaspp for semantic segmentation in street scenes","authors":"Yang","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2019] Xin Yang, Haiyang Mei, Ke Xu, Xi- aopeng Wei, Baocai Yin, and Rynson WH Lau. Where is my mirror? In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 8809-8818, 2019.","title":"Where is my mirror?","authors":"Xin Yang, Haiyang Mei, Ke Xu, Xiaopeng Wei, Baocai Yin, Rynson Wh Lau","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yu et al., 2018] Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Bisenet: Bi- lateral segmentation network for real-time semantic seg- mentation. In Proceedings of the European conference on computer vision (ECCV), pages 325-341, 2018.","title":"Bisenet: Bilateral segmentation network for real-time semantic segmentation","authors":"Yu","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yu et al., 2022] Letian Yu, Haiyang Mei, Wen Dong, Ziqi Wei, Li Zhu, Yuxin Wang, and Xin Yang. Progressive glass segmentation. IEEE Transactions on Image Processing, 31:2920-2933, 2022.","title":"Progressive glass segmentation","authors":"Letian Yu, Haiyang Mei, Wen Dong, Ziqi Wei, Li Zhu, Yuxin Wang, Xin Yang","year":"2022","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhao et al., 2017] Hengshuang Zhao, Jianping Shi, Xiao- juan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881- 2890, 2017.","title":"Pyramid scene parsing network","authors":"Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhao et al., 2019] Jia-Xing Zhao, Jiang-Jiang Liu, Deng- Ping Fan, Yang Cao, Jufeng Yang, and Ming-Ming Cheng. Egnet: Edge guidance network for salient object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8779-8788, 2019.","title":"Egnet: Edge guidance network for salient object detection","authors":"Jia-Xing Zhao, Jiang-Jiang Liu, Deng-Ping Fan, Yang Cao, Jufeng Yang, Ming-Ming Cheng","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zheng et al., 2022] Chengyu Zheng, Ding Shi, Xuefeng Yan, Dong Liang, Mingqiang Wei, Xin Yang, Yanwen Guo, and Haoran Xie. Glassnet: Label decoupling-based three-stream neural network for robust image glass detec- tion. In Computer Graphics Forum, volume 41, pages 377-388. Wiley Online Library, 2022.","title":"Glassnet: Label decoupling-based three-stream neural network for robust image glass detection","authors":"Chengyu Zheng, Ding Shi, Xuefeng Yan, Dong Liang, Mingqiang Wei, Xin Yang, Yanwen Guo, Haoran Xie","year":"2022","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhuge et al., 2022] Mingchen Zhuge, Deng-Ping Fan, Nian Liu, Dingwen Zhang, Dong Xu, and Ling Shao. Salient object detection via integrity learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.","title":"Salient object detection via integrity learning","authors":"Zhuge","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}