{"type":"ScholarlyArticle","title":"Global-residual and Local-boundary Refinement Networks for Rectifying Scene Parsing Predictions","authors":[{"name":"Rui Zhang","slug":"rui-zhang","given":"Rui","family":"Zhang","affiliation":"Key Laboratory of Intelligent Information Processing , Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China, 100190.; University of Chinese Academy of Sciences , Beijing , China, 100039."},{"name":"Sheng Tang","slug":"sheng-tang","given":"Sheng","family":"Tang","affiliation":"Key Laboratory of Intelligent Information Processing , Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China, 100190."},{"name":"Min Lin","slug":"min-lin","given":"Min","family":"Lin","affiliation":"Artificial Intelligence Institute , 360 company , Beijing , China, 100025 ."},{"name":"Jintao Li","slug":"jintao-li","given":"Jintao","family":"Li","affiliation":"Key Laboratory of Intelligent Information Processing , Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China, 100190."},{"name":"Shuicheng Yan","slug":"shuicheng-yan","given":"Shuicheng","family":"Yan","affiliation":"Artificial Intelligence Institute , 360 company , Beijing , China, 100025 ."}],"abstract":"Most of existing scene parsing methods suffer from the serious problems of both inconsistent parsing results and object boundary shift. To tackle these problems, we first propose an iterative Global-residual Refinement Network (GRN) through exploiting global contextual information to predict the parsing residuals and iteratively smoothen the inconsistent parsing labels. Furthermore, we propose a Local-boundary Refinement Network (LRN) to learn the position-adaptive propagation coefficients so that local contextual information from neighbors can be optimally captured for refining object boundaries. Finally, we cascade the proposed two refinement networks after a fully residual convolutional neural network within a uniform framework. Extensive experiments on ADE20K and Cityscapes datasets well demonstrate the effectiveness of the two refinement methods for refining scene parsing predictions.","doi":"10.24963/ijcai.2017/479","url":"https://ijcai.org/proceedings/2017/479","pdf":"https://ijcai.org/proceedings/2017/0479.pdf","markdown":"https://ijcai.org/proceedings/2017/479.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/479","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":479,"pages":{"first":3427,"last":3433},"keywords":["Machine Learning: Deep Learning","Machine Learning: Neural Networks","Robotics and Vision: Vision and Perception"],"videos":[],"references":[{"position":1,"raw":"[Badrinarayanan et al., 2015] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolutional encoder- decoder architecture for image segmentation. arXiv:1511.00561, 2015.","title":"Segnet: A deep convolutional encoderdecoder architecture for image segmentation","authors":"Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bell et al., 2016] Sean Bell, C. Lawrence Zitnick, Kavita Bala, and Ross B. Girshick. Inside-outside net: Detecting objects in con- text with skip pooling and recurrent neural networks. In Proc. of CVPR, 2016.","title":"Inside-outside net: Detecting objects in context with skip pooling and recurrent neural networks","authors":"Sean Bell, C Lawrence Zitnick, Kavita Bala, Ross B Girshick","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Byeon et al., 2015] Wonmin Byeon, Thomas M. Breuel, Federico Raue, and Marcus Liwicki. Scene labeling with LSTM recurrent neural networks. In Proc. of CVPR, 2015.","title":"Scene labeling with LSTM recurrent neural networks","authors":"Byeon","year":"2015","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2015a] Liang-Chieh Chen, George Papandreou, Ia- sonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Semantic image segmentation with deep convolutional nets and fully con- nected crfs. In ICLR, 2015.","title":"Semantic image segmentation with deep convolutional nets and fully connected crfs","authors":"Chen","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2015b] Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. Mxnet: A flexible and efficient ma- chine learning library for heterogeneous distributed systems. In Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015.","title":"Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems","authors":"Chen","year":"2015","venue":"Neural Information Processing Systems, Workshop on Machine Learning Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"Proc. of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale hierarchical im- age database. In Proc. of CVPR, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Fei-Fei Li","year":"2009","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ghiasi and Fowlkes, 2016] Golnaz Ghiasi and Charless C. Fowlkes. Laplacian pyramid reconstruction and refinement for semantic segmentation. In Proc. of ECCV, 2016.","title":"Laplacian pyramid reconstruction and refinement for semantic segmentation","authors":"Golnaz Ghiasi, Charless C Fowlkes","year":"2016","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gidaris and Komodakis, 2015] Spyros Gidaris and Nikos Ko- modakis. Object detection via a multi-region and semantic segmentation-aware CNN model. In Proc. of ICCV, 2015.","title":"Object detection via a multi-region and semantic segmentation-aware CNN model","authors":"Spyros Gidaris, Nikos Komodakis","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hariharan et al., 2015] Bharath Hariharan, Pablo Andrés Ar- beláez, Ross B. Girshick, and Jitendra Malik. Hypercolumns for object segmentation and fine-grained localization. In Proc. of CVPR, 2015.","title":"Hypercolumns for object segmentation and fine-grained localization","authors":"Bharath Hariharan, Pablo Andrés Arbeláez, Ross B Girshick, Jitendra Malik","year":"2015","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. of CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by re- ducing internal covariate shift. In Proc. of ICML, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jin et al., 2016] Xiaojie Jin, Yunpeng Chen, Jiashi Feng, Zequn Jie, and Shuicheng Yan. Multi-path feedback recurrent neural network for scene parsing. CoRR, abs/1608.07706, 2016.","title":"Multi-path feedback recurrent neural network for scene parsing","authors":"Xiaojie Jin, Yunpeng Chen, Jiashi Feng, Zequn Jie, Shuicheng Yan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krähenbühl and Koltun, 2011] Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. In Advances in Neural Information Processing Systems, 2011.","title":"Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials","authors":"Koltun Krähenbühl","year":"2011","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Ge- offrey E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2016] Ke Li, Bharath Hariharan, and Jitendra Malik. It- erative instance segmentation. In Proc. of CVPR, 2016.","title":"Iterative instance segmentation","authors":"Ke Li, Bharath Hariharan, Jitendra Malik","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liang et al., 2015] Xiaodan Liang, Chunyan Xu, Xiaohui Shen, Jianchao Yang, Si Liu, Jinhui Tang, Liang Lin, and Shuicheng Yan. Human parsing with contextualized convolutional neural network. In Proc. of ICCV, 2015.","title":"Human parsing with contextualized convolutional neural network","authors":"Xiaodan Liang, Chunyan Xu, Xiaohui Shen, Jianchao Yang, Si Liu, Jinhui Tang, Liang Lin, Shuicheng Yan","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liang et al., 2016a] Xiaodan Liang, Xiaohui Shen, Jiashi Feng, Liang Lin, and Shuicheng Yan. Semantic object parsing with graph LSTM. In Proc. of ECCV, 2016.","title":"Semantic object parsing with graph LSTM","authors":"Liang","year":"2016","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liang et al., 2016b] Xiaodan Liang, Xiaohui Shen, Donglai Xi- ang, Jiashi Feng, Liang Lin, and Shuicheng Yan. Semantic object parsing with local-global long short-term memory. In Proc. of CVPR, 2016.","title":"Semantic object parsing with local-global long short-term memory","authors":"Liang","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin et al., 2016] Guosheng Lin, Chunhua Shen, Ian D. Reid, and Anton van den Hengel. Efficient piecewise training of deep struc- tured models for semantic segmentation. In Proc. of CVPR, 2016.","title":"Efficient piecewise training of deep structured models for semantic segmentation","authors":"Guosheng Lin, Chunhua Shen, Ian D Reid, Anton Van Den, Hengel","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2015] Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, and Xiaoou Tang. Semantic image segmentation via deep parsing network. In Proc. of ICCV, 2015.","title":"Semantic image segmentation via deep parsing network","authors":"Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, Xiaoou Tang","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proc. of CVPR, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mostajabi et al., 2015] Mohammadreza Mostajabi, Payman Yadollahpour, and Gregory Shakhnarovich. Feedforward se- mantic segmentation with zoom-out features. In Proc. of CVPR, 2015.","title":"Feedforward semantic segmentation with zoom-out features","authors":"Mohammadreza Mostajabi, Payman Yadollahpour, Gregory Shakhnarovich","year":"2015","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Noh et al., 2015] Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmenta- tion. In Proc. of ICCV, 2015.","title":"Learning deconvolution network for semantic segmentation","authors":"Noh","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pinheiro and Collobert, 2014] Pedro H. O. Pinheiro and Ronan Collobert. Recurrent convolutional neural networks for scene la- beling. In Proc. of ICML, 2014.","title":"Recurrent convolutional neural networks for scene labeling","authors":"H O Pedro, Ronan Pinheiro, Collobert","year":"2014","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shrivastava et al., 2016] Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proc. of CVPR, 2016.","title":"Training region-based object detectors with online hard example mining","authors":"Abhinav Shrivastava, Abhinav Gupta, Ross Girshick","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Shuai et al., 2016] Bing Shuai, Zhen Zuo, Gang Wang, and Bing Wang. Dag-recurrent neural networks for scene labeling. In Proc. of CVPR, 2016.","title":"Dag-recurrent neural networks for scene labeling","authors":"Shuai","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vemulapalli et al., 2016] Raviteja Vemulapalli, Oncel Tuzel, Ming-Yu Liu, and Rama Chellapa. Gaussian conditional random field network for semantic segmentation. In Proc. of CVPR, 2016.","title":"Gaussian conditional random field network for semantic segmentation","authors":"Vemulapalli","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yu and Koltun, 2016] Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. In Proc. of ICLR, 2016.","title":"Multi-scale context aggregation by dilated convolutions","authors":"Koltun ; Fisher Yu, Vladlen Yu, Koltun","year":"2016","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zeiler et al., 2011] Matthew D. Zeiler, Graham W. Taylor, and Rob Fergus. Adaptive deconvolutional networks for mid and high level feature learning. In Proc. of ICCV, 2011.","title":"Adaptive deconvolutional networks for mid and high level feature learning","authors":"Matthew D Zeiler, Graham W Taylor, Rob Fergus","year":"2011","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zheng et al., 2015] Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip H. S. Torr. Conditional random fields as recurrent neural networks. In Proc. of ICCV, 2015.","title":"Conditional random fields as recurrent neural networks","authors":"Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, Philip H S Torr","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhou et al., 2016] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic un- derstanding of scenes through the ADE20K dataset. CoRR, abs/1608.05442, 2016.","title":"Semantic understanding of scenes through the ADE20K dataset","authors":"Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, Antonio Torralba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}