{"type":"ScholarlyArticle","title":"High Resolution Feature Recovering for Accelerating Urban Scene Parsing","authors":[{"name":"Rui Zhang","slug":"rui-zhang","given":"Rui","family":"Zhang","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Sheng Tang","slug":"sheng-tang","given":"Sheng","family":"Tang","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China"},{"name":"Luoqi Liu","slug":"luoqi-liu","given":"Luoqi","family":"Liu","affiliation":"Qihoo"},{"name":"Yongdong Zhang","slug":"yongdong-zhang","given":"Yongdong","family":"Zhang","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China; Artificial Intelligence Institute , Beijing , China"},{"name":"Jintao Li","slug":"jintao-li","given":"Jintao","family":"Li","affiliation":"Institute of Computing Technology , Chinese Academy of Sciences , Beijing , China"},{"name":"Shuicheng Yan","slug":"shuicheng-yan","given":"Shuicheng","family":"Yan","affiliation":"Qihoo; Department of Electrical and Computer Engineering , National University of Singapore , Singapore"}],"abstract":"Both accuracy and speed are equally important in urban scene parsing. Most of the existing methods mainly focus on improving parsing accuracy, ignoring the problem of low inference speed due to large-sized input and high resolution feature maps. To tackle this issue, we propose a High Resolution Feature Recovering (HRFR) framework to accelerate a given parsing network. A Super-Resolution Recovering module is employed to recover features of large original-sized images from features of down-sampled input. Therefore, our framework can combine the advantages of (1) fast speed of networks with down-sampled input and (2) high accuracy of networks with large original-sized input. Additionally, we employ auxiliary intermediate supervision and boundary region re-weighting to facilitate the optimization of the network. Extensive experiments on the two challenging Cityscapes and CamVid datasets well demonstrate the effectiveness of the proposed HRFR framework, which can accelerate the scene parsing inference process by about 3.0x speedup from 1/2 down-sampled input with negligible accuracy reduction.","doi":"10.24963/ijcai.2018/161","url":"https://ijcai.org/proceedings/2018/161","pdf":"https://ijcai.org/proceedings/2018/0161.pdf","markdown":"https://ijcai.org/proceedings/2018/161.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/161","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":161,"pages":{"first":1156,"last":1162},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Ba and Caruana, 2014] Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? In NIPS, 2014.","title":"Do deep nets really need to be deep?","authors":"Caruana Ba, Jimmy Ba, Rich Caruana","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Badrinarayanan et al., 2017] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolutional encoder- decoder architecture for scene segmentation. TPAMI, 2017.","title":"Segnet: A deep convolutional encoderdecoder architecture for scene segmentation","authors":"Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brostow et al., 2009] Gabriel J. Brostow, Julien Fauqueur, and Roberto Cipolla. Semantic object classes in video: A high- definition ground truth database. Pattern Recognition Letters, 2009.","title":"Semantic object classes in video: A highdefinition ground truth database","authors":"J Gabriel, Julien Brostow, Roberto Fauqueur, Cipolla","year":"2009","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2016a] Liang-Chieh Chen, George Papandreou, Ia- sonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. arXiv preprint arXiv:1606.00915, 2016.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Chen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2016b] Liang-Chieh Chen, Yi Yang, Jiang Wang, Wei Xu, and Alan L. Yuille. Attention to scale: Scale-aware semantic image segmentation. In CVPR, 2016.","title":"Attention to scale: Scale-aware semantic image segmentation","authors":"Chen","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Courbariaux et al., 2015] Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural net- works with binary weights during propagations. In NIPS, 2015.","title":"Binaryconnect: Training deep neural networks with binary weights during propagations","authors":"Matthieu Courbariaux, Yoshua Bengio, Jean-Pierre David","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Denton et al., 2014] Emily L. Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus. Exploiting linear structure within convolutional networks for efficient evaluation. In NIPS, 2014.","title":"Exploiting linear structure within convolutional networks for efficient evaluation","authors":"Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann Lecun, Rob Fergus","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghiasi and Fowlkes, 2016] Golnaz Ghiasi and Charless C. Fowlkes. Laplacian pyramid reconstruction and refinement for semantic segmentation. In ECCV, 2016.","title":"Laplacian pyramid reconstruction and refinement for semantic segmentation","authors":"Golnaz Ghiasi, Charless C Fowlkes","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Han et al., 2016] Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural network with prun- ing, trained quantization and huffman coding. In ICLR, 2016.","title":"Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding","authors":"Han","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Iandola et al., 2016] Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 0.5 mb model size. arXiv preprint arXiv:1602.07360, 2016.","title":"Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 0.5 mb model size","authors":"Song Forrest N Iandola, Matthew W Han, Khalid Moskewicz, William J Ashraf, Kurt Dally, Keutzer","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jaderberg et al., 2014] Max Jaderberg, Andrea Vedaldi, and An- drew Zisserman. Speeding up convolutional neural networks with low rank expansions. In BMVC, 2014.","title":"Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. Speeding up convolutional neural networks with low rank expansions","authors":"Jaderberg","year":"2014","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Ge- offrey E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2017] Quanquan Li, Shengying Jin, and Junjie Yan. Mimicking very efficient network for object detection. In CVPR, 2017.","title":"Mimicking very efficient network for object detection","authors":"Li","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liang et al., 2016] Xiaodan Liang, Xiaohui Shen, Jiashi Feng, Liang Lin, and Shuicheng Yan. Semantic object parsing with graph LSTM. In ECCV, 2016.","title":"Semantic object parsing with graph LSTM","authors":"Xiaodan Liang, Xiaohui Shen, Jiashi Feng, Liang Lin, Shuicheng Yan","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lin et al., 2016] Guosheng Lin, Chunhua Shen, Ian D. Reid, and Anton van den Hengel. Efficient piecewise training of deep struc- tured models for semantic segmentation. In CVPR, 2016.","title":"Efficient piecewise training of deep structured models for semantic segmentation","authors":"Guosheng Lin, Chunhua Shen, Ian D Reid, Anton Van Den, Hengel","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2015] Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, and Xiaoou Tang. Semantic image segmentation via deep parsing network. In ICCV, 2015.","title":"Semantic image segmentation via deep parsing network","authors":"Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, Xiaoou Tang","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mostajabi et al., 2015] Mohammadreza Mostajabi, Payman Yadollahpour, and Gregory Shakhnarovich. Feedforward semantic segmentation with zoom-out features. In CVPR, 2015.","title":"Feedforward semantic segmentation with zoom-out features","authors":"Mohammadreza Mostajabi, Payman Yadollahpour, Gregory Shakhnarovich","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Paszke et al., 2016] Adam Paszke, Abhishek Chaurasia, Sangpil Kim, and Eugenio Culurciello. Enet: A deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147, 2016.","title":"Enet: A deep neural network architecture for real-time semantic segmentation","authors":"Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pohlen et al., 2017] Tobias Pohlen, Alexander Hermans, Markus Mathias, and Bastian Leibe. Full-resolution residual networks for semantic segmentation in street scenes. In CVPR, 2017.","title":"Full-resolution residual networks for semantic segmentation in street scenes","authors":"Pohlen","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Romero et al., 2015] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015.","title":"Fitnets: Hints for thin deep nets","authors":"Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shelhamer et al., 2017] Evan Shelhamer, Jonathan Long, and Trevor Darrell. Fully convolutional networks for semantic seg- mentation. TPAMI, 2017.","title":"Fully convolutional networks for semantic segmentation","authors":"Evan Shelhamer, Jonathan Long, Trevor Darrell","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shuai et al., 2016] Bing Shuai, Zhen Zuo, Gang Wang, and Bing Wang. Dag-recurrent neural networks for scene labeling. In CVPR, 2016.","title":"Dag-recurrent neural networks for scene labeling","authors":"Shuai","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Soudry et al., 2014] Daniel Soudry, Itay Hubara, and Ron Meir. Expectation backpropagation: Parameter-free training of multi- layer neural networks with continuous or discrete weights. In NIPS, 2014.","title":"Expectation backpropagation: Parameter-free training of multilayer neural networks with continuous or discrete weights","authors":"Daniel Soudry, Itay Hubara, Ron Meir","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vemulapalli et al., 2016] Raviteja Vemulapalli, Oncel Tuzel, Ming-Yu Liu, and Rama Chellapa. Gaussian conditional random field network for semantic segmentation. In CVPR, 2016.","title":"Gaussian conditional random field network for semantic segmentation","authors":"Vemulapalli","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wu et al., 2016a] Jiaxiang Wu, Cong Leng, Yuhang Wang, Qing- hao Hu, and Jian Cheng. Quantized convolutional neural net- works for mobile devices. In CVPR, 2016.","title":"Quantized convolutional neural networks for mobile devices","authors":"Wu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wu et al., 2016b] Zifeng Wu, Chunhua Shen, and Anton van den Hengel. Wider or deeper: Revisiting the resnet model for visual recognition. arXiv preprint arXiv:1611.10080, 2016.","title":"Wider or deeper: Revisiting the resnet model for visual recognition","authors":"Wu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xie et al., 2014] Wenxuan Xie, Yuxin Peng, and Jianguo Xiao. Se- mantic graph construction for weakly-supervised image parsing. In AAAI, pages 2853-2859, 2014.","title":"Semantic graph construction for weakly-supervised image parsing","authors":"Wenxuan Xie, Yuxin Peng, Jianguo Xiao","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2015] Zichao Yang, Marcin Moczulski, Misha Denil, Nando de Freitas, Alexander J. Smola, Le Song, and Ziyu Wang. Deep fried convnets. In ICCV, 2015.","title":"Deep fried convnets","authors":"Yang","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yu and Koltun, 2016] Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. In ICLR, 2016.","title":"Multi-scale context aggregation by dilated convolutions","authors":"Koltun ; Fisher Yu, Vladlen Yu, Koltun","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2017a] Rui Zhang, Sheng Tang, Min Lin, Jintao Li, and Shuicheng Yan. Global-residual and local-boundary refine- ment networks for rectifying scene parsing predictions. In IJCAI, 2017.","title":"Global-residual and local-boundary refinement networks for rectifying scene parsing predictions","authors":"Zhang","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/479"},{"position":34,"raw":"[Zhang et al., 2017b] Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, and Shuicheng Yan. Scale-adaptive convolutions for scene parsing. In ICCV, 2017.","title":"Scale-adaptive convolutions for scene parsing","authors":"Zhang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhao et al., 2017] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017.","title":"Pyramid scene parsing network","authors":"Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}