{"type":"ScholarlyArticle","title":"Learning Multi-level Region Consistency with Dense Multi-label Networks for Semantic Segmentation","authors":[{"name":"Tong Shen","slug":"tong-shen","given":"Tong","family":"Shen","affiliation":"School of Computer Science , The University of Adelaide , Australia"},{"name":"Guosheng Lin","slug":"guosheng-lin","given":"Guosheng","family":"Lin","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore; The University of Adelaide ."},{"name":"Chunhua Shen","slug":"chunhua-shen","given":"Chunhua","family":"Shen","affiliation":"School of Computer Science , The University of Adelaide , Australia"},{"name":"Ian Reid","slug":"ian-reid","given":"Ian","family":"Reid","affiliation":"School of Computer Science , The University of Adelaide , Australia"}],"abstract":"Semantic image segmentation is a fundamental task in image understanding. Per-pixel semantic labelling of an image benefits greatly from the ability to consider region consistency both locally and globally. However, many Fully Convolutional Network based methods do not impose such consistency, which may give rise to noisy and implausible predictions. We address this issue by proposing a dense multi-label network module that is able to encourage the region consistency at different levels. This simple but effective module can be easily integrated into any semantic segmentation systems. With comprehensive experiments, we show that the dense multi-label can successfully remove the implausible labels and clear the confusion so as to boost the performance of semantic segmentation systems.","doi":"10.24963/ijcai.2017/377","url":"https://ijcai.org/proceedings/2017/377","pdf":"https://ijcai.org/proceedings/2017/0377.pdf","markdown":"https://ijcai.org/proceedings/2017/377.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/377","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":377,"pages":{"first":2708,"last":2714},"keywords":["Machine Learning: Neural Networks","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Arnab et al., 2015] Anurag Arnab, Sadeep Jayasumana, Shuai Zheng, and Philip Torr. Higher Order Conditional Random Fields in Deep Neural Networks. Arxiv, page 10, 2015.","title":"Higher Order Conditional Random Fields in Deep Neural Networks","authors":"Anurag Arnab, Sadeep Jayasumana, Shuai Zheng, Philip Torr","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2015] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. In ICLR, 2015.","title":"Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2016] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. DeepLab: Semantic Image Segmentation with Deep Con- volutional Nets, Atrous Convolution, and Fully Connected CRFs. CoRR, abs/1606.0, 2016.","title":"DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dai et al., 2015] Jifeng Dai, Kaiming He, and Jian Sun. [M] BoxSup: Exploiting Bounding Boxes to Supervise Convo- lutional Networks for Semantic Segmentation. In ICCV, pages 1635-1643, 2015.","title":"M] BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation","authors":"Jifeng Dai, Kaiming He, Jian Sun","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher K I Williams, John Winn, and Andrew Zisser- man. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2):303-338, 2010.","title":"The pascal visual object classes (VOC) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Farabet et al., 2013] Clément Farabet, Camille Couprie, Laurent Najman, and Yann Lecun. Learning Hierarchical Features for Scene Labeling. TPAMI, 35(8):1915-1929, 2013.","title":"Learning Hierarchical Features for Scene Labeling","authors":"Clément Farabet, Camille Couprie, Laurent Najman, Yann Lecun","year":"2013","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Girshick et al., 2014] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In CVPR, pages 580-587, 2014.","title":"Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo and Gu, 2011] Yuhong Guo and Suicheng Gu. Multi- label classification using conditional dependency net- works. In IJCAI, pages 1300-1305, 2011.","title":"Multilabel classification using conditional dependency networks","authors":"Gu ; ] Yuhong Guo, Suicheng Guo, Gu","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hariharan et al., 2014] Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. Simultaneous Detec- tion and Segmentation. ECCV, pages 297-312, 2014.","title":"Simultaneous Detection and Segmentation","authors":"Bharath Hariharan, Pablo Arbeláez, Ross Girshick, Jitendra Malik","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, SHaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In CVPR, pages 770-778, 2016.","title":"Deep Residual Learning for Image Recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jiang, 2016] Wang Jiang. CNN-RNN : A Unified Frame- work for Multi-label Image Classification. CVPR, 2016.","title":"CNN-RNN : A Unified Framework for Multi-label Image Classification","authors":"Wang Jiang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kendall et al., 2015] Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla. Bayesian SegNet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv:1511.02680v1 [cs.CV], 2015.","title":"Bayesian SegNet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding","authors":"Alex Kendall, Vijay Badrinarayanan, Roberto Cipolla","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, Lubomir D Bourdev, Ross B Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft {COCO:} Common Ob- jects in Context. {arXiv}:1405.0312, pages 740-755, 2014.","title":"Microsoft {COCO:} Common Objects in Context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, D Lubomir, Ross B Bourdev, James Girshick, Pietro Hays, Deva Perona, Piotr Ramanan, C Dollár, Zitnick Lawrence","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2015] Guosheng Lin, Chunhua Shen, Anton van dan Hengel, and Ian Reid. Efficient piecewise train- ing of deep structured models for semantic segmentation. pages 1-13, apr 2015.","title":"Efficient piecewise training of deep structured models for semantic segmentation","authors":"Guosheng Lin, Chunhua Shen, Anton Van Dan, Ian Hengel, Reid","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2016] Guosheng Lin, Chunhua Shen, Anton Van Den Hengel, and Ian Reid. Exploring Context with Deep Structured models for Semantic Segmentation. Arxiv 2016, pages 1-14, 2016.","title":"Exploring Context with Deep Structured models for Semantic Segmentation","authors":"Guosheng Lin, Chunhua Shen, Anton Van Den, Ian Hengel, Reid","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2015] Wei Liu, Andrew Rabinovich, and Alexan- der C. Berg. ParseNet: Looking Wider to See Better. arXiv preprint: arXiv:1506.04579, pages 1-11, 2015.","title":"ParseNet: Looking Wider to See Better","authors":"Wei Liu, Andrew Rabinovich, Alexander C Berg","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully Convolutional Networks for Seman- tic Segmentation. CVPR, pages 3431-3440, 2015.","title":"Fully Convolutional Networks for Semantic Segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mottaghi et al., 2010] Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-gyu Cho, Seong-whan Lee, Raquel Ur- tasun, and Alan Yuille. The Role of Context for Object Detection and Semantic Segmentation in the Wild. 2010.","title":"The Role of Context for Object Detection and Semantic Segmentation in the Wild","authors":"Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Raquel Urtasun, Alan Yuille","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Silberman et al., 2012] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from RGBD images. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformat- ics), 7576 LNCS(PART 5):746-760, 2012.","title":"Indoor segmentation and support inference from RGBD images","authors":"Nathan Silberman, Derek Hoiem, Pushmeet Kohli, Rob Fergus","year":"2012","venue":"Lecture Notes in Computer Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. pages 1-14, 2015.","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2015] Shuran Song, Samuel P. Lichtenberg, and Jianxiong Xiao. SUN RGB-D: A RGB-D scene under- standing benchmark suite. CVPR, pages 567-576, 2015.","title":"SUN RGB-D: A RGB-D scene understanding benchmark suite","authors":"Song","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"Szegedy et al., ] Christian Szegedy, Sergey Ioffe, and Vin- cent Vanhoucke. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning.","title":"Christian Szegedy, Sergey Ioffe, and Vincent Vanhoucke. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning","authors":"Szegedy","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wei et al., 2016] Yunchao Wei, Wei Xia, Min Lin, Junshi Huang, Bingbing Ni, Jian Dong, Shuicheng Yan, and Yao Zhao. HCP: A Flexible CNN Framework for Multi-Label Image Classification. TPAMI, 38(2):1901-1907, 2016.","title":"HCP: A Flexible CNN Framework for Multi-Label Image Classification","authors":"Yunchao Wei, Wei Xia, Min Lin, Junshi Huang, Bingbing Ni, Jian Dong, Shuicheng Yan, Yao Zhao","year":"2016","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2016] Zifeng Wu, Chunhua Shen, and Anton Van Den Hengel. Bridging Category-level and Instance- level Semantic Image Segmentation. 2016.","title":"Bridging Category-level and Instancelevel Semantic Image Segmentation","authors":"Wu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xue et al., 2011] Xiangyang Xue, Wei Zhang, Jie Zhang, Bin Wu, Jianping Fan, and Yao Lu. Correlative multi-label multi-instance image annotation. ICCV, pages 651-658, 2011.","title":"Correlative multi-label multi-instance image annotation","authors":"Xiangyang Xue, Wei Zhang, Jie Zhang, Bin Wu, Jianping Fan, Yao Lu","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yadollahpour et al., 2013] Payman Yadollahpour, Dhruv Batra, and Gregory Shakhnarovich. Discriminative re- ranking of diverse segmentations. CVPR, pages 1923- 1930, 2013.","title":"Discriminative reranking of diverse segmentations","authors":"Payman Yadollahpour, Dhruv Batra, Gregory Shakhnarovich","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2016] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Se- mantic Understanding of Scenes through the ADE20K Dataset. arXiv, 2016.","title":"Semantic Understanding of Scenes through the ADE20K Dataset","authors":"Zhou","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}