{"type":"ScholarlyArticle","title":"Boundary Perception Guidance: A Scribble-Supervised Semantic Segmentation Approach","authors":[{"name":"Bin Wang","slug":"bin-wang","given":"Bin","family":"Wang","affiliation":"Key Lab of Intelligent Information Processing , Institute of Computing Technology , Chinese Academy of Sciences , China; University of the Chinese Academy of Sciences , China"},{"name":"Guojun Qi","slug":"guojun-qi","given":"Guojun","family":"Qi","affiliation":"Huawei Cloud , China"},{"name":"Sheng Tang","slug":"sheng-tang","given":"Sheng","family":"Tang","affiliation":"Key Lab of Intelligent Information Processing , Institute of Computing Technology , Chinese Academy of Sciences , China"},{"name":"Tianzhu Zhang","slug":"tianzhu-zhang","given":"Tianzhu","family":"Zhang","affiliation":"University of Science and Technology of China , China"},{"name":"Yunchao Wei","slug":"yunchao-wei","given":"Yunchao","family":"Wei","affiliation":"University of Technology Sydney , Australia"},{"name":"Linghui Li","slug":"linghui-li","given":"Linghui","family":"Li","affiliation":"Key Lab of Intelligent Information Processing , Institute of Computing Technology , Chinese Academy of Sciences , China; University of the Chinese Academy of Sciences , China"},{"name":"Yongdong Zhang","slug":"yongdong-zhang","given":"Yongdong","family":"Zhang","affiliation":"Key Lab of Intelligent Information Processing , Institute of Computing Technology , Chinese Academy of Sciences , China"}],"abstract":"Semantic segmentation suffers from the fact that densely annotated masks are expensive to obtain. To tackle this problem, we aim at learning to segment by only leveraging scribbles that are much easier to collect for supervision. To fully explore the limited pixel-level annotations from scribbles, we present a novel Boundary Perception Guidance (BPG) approach, which consists of two basic components, i.e., prediction refinement and boundary regression. Specifically, the prediction refinement progressively makes a better segmentation by adopting an iterative upsampling and a semantic feature  enhancement strategy. In the boundary regression, we employ class-agnostic edge maps for supervision to effectively guide the segmentation network in localizing the boundaries between different semantic regions, leading to producing finer-grained representation of feature maps for semantic segmentation. The experiment results on the PASCAL VOC 2012 demonstrate the proposed BPG achieves mIoU of 73.2% without fully connected Conditional Random Field (CRF) and 76.0% with CRF, setting up the new state-of-the-art in literature.","doi":"10.24963/ijcai.2019/508","url":"https://ijcai.org/proceedings/2019/508","pdf":"https://ijcai.org/proceedings/2019/0508.pdf","markdown":"https://ijcai.org/proceedings/2019/508.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/508","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":508,"pages":{"first":3663,"last":3669},"keywords":["Computer Vision: Computer Vision","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Bearman et al., 2016] Amy Bearman, Olga Russakovsky, Vittorio Ferrari, and Li Fei-Fei. What's the point: Seman- tic segmentation with point supervision. In ECCV, 2016.","title":"What's the point: Semantic segmentation with point supervision","authors":"Amy Bearman, Olga Russakovsky, Vittorio Ferrari, Li Fei-Fei","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2018] Jinzheng Cai, Youbao Tang, Le Lu, Adam P Harrison, Ke Yan, Jing Xiao, Lin Yang, and Ronald M Summers. Accurate weakly-supervised deep lesion segmentation using large-scale clinical annotations: Slice-propagated 3d mask generation from 2d recist. In MICCAI, 2018.","title":"Accurate weakly-supervised deep lesion segmentation using large-scale clinical annotations: Slice-propagated 3d mask generation from 2d recist","authors":"Cai","year":"2018","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2015] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Se- mantic image segmentation with deep convolutional nets and fully connected crfs. In ICLR, 2015.","title":"Semantic image segmentation with deep convolutional nets and fully connected crfs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018a] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convo- lutional nets, atrous convolution, and fully connected crfs. TPAMI, 40(4):834-848, 2018.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Chen","year":"2018","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2018b] Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, and Shih-Fu Chang. Zero-shot visual recog- nition using semantics-preserving adversarial embedding networks. In CVPR, 2018.","title":"Zero-shot visual recognition using semantics-preserving adversarial embedding networks","authors":"Chen","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dai et al., 2015] Jifeng Dai, Kaiming He, and Jian Sun. Boxsup: Exploiting bounding boxes to supervise convolu- tional networks for semantic segmentation. In ICCV, 2015.","title":"Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation","authors":"Jifeng Dai, Kaiming He, Jian Sun","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. IJCV, 88(2):303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hariharan et al., 2011] Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. 2011.","title":"Semantic contours from inverse detectors","authors":"Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, Jitendra Malik","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khoreva et al., 2017] Anna Khoreva, Rodrigo Benenson, Jan Hosang, Matthias Hein, and Bernt Schiele. Simple does it: Weakly supervised instance and semantic segmen- tation. In CVPR, 2017.","title":"Simple does it: Weakly supervised instance and semantic segmentation","authors":"Anna Khoreva, Rodrigo Benenson, Jan Hosang, Matthias Hein, Bernt Schiele","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In ECCV, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2016] Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He, and Jian Sun. Scribblesup: Scribble-supervised convo- lutional networks for semantic segmentation. In CVPR, 2016.","title":"Scribblesup: Scribble-supervised convolutional networks for semantic segmentation","authors":"Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2016] Wei Liu, Dragomir Anguelov, Dumitru Er- han, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In ECCV, 2016.","title":"Ssd: Single shot multibox detector","authors":"Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C Berg","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In CVPR, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Long","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pinheiro and Collobert, 2015] Pedro O Pinheiro and Ronan Collobert. From image-level to pixel-level labeling with convolutional networks. In CVPR, 2015.","title":"From image-level to pixel-level labeling with convolutional networks","authors":"O Pedro, Ronan Pinheiro, Collobert","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pont-Tuset et al., 2017] Jordi Pont-Tuset, Pablo Arbelaez, Jonathan T Barron, Ferran Marques, and Jitendra Malik. Multiscale combinatorial grouping for image segmenta- tion and object proposal generation. TPAMI, 39(1):128- 140, 2017.","title":"Multiscale combinatorial grouping for image segmentation and object proposal generation","authors":"Jordi Pont-Tuset, Pablo Arbelaez, Jonathan T Barron, Ferran Marques, Jitendra Malik","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Redmon et al., 2016] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Uni- fied, real-time object detection. In CVPR, 2016.","title":"You only look once: Unified, real-time object detection","authors":"Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Saleh et al., 2016] Fatemehsadat Saleh, Moham- mad Sadegh Aliakbarian, Mathieu Salzmann, Lars Petersson, Stephen Gould, and Jose M Alvarez. Built-in foreground/background prior for weakly-supervised semantic segmentation. In ECCV, 2016.","title":"Built-in foreground/background prior for weakly-supervised semantic segmentation","authors":"Saleh","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabi- novich. Going deeper with convolutions. In CVPR, 2015.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tang et al., 2018a] Meng Tang, Abdelaziz Djelouah, Fed- erico Perazzi, Yuri Boykov, and Christopher Schroers. Normalized cut loss for weakly-supervised cnn segmen- tation. In CVPR, 2018.","title":"Normalized cut loss for weakly-supervised cnn segmentation","authors":"Tang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tang et al., 2018b] Meng Tang, Federico Perazzi, Abde- laziz Djelouah, Ismail Ben Ayed, Christopher Schroers, and Yuri Boykov. On regularized losses for weakly- supervised cnn segmentation. In ECCV, 2018.","title":"On regularized losses for weaklysupervised cnn segmentation","authors":"Tang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wei et al., 2017] Yunchao Wei, Xiaodan Liang, Yunpeng Chen, Xiaohui Shen, Ming-Ming Cheng, Jiashi Feng, Yao Zhao, and Shuicheng Yan. Stc: A simple to complex framework for weakly-supervised semantic segmentation. TPAMI, 39(11):2314-2320, 2017.","title":"Stc: A simple to complex framework for weakly-supervised semantic segmentation","authors":"Yunchao Wei, Xiaodan Liang, Yunpeng Chen, Xiaohui Shen, Ming-Ming Cheng, Jiashi Feng, Yao Zhao, Shuicheng Yan","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xie and Tu, 2015] Saining Xie and Zhuowen Tu. Holistically-nested edge detection. In ICCV, 2015.","title":"Holistically-nested edge detection","authors":"Tu ; Xie, Saining Xie, Zhuowen Tu","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2017] Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, and Shuicheng Yan. Scale-adaptive con- volutions for scene parsing. In ICCV, 2017.","title":"Scale-adaptive convolutions for scene parsing","authors":"Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, Shuicheng Yan","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2019] Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, and Shuicheng Yan. Perspective- adaptive convolutions for scene parsing. TPAMI, 2019.","title":"Perspectiveadaptive convolutions for scene parsing","authors":"Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, Shuicheng Yan","year":"2019","venue":"TPAMI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}