{"type":"ScholarlyArticle","title":"Weakly Supervised RBM for Semantic Segmentation","authors":[{"name":"Yong Li","slug":"yong-li","given":"Yong","family":"Li","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Jing Liu","slug":"jing-liu","given":"Jing","family":"Liu","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Yuhang Wang","slug":"yuhang-wang","given":"Yuhang","family":"Wang","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Hanqing Lu","slug":"hanqing-lu","given":"Hanqing","family":"Lu","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Songde Ma","slug":"songde-ma","given":"Songde","family":"Ma","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"}],"abstract":"In this paper, we propose a weakly supervised Restricted Boltzmann Machines (WRBM) approach to deal with the task of semantic segmentation with only image-level labels available. In WRBM, its hidden nodes are divided into multiple blocks, and each block corresponds to a specific label. Accordingly, semantic segmentation can be directly modeled by learning the mapping from visible layer to the hidden layer of WRBM. Specifically, based on the standard RBM, we import another two terms to make full use of image-level labels and alleviate the effect of noisy labels. First, we expect the hidden response of each superpixel is suppressed on the labels outside its parent image-level label set, and a non-image-level label suppression term is formulated to implicitly import the image-level labels as weak supervision. Second, semantic graph propagation is employed to exploit the cooccurrence between visually similar regions and labels. Besides, we deal with the problems of label imbalance and diverse backgrounds by adapting the block size to the label frequency and appending hidden response blocks corresponding to backgrounds respectively. Extensive experiments on two real-world datasets demonstrate the good performance of our approach compared with some state-of-the-art methods.","doi":null,"url":"https://ijcai.org/proceedings/2015/268","pdf":"https://ijcai.org/proceedings/2015/0268.pdf","markdown":"https://ijcai.org/proceedings/2015/268.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/268","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Robotics and Vision","track":"Main Track","number":268,"pages":{"first":1888,"last":1893},"keywords":[],"videos":[],"references":[{"position":1,"raw":"[Achanta et al., 2012] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Ssstrunk. Slic superpixels compared to state-of-the-art superpixel methods. TPAMI, 34(11):2274-2282, 2012.","title":"Slic superpixels compared to state-of-the-art superpixel methods","authors":"R Achanta, A Shaji, K Smith, A Lucchi, P Fua, S Ssstrunk","year":"2012","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alexe et al., 2010] B. Alexe, T. Deselaers, and V. Ferrari. What is an object? In CVPR, pages 73-80, 2010.","title":"What is an object?","authors":"B Alexe, T Deselaers, V Ferrari","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio, 2009] Yoshua Bengio. Learning deep architectures for ai. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.","title":"Learning deep architectures for ai","authors":"Yoshua Bengio","year":"2009","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, ChristopherK.I. Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. IJCV, 88(2):303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, Christopherk I Williams, John Winn, Andrew Zisserman","year":"2010","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Farabet et al., 2013] C. Farabet, C. Couprie, L. Najman, and Y. LeCun. Learning hierarchical features for scene label- ing. TPAMI, 35(8):1915-1929, 2013.","title":"Learning hierarchical features for scene labeling","authors":"C Farabet, C Couprie, L Najman, Y Lecun","year":"2013","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hinton and Salakhutdinov, 2006] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, 2006.","title":"Reducing the dimensionality of data with neural networks","authors":"G E Hinton, R R Salakhutdinov","year":"2006","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hinton, 2002] Geoffrey E. Hinton. Training products of ex- perts by minimizing contrastive divergence. Neural Com- put., 14(8):1771-1800, 2002.","title":"Training products of experts by minimizing contrastive divergence","authors":"Geoffrey E Hinton","year":"2002","venue":"Neural Comput","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ladicky et al., 2009] L. Ladicky, C. Russell, P. Kohli, and P.H.S. Torr. Associative hierarchical crfs for object class image segmentation. In ICCV, 2009.","title":"Associative hierarchical crfs for object class image segmentation","authors":"L Ladicky, C Russell, P Kohli, P H S Torr","year":"2009","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ladicky et al., 2010] Lubor Ladicky, Chris Russell, Push- meet Kohli, and PhilipH.S. Torr. Graph cut based infer- ence with co-occurrence statistics. In ECCV, volume 6315, pages 239-253. 2010.","title":"Graph cut based inference with co-occurrence statistics","authors":"Lubor Ladicky, Chris Russell, Pushmeet Kohli, Philiph S Torr","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2009a] Ce Liu, J. Yuen, and A. Torralba. Non- parametric scene parsing: Label transfer via dense scene alignment. In CVPR, 2009.","title":"Nonparametric scene parsing: Label transfer via dense scene alignment","authors":"Liu","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2009b] Xiaobai Liu, Bin Cheng, Shuicheng Yan, Jinhui Tang, Tat Seng Chua, and Hai Jin. Label to region by bi-layer sparsity priors. In MM, 2009.","title":"Label to region by bi-layer sparsity priors","authors":"Liu","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2011] Ce Liu, J. Yuen, and A. Torralba. Non- parametric scene parsing via label transfer. TPAMI, 33(12):2368-2382, 2011.","title":"Nonparametric scene parsing via label transfer","authors":"Ce Liu, J Yuen, A Torralba","year":"2011","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2012] Si Liu, Shuicheng Yan, Tianzhu Zhang, Changsheng Xu, Jing Liu, and Hanqing Lu. Weakly super- vised graph propagation towards collective image parsing. Multimedia, IEEE Transactions on, 14(2):361-373, 2012.","title":"Weakly supervised graph propagation towards collective image parsing","authors":"Si Liu, Shuicheng Yan, Tianzhu Zhang, Changsheng Xu, Jing Liu, Hanqing Lu","year":"2012","venue":"Multimedia, IEEE Transactions on","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2013] Yang Liu, Jing Liu, Zechao Li, Jinhui Tang, and Hanqing Lu. Weakly-supervised dual cluster- ing for image semantic segmentation. In CVPR, 2013.","title":"Weakly-supervised dual clustering for image semantic segmentation","authors":"Yang Liu, Jing Liu, Zechao Li, Jinhui Tang, Hanqing Lu","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lowe, 2004] David G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91-110, 2004.","title":"Distinctive image features from scale-invariant keypoints","authors":"G David, Lowe","year":"2004","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Myeong and Lee, 2013] Heesoo Myeong and Kyoung Mu Lee. Tensor-based high-order semantic relation transfer for semantic scene segmentation. In CVPR, 2013.","title":"Tensor-based high-order semantic relation transfer for semantic scene segmentation","authors":"Lee ; Myeong, Heesoo Myeong, Kyoung Mu, Lee","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rother et al., 2004] Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. \"grabcut\" -interactive foreground ex- traction using iterated graph cuts. ACM Trans. Graph., 23(3):309-314, 2004.","title":"grabcut\" -interactive foreground extraction using iterated graph cuts","authors":"Rother","year":"2004","venue":"ACM Trans. Graph","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Russell et al., 2008] BryanC. Russell, Antonio Torralba, KevinP. Murphy, and WilliamT. Freeman. Labelme: a database and web-based tool for image annotation. IJCV, 77(1), 2008.","title":"Labelme: a database and web-based tool for image annotation","authors":"Russell","year":"2008","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shotton et al., 2006] Jamie Shotton, John Winn, Carsten Rother, and Antonio Criminisi. Textonboost: Joint appear- ance, shape and context modeling for multi-class object recognition and segmentation. In ECCV, 2006.","title":"Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation","authors":"Jamie Shotton, John Winn, Carsten Rother, Antonio Criminisi","year":"2006","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shotton et al., 2008] J. Shotton, M. Johnson, and R. Cipolla. Semantic texton forests for image cate- gorization and segmentation. In CVPR, pages 1-8, 2008.","title":"Semantic texton forests for image categorization and segmentation","authors":"J Shotton, M Johnson, R Cipolla","year":"2008","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shotton et al., 2009] Jamie Shotton, John Winn, Carsten Rother, and Antonio Criminisi. Textonboost for image un- derstanding: Multi-class object recognition and segmenta- tion by jointly modeling texture, layout, and context. IJCV, 81(1):2-23, 2009.","title":"Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context","authors":"Jamie Shotton, John Winn, Carsten Rother, Antonio Criminisi","year":"2009","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tighe and Lazebnik, 2010] Joseph Tighe and Svetlana Lazebnik. Superparsing: Scalable nonparametric image parsing with superpixels. In ECCV, 2010.","title":"Superparsing: Scalable nonparametric image parsing with superpixels","authors":"Lazebnik Tighe, Joseph Tighe, Svetlana Lazebnik","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Verbeek and Triggs, 2007] J. Verbeek and B. Triggs. Re- gion classification with markov field aspect models. In CVPR, 2007.","title":"Region classification with markov field aspect models","authors":"J Verbeek, B Triggs","year":"2007","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vezhnevets and Buhmann, 2010] A. Vezhnevets and J.M. Buhmann. Towards weakly supervised semantic segmen- tation by means of multiple instance and multitask learn- ing. In CVPR, 2010.","title":"Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning","authors":"A Vezhnevets, J M Buhmann","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vezhnevets et al., 2011] A. Vezhnevets, V. Ferrari, and J.M. Buhmann. Weakly supervised semantic segmentation with a multi-image model. In ICCV, 2011.","title":"Weakly supervised semantic segmentation with a multi-image model","authors":"A Vezhnevets, V Ferrari, J M Buhmann","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vezhnevets et al., 2012] A. Vezhnevets, V. Ferrari, and J.M. Buhmann. Weakly supervised structured output learning for semantic segmentation. In CVPR, 2012.","title":"Weakly supervised structured output learning for semantic segmentation","authors":"A Vezhnevets, V Ferrari, J M Buhmann","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xie et al., 2014] Wenxuan Xie, Yuxin Peng, and Jianguo Xiao. Semantic graph construction for weakly-supervised image parsing. In AAAI, 2014.","title":"Semantic graph construction for weakly-supervised image parsing","authors":"Wenxuan Xie, Yuxin Peng, Jianguo Xiao","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2012] Dengsheng Zhang, M.Monirul Islam, Guojun Lu, and Ishrat Jahan Sumana. Rotation invariant curvelet features for region based image retrieval. IJCV, 98(2):187-201, 2012.","title":"Rotation invariant curvelet features for region based image retrieval","authors":"Dengsheng Zhang, M Monirul Islam, Guojun Lu, Ishrat Jahan Sumana","year":"2012","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2013] Ke Zhang, Wei Zhang, Yingbin Zheng, and Xiangyang Xue. Sparse reconstruction for weakly su- pervised semantic segmentation. In IJCAI, 2013.","title":"Sparse reconstruction for weakly supervised semantic segmentation","authors":"Ke Zhang, Wei Zhang, Yingbin Zheng, Xiangyang Xue","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/279"},{"position":30,"raw":"[Zhang et al., 2014] Ke Zhang, Wei Zhang, Sheng Zeng, and Xiangyang Xue. Semantic segmentation using multiple graphs with block-diagonal constraints. In AAAI, 2014.","title":"Semantic segmentation using multiple graphs with block-diagonal constraints","authors":"Ke Zhang, Wei Zhang, Sheng Zeng, Xiangyang Xue","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}