{"type":"ScholarlyArticle","title":"R³Net: Recurrent Residual Refinement Network for Saliency Detection","authors":[{"name":"Zijun Deng","slug":"zijun-deng","given":"Zijun","family":"Deng","affiliation":"South China University of Technology ,"},{"name":"Xiaowei Hu","slug":"xiaowei-hu","given":"Xiaowei","family":"Hu","affiliation":"The Chinese University of Hong Kong ,"},{"name":"Lei Zhu","slug":"lei-zhu","given":"Lei","family":"Zhu","affiliation":"The Hong Kong Polytechnic University ,; The Chinese University of Hong Kong ,"},{"name":"Xuemiao Xu","slug":"xuemiao-xu","given":"Xuemiao","family":"Xu","affiliation":"South China University of Technology ,"},{"name":"Jing Qin","slug":"jing-qin","given":"Jing","family":"Qin","affiliation":"The Hong Kong Polytechnic University ,"},{"name":"Guoqiang Han","slug":"guoqiang-han","given":"Guoqiang","family":"Han","affiliation":"South China University of Technology ,"},{"name":"Pheng-Ann Heng","slug":"pheng-ann-heng","given":"Pheng-Ann","family":"Heng","affiliation":"The Chinese University of Hong Kong ,; Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology , Shenzhen Institutes of Advanced Technology , Chinese Academy of Sciences , China"}],"abstract":"Saliency detection is a fundamental yet challenging task in computer vision, aiming at highlighting the most visually distinctive objects in an image. We propose a novel recurrent residual refinement network (R^3Net) equipped with residual refinement blocks (RRBs) to more accurately detect salient regions of an input image. Our RRBs learn the residual between the intermediate saliency prediction and the ground truth by alternatively leveraging the low-level integrated features and the high-level integrated features of a fully convolutional network (FCN). While the low-level integrated features are capable of capturing more saliency details, the high-level integrated features can reduce non-salient regions in the intermediate prediction. Furthermore, the RRBs can obtain complementary saliency information of the intermediate prediction, and add the residual into the intermediate prediction to refine the saliency maps. We evaluate the proposed R^3Net on five widely-used saliency detection benchmarks by comparing it with 16 state-of-the-art saliency detectors. Experimental results show that our network outperforms our competitors in all the benchmark datasets.","doi":"10.24963/ijcai.2018/95","url":"https://ijcai.org/proceedings/2018/95","pdf":"https://ijcai.org/proceedings/2018/0095.pdf","markdown":"https://ijcai.org/proceedings/2018/95.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/95","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":95,"pages":{"first":684,"last":690},"keywords":["Computer Vision: Perception","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Machine Learning Applications: Applications of Supervised Learning","Machine Learning: Deep Learning","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Baisheng Lai, 2017] Xiaojin Gong Baisheng Lai. Saliency guided end-to-end learning for weakly supervised object detection. In IJCAI, 2017.","title":"Saliency guided end-to-end learning for weakly supervised object detection","authors":"Xiaojin Gong, Baisheng Lai","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/285"},{"position":2,"raw":"[Cheng et al., 2015] Ming-Ming Cheng, Niloy J Mitra, Xi- aolei Huang, Philip HS Torr, and Shi-Min Hu. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3):569- 582, 2015.","title":"Global contrast based salient region detection","authors":"Ming-Ming Cheng, J Niloy, Xiaolei Mitra, Philip Huang, Shi-Min Hs Torr, Hu","year":"2015","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpass- ing human-level performance on imagenet classification. In CVPR, 2015.","title":"Delving deep into rectifiers: Surpassing human-level performance on imagenet classification","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hong et al., 2015] Seunghoon Hong, Tackgeun You, Suha Kwak, and Bohyung Han. Online tracking by learning discriminative saliency map with convolutional neural net- work. In ICML, 2015.","title":"Online tracking by learning discriminative saliency map with convolutional neural network","authors":"Hong","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hou et al., 2017] Qibin Hou, Ming-Ming Cheng, Xiao-Wei Hu, Ali Borji, Zhuowen Tu, and Philip Torr. Deeply su- pervised salient object detection with short connections. In CVPR, 2017.","title":"Deeply supervised salient object detection with short connections","authors":"Hou","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2017] Gao Huang, Zhuang Liu, K.Q. Wein- berger, and L. van der Maaten. Densely connected con- volutional networks. In CVPR, 2017.","title":"Densely connected convolutional networks","authors":"Gao Huang, Zhuang Liu, K Q Weinberger, L Van Der Maaten","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kong et al., 2016] Tao Kong, Anbang Yao, Yurong Chen, and Fuchun Sun. Hypernet: Towards accurate region pro- posal generation and joint object detection. In CVPR, 2016.","title":"Hypernet: Towards accurate region proposal generation and joint object detection","authors":"Tao Kong, Anbang Yao, Yurong Chen, Fuchun Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Krähenbühl and Koltun, 2011] Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. In NIPS, 2011.","title":"Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials","authors":"Koltun Krähenbühl","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lee et al., 2016] Gayoung Lee, Yu-Wing Tai, and Junmo Kim. Deep saliency with encoded low level distance map and high level features. In CVPR, 2016.","title":"Deep saliency with encoded low level distance map and high level features","authors":"Lee","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li and Yu, 2015] Guanbin Li and Yizhou Yu. Visual saliency based on multiscale deep features. In CVPR, 2015.","title":"Visual saliency based on multiscale deep features","authors":"Yu ; ] Guanbin Li, Yizhou Li, Yu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Yu, 2016] Guanbin Li and Yizhou Yu. Deep contrast learning for salient object detection. In CVPR, 2016.","title":"Deep contrast learning for salient object detection","authors":"Yu ; ] Guanbin Li, Yizhou Li, Yu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2016] Xi Li, Liming Zhao, Lina Wei, Ming-Hsuan Yang, Fei Wu, Yueting Zhuang, Haibin Ling, and Jing- dong Wang. Deepsaliency: Multi-task deep neural net- work model for salient object detection. IEEE Transac- tions on Image Processing, 25(8):3919-3930, 2016.","title":"Deepsaliency: Multi-task deep neural network model for salient object detection","authors":"Xi Li, Liming Zhao, Lina Wei, Ming-Hsuan Yang, Fei Wu, Yueting Zhuang, Haibin Ling, Jingdong Wang","year":"2016","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu and Han, 2016] Nian Liu and Junwei Han. Dhsnet: Deep hierarchical saliency network for salient object de- tection. In CVPR, 2016.","title":"Dhsnet: Deep hierarchical saliency network for salient object detection","authors":"Nian Liu, Junwei Han","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2011] Tie Liu, Zejian Yuan, Jian Sun, Jingdong Wang, Nanning Zheng, Xiaoou Tang, and Heung-Yeung Shum. Learning to detect a salient object. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 33(2):353-367, 2011.","title":"Learning to detect a salient object","authors":"Liu","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2015] Wei Liu, Andrew Rabinovich, and Alexan- der C Berg. Parsenet: Looking wider to see better. arXiv preprint arXiv:1506.04579, 2015.","title":"Parsenet: Looking wider to see better","authors":"Wei Liu, Andrew Rabinovich, Alexander C Berg","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In CVPR, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Long","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Luo et al., 2017] Zhiming Luo, Akshaya Mishra, Andrew Achkar, Justin Eichel, Shaozi Li, and Pierre-Marc Jodoin. Non-local deep features for salient object detection. In CVPR, 2017.","title":"Non-local deep features for salient object detection","authors":"Zhiming Luo, Akshaya Mishra, Andrew Achkar, Justin Eichel, Shaozi Li, Pierre-Marc Jodoin","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Qin et al., 2015] Yao Qin, Huchuan Lu, Yiqun Xu, and He Wang. Saliency detection via cellular automata. In CVPR, 2015.","title":"Saliency detection via cellular automata","authors":"Huchuan Yao Qin, Yiqun Lu, He Xu, Wang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shen and Liu, 2017] Wei Shen and Rujie Liu. Learning residual images for face attribute manipulation. In CVPR, 2017.","title":"Learning residual images for face attribute manipulation","authors":"Wei Shen, Rujie Liu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2015] Lijun Wang, Huchuan Lu, Xiang Ruan, and Ming-Hsuan Yang. Deep networks for saliency de- tection via local estimation and global search. In CVPR, 2015.","title":"Deep networks for saliency detection via local estimation and global search","authors":"Lijun Wang, Huchuan Lu, Xiang Ruan, Ming-Hsuan Yang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2016] Linzhao Wang, Lijun Wang, Huchuan Lu, Pingping Zhang, and Xiang Ruan. Saliency detec- tion with recurrent fully convolutional networks. In ECCV, 2016.","title":"Saliency detection with recurrent fully convolutional networks","authors":"Wang","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Tiantian Wang, Ali Borji, Lihe Zhang, Pingping Zhang, and Huchuan Lu. A stagewise refinement model for detecting salient objects in images. In CVPR, 2017.","title":"A stagewise refinement model for detecting salient objects in images","authors":"Tiantian Wang, Ali Borji, Lihe Zhang, Pingping Zhang, Huchuan Lu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie and Tu, 2015] Saining Xie and Zhuowen Tu. Holistically-nested edge detection. In ICCV, 2015.","title":"Holistically-nested edge detection","authors":"Tu ; Xie, Saining Xie, Zhuowen Tu","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2017] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In CVPR, 2017.","title":"Aggregated residual transformations for deep neural networks","authors":"Xie","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yan et al., 2013] Qiong Yan, Li Xu, Jianping Shi, and Jiaya Jia. Hierarchical saliency detection. In CVPR, 2013.","title":"Hierarchical saliency detection","authors":"Qiong Yan, Li Xu, Jianping Shi, Jiaya Jia","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2013] Chuan Yang, Lihe Zhang, Huchuan Lu, Xiang Ruan, and Ming-Hsuan Yang. Saliency detection via graph-based manifold ranking. In CVPR, 2013.","title":"Saliency detection via graph-based manifold ranking","authors":"Chuan Yang, Lihe Zhang, Huchuan Lu, Xiang Ruan, Ming-Hsuan Yang","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2017a] Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, and Xiang Ruan. Amulet: Aggregating multi-level convolutional features for salient object detection. In ICCV, 2017.","title":"Amulet: Aggregating multi-level convolutional features for salient object detection","authors":"Zhang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2017b] Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, and Baocai Yin. Learning uncertain convolutional features for accurate saliency detection. In ICCV, 2017.","title":"Learning uncertain convolutional features for accurate saliency detection","authors":"Zhang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhao et al., 2015] Rui Zhao, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang. Saliency detection by multi-context deep learning. In CVPR, 2015.","title":"Saliency detection by multi-context deep learning","authors":"Rui Zhao, Wanli Ouyang, Hongsheng Li, Xiaogang Wang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhu et al., 2014] Wangjiang Zhu, Shuang Liang, Yichen Wei, and Jian Sun. Saliency optimization from robust background detection. In CVPR, 2014.","title":"Saliency optimization from robust background detection","authors":"Wangjiang Zhu, Shuang Liang, Yichen Wei, Jian Sun","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}