{"type":"ScholarlyArticle","title":"Bottom-up and Top-down: Bidirectional Additive Net for Edge Detection","authors":[{"name":"Lianli Gao","slug":"lianli-gao","given":"Lianli","family":"Gao","affiliation":"Center for Future Media and School of Computer Science and Technology , University of Electronic Science and Technology of China"},{"name":"Zhilong Zhou","slug":"zhilong-zhou","given":"Zhilong","family":"Zhou","affiliation":"Center for Future Media and School of Computer Science and Technology , University of Electronic Science and Technology of China"},{"name":"Heng Tao Shen","slug":"heng-tao-shen","given":"Heng Tao","family":"Shen","affiliation":"Center for Future Media and School of Computer Science and Technology , University of Electronic Science and Technology of China"},{"name":"Jingkuan Song","slug":"jingkuan-song","given":"Jingkuan","family":"Song","affiliation":"Center for Future Media and School of Computer Science and Technology , University of Electronic Science and Technology of China"}],"abstract":"Image edge detection is considered as a cornerstone task in computer vision. Due to the nature of hierarchical representations learned in CNN, it is intuitive to design side networks utilizing the richer convolutional features to improve the edge detection. However, there is no consensus way to integrate the hierarchical information. \r\nIn this paper, we propose an effective and end-to-end framework, named Bidirectional Additive Net (BAN), for image edge detection. In the proposed framework, we focus on two main problems: 1) how to design a universal network for incorporating hierarchical information sufficiently; and 2) how to achieve effective information flow between different stages and gradually improve the edge map stage by stage. \r\nTo tackle these problems, we design a consecutive bottom-up and top-down architecture, where a bottom-up branch can gradually remove detailed or sharp boundaries to enable accurate edge detection and a top-down branch offers a chance of error-correcting by revisiting the low-level features that contain rich textual and spatial information.\r\nAnd attended additive module (AAM) is designed to cumulatively refine edges by selecting pivotal features in each stage.\r\nExperimental results show that our proposed methods can improve the edge detection performance to new records and achieve state-of-the-art results on two public benchmarks: BSDS500 and NYUDv2.","doi":"10.24963/ijcai.2020/83","url":"https://ijcai.org/proceedings/2020/83","pdf":"https://ijcai.org/proceedings/2020/0083.pdf","markdown":"https://ijcai.org/proceedings/2020/83.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/83","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":83,"pages":{"first":594,"last":600},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Perception"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24974/file_24974.m3u8","length_sec":287,"provider":"hls"}],"references":[{"position":1,"raw":"[Arbelaez et al., 2011] Pablo Arbelaez, Michael Maire, Charless C. Fowlkes, and Jitendra Malik. Contour detec- tion and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 33(5):898-916, 2011.","title":"Contour detection and hierarchical image segmentation","authors":"Arbelaez","year":"2011","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bertasius et al., 2015] Gedas Bertasius, Jianbo Shi, and Lorenzo Torresani. Deepedge: A multi-scale bifurcated deep network for top-down contour detection. In CVPR, pages 4380-4389, 2015.","title":"Deepedge: A multi-scale bifurcated deep network for top-down contour detection","authors":"Gedas Bertasius, Jianbo Shi, Lorenzo Torresani","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Canny, 1986] John F. Canny. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell., 8(6):679-698, 1986.","title":"A computational approach to edge detection","authors":"John F Canny","year":"1986","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2016] Liang-Chieh Chen, Jonathan T. Barron, George Papandreou, Kevin Murphy, and Alan L. Yuille. Semantic image segmentation with task-specific edge de- tection using cnns and a discriminatively trained domain transform. In CVPR, pages 4545-4554, 2016.","title":"Semantic image segmentation with task-specific edge detection using cnns and a discriminatively trained domain transform","authors":"Liang-Chieh Chen, Jonathan T Barron, George Papandreou, Kevin Murphy, Alan L Yuille","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2018] Ruoxi Deng, Chunhua Shen, Shengjun Liu, Huibing Wang, and Xinru Liu. Learning to predict crisp boundaries. In ECCV, pages 570-586, 2018.","title":"Learning to predict crisp boundaries","authors":"Deng","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dollár and Zitnick, 2015] Piotr Dollár and C. Lawrence Zitnick. Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell., 37(8):1558-1570, 2015.","title":"Fast edge detection using structured forests","authors":"Piotr Dollár, C Lawrence Zitnick","year":"2015","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao et al., 2019] Lianli Gao, Xiangpeng Li, Jingkuan Song, and Heng Tao Shen. Hierarchical lstms with adaptive at- tention for visual captioning. IEEE transactions on pattern analysis and machine intelligence, 2019.","title":"Hierarchical lstms with adaptive attention for visual captioning","authors":"Gao","year":"2019","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2019] Yuyu Guo, Lianli Gao, Jingkuan Song, Peng Wang, Wuyuan Xie, and Heng Tao Shen. Adaptive multi-path aggregation for human densepose estimation in the wild. In ACM Multimedia, pages 356-364, 2019.","title":"Adaptive multi-path aggregation for human densepose estimation in the wild","authors":"Yuyu Guo, Lianli Gao, Jingkuan Song, Peng Wang, Wuyuan Xie, Heng Tao Shen","year":"2019","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gupta et al., 2013] Saurabh Gupta, Pablo Arbelaez, and Ji- tendra Malik. Perceptual organization and recognition of indoor scenes from RGB-D images. In CVPR, pages 564- 571, 2013.","title":"Perceptual organization and recognition of indoor scenes from RGB-D images","authors":"Saurabh Gupta, Pablo Arbelaez, Jitendra Malik","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gupta et al., 2014] Saurabh Gupta, Ross B. Girshick, Pablo Andrés Arbeláez, and Jitendra Malik. Learning rich features from RGB-D images for object detection and seg- mentation. In ECCV, pages 345-360, 2014.","title":"Learning rich features from RGB-D images for object detection and segmentation","authors":"Saurabh Gupta, Ross B Girshick, Pablo Andrés Arbeláez, Jitendra Malik","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hallman and Fowlkes, 2015] Sam Hallman and Charless C. Fowlkes. Oriented edge forests for boundary detection. In CVPR, pages 1732-1740, 2015.","title":"Oriented edge forests for boundary detection","authors":"Sam Hallman, Charless C Fowlkes","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2019] Jianzhong He, Shiliang Zhang, Ming Yang, Yanhu Shan, and Tiejun Huang. Bi-directional cascade network for perceptual edge detection. In CVPR, pages 3828-3837, 2019.","title":"Bi-directional cascade network for perceptual edge detection","authors":"Jianzhong He, Shiliang Zhang, Ming Yang, Yanhu Shan, Tiejun Huang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2019] Yun Liu, Ming-Ming Cheng, Xiaowei Hu, Jia-Wang Bian, Le Zhang, Xiang Bai, and Jinhui Tang. Richer convolutional features for edge detection. IEEE Trans. Pattern Anal. Mach. Intell., 41(8):1939-1946, 2019.","title":"Richer convolutional features for edge detection","authors":"Yun Liu, Ming-Ming Cheng, Xiaowei Hu, Jia-Wang Bian, Le Zhang, Xiang Bai, Jinhui Tang","year":"2019","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Maninis et al., 2018] Kevis-Kokitsi Maninis, Jordi Pont- Tuset, Pablo Arbelaez, and Luc Van Gool. Convolu- tional oriented boundaries: From image segmentation to high-level tasks. IEEE Trans. Pattern Anal. Mach. Intell., 40(4):819-833, 2018.","title":"Convolutional oriented boundaries: From image segmentation to high-level tasks","authors":"Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Pablo Arbelaez, Luc Van Gool","year":"2018","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Martin et al., 2004] David R. Martin, Charless C. Fowlkes, and Jitendra Malik. Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell., 26(5):530-549, 2004.","title":"Learning to detect natural image boundaries using local brightness, color, and texture cues","authors":"David R Martin, Charless C Fowlkes, Jitendra Malik","year":"2004","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pont-Tuset et al., 2017] Jordi Pont-Tuset, Pablo Arbelaez, Jonathan T. Barron, Ferran Marqués, and Jitendra Malik. Multiscale combinatorial grouping for image segmenta- tion and object proposal generation. IEEE Trans. Pattern Anal. Mach. Intell., 39(1):128-140, 2017.","title":"Multiscale combinatorial grouping for image segmentation and object proposal generation","authors":"Pont-Tuset","year":"2017","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2015a] Fumin Shen, Chunhua Shen, Qinfeng Shi, Anton van den Hengel, Zhenmin Tang, and Heng Tao Shen. Hashing on nonlinear manifolds. IEEE Trans. Image Processing, 24(6):1839-1851, 2015.","title":"Hashing on nonlinear manifolds","authors":"Shen","year":"2015","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shen et al., 2015b] Wei Shen, Xinggang Wang, Yan Wang, Xiang Bai, and Zhijiang Zhang. Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. In CVPR, pages 3982-3991, 2015.","title":"Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection","authors":"Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Silberman et al., 2012] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from RGBD images. In ECCV, pages 746-760, 2012.","title":"Indoor segmentation and support inference from RGBD images","authors":"Nathan Silberman, Derek Hoiem, Pushmeet Kohli, Rob Fergus","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Song et al., 2018] Jingkuan Song, Zhilong Zhou, Lianli Gao, Xing Xu, and Heng Tao Shen. Cumulative nets for edge detection. In ACM MM, pages 1847-1855, 2018.","title":"Cumulative nets for edge detection","authors":"Song","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2019a] Wenguan Wang, Shuyang Zhao, Jian- bing Shen, Steven C. H. Hoi, and Ali Borji. Salient ob- ject detection with pyramid attention and salient edges. In CVPR, pages 1448-1457, 2019.","title":"Salient object detection with pyramid attention and salient edges","authors":"Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2019b] Yupei Wang, Xin Zhao, Yin Li, and Kaiqi Huang. Deep crisp boundaries: From boundaries to higher-level tasks. IEEE Trans. Image Processing, 28(3):1285-1298, 2019.","title":"Deep crisp boundaries: From boundaries to higher-level tasks","authors":"Wang","year":"2019","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xie and Tu, 2017] Saining Xie and Zhuowen Tu. Holistically-nested edge detection. International Journal of Computer Vision, 125(1-3):3-18, 2017.","title":"Saining Xie and Zhuowen Tu. Holistically-nested edge detection","authors":"Tu Xie","year":"2017","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu et al., 2017] Dan Xu, Wanli Ouyang, Xavier Alameda- Pineda, Elisa Ricci, Xiaogang Wang, and Nicu Sebe. Learning deep structured multi-scale features using attention-gated crfs for contour prediction. In NeurIPS, pages 3961-3970, 2017.","title":"Learning deep structured multi-scale features using attention-gated crfs for contour prediction","authors":"Dan Xu, Wanli Ouyang, Xavier Alameda-Pineda, Elisa Ricci, Xiaogang Wang, Nicu Sebe","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2016] Jimei Yang, Brian L. Price, Scott Cohen, Honglak Lee, and Ming-Hsuan Yang. Object contour de- tection with a fully convolutional encoder-decoder net- work. In CVPR, pages 193-202, 2016.","title":"Object contour detection with a fully convolutional encoder-decoder network","authors":"Yang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2016] Xiaofeng Zhu, Xuelong Li, Shichao Zhang, Chunhua Ju, and Xindong Wu. Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE transactions on neural networks and learning sys- tems, 28(6):1263-1275, 2016.","title":"Robust joint graph sparse coding for unsupervised spectral feature selection","authors":"Xiaofeng Zhu, Xuelong Li, Shichao Zhang, Chunhua Ju, Xindong Wu","year":"2016","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}