{"type":"ScholarlyArticle","title":"DSRN: A Deep Scale Relationship Network for Scene Text Detection","authors":[{"name":"Yuxin Wang","slug":"yuxin-wang","given":"Yuxin","family":"Wang","affiliation":"School of Information Science and Technology , University of Science and Technology of China"},{"name":"Hongtao Xie","slug":"hongtao-xie","given":"Hongtao","family":"Xie","affiliation":"School of Information Science and Technology , University of Science and Technology of China"},{"name":"Zilong Fu","slug":"zilong-fu","given":"Zilong","family":"Fu","affiliation":"School of Information Science and Technology , University of Science and Technology of China"},{"name":"Yongdong Zhang","slug":"yongdong-zhang","given":"Yongdong","family":"Zhang","affiliation":"School of Information Science and Technology , University of Science and Technology of China"}],"abstract":"Nowadays, scene text detection has become increasingly important and popular. However, the large variance of text scale remains the main challenge and limits the detection performance in most previous methods. To address this problem, we propose an end-to-end architecture called Deep Scale Relationship Network (DSRN) to map multi-scale convolution features onto a scale invariant space to obtain uniform activation of multi-size text instances. Firstly, we develop a Scale-transfer module to transfer the multi-scale feature maps to a unified dimension. Due to the heterogeneity of features, simply concatenating feature maps with multi-scale information would limit the detection performance. Thus we propose a Scale Relationship module to aggregate the multi-scale information through bi-directional convolution operations. Finally, to further reduce the miss-detected instances, a novel Recall Loss is proposed to force the network to concern more about miss-detected text instances by up-weighting poor-classified examples. Compared with previous approaches, DSRN efficiently handles the large-variance scale problem without complex hand-crafted hyperparameter settings (e.g. scale of default boxes) and complicated post processing. On standard datasets including ICDAR2015 and MSRA-TD500, the proposed algorithm achieves the state-of-art performance with impressive speed (8.8 FPS on ICDAR2015 and 13.3 FPS on MSRA-TD500).","doi":"10.24963/ijcai.2019/133","url":"https://ijcai.org/proceedings/2019/133","pdf":"https://ijcai.org/proceedings/2019/0133.pdf","markdown":"https://ijcai.org/proceedings/2019/133.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/133","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":133,"pages":{"first":947,"last":953},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Berman et al., 2018] Maxim Berman, Amal Rannen Triki, and Matthew B. Blaschko. The lovász-softmax loss: A tractable surrogate for the optimization of the intersection- over-union measure in neural networks. In CVPR, pages 4413-4421, 2018.","title":"The lovász-softmax loss: A tractable surrogate for the optimization of the intersectionover-union measure in neural networks","authors":"Berman","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Deng et al., 2018] Dan Deng, Haifeng Liu, Xuelong Li, and Deng Cai. Pixellink: Detecting scene text via instance segmentation. In AAAI, pages 6773-6780, 2018.","title":"Pixellink: Detecting scene text via instance segmentation","authors":"Dan Deng, Haifeng Liu, Xuelong Li, Deng Cai","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fang et al., 2018] Shancheng Fang, Hongtao Xie, Zheng- Jun Zha, Nannan Sun, Jianlong Tan, and Yongdong Zhang. Attention and language ensemble for scene text recognition with convolutional sequence modeling. In 2018 ACM MM, pages 248-256. ACM, 2018.","title":"Attention and language ensemble for scene text recognition with convolutional sequence modeling","authors":"Shancheng Fang, Hongtao Xie, Zheng-Jun Zha, Nannan Sun, Jianlong Tan, Yongdong Zhang","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gupta et al., 2016] Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman. Synthetic data for text localisation in natural images. In CVPR, pages 2315-2324, 2016.","title":"Synthetic data for text localisation in natural images","authors":"Ankush Gupta, Andrea Vedaldi, Andrew Zisserman","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2017a] Pan He, Weilin Huang, Tong He, Qile Zhu, Yu Qiao, and Xiaolin Li. Single shot text detector with regional attention. In ICCV, pages 3066-3074, 2017.","title":"Single shot text detector with regional attention","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017b] Wenhao He, Xu-Yao Zhang, Fei Yin, and Cheng-Lin Liu. Deep direct regression for multi-oriented scene text detection. In ICCV, pages 745-753, 2017.","title":"Deep direct regression for multi-oriented scene text detection","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jiang et al., 2018] Yingying Jiang, Xiangyu Zhu, Xiaobing Wang, Shuli Yang, Wei Li, Hua Wang, Pei Fu, and Zhenbo Luo. R 2 cnn: Rotational region cnn for arbitrarily- oriented scene text detection. In 2018 24th ICPR, pages 3610-3615. IEEE, 2018.","title":"R 2 cnn: Rotational region cnn for arbitrarilyoriented scene text detection","authors":"Yingying Jiang, Xiangyu Zhu, Xiaobing Wang, Shuli Yang, Wei Li, Hua Wang, Pei Fu, Zhenbo Luo","year":"2018","venue":"24th ICPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Karatzas et al., 2015] Dimosthenis Karatzas, Lluis Gomez- Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bag- danov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, et al. Icdar 2015 competition on robust reading. In 2015 13th ICDAR, pages 1156-1160. IEEE, 2015.","title":"Icdar 2015 competition on robust reading","authors":"Dimosthenis Karatzas, Lluis Gomez-Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bagdanov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu","year":"2015","venue":"13th ICDAR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2018] Yonghyun Kim, Bong-Nam Kang, and Daijin Kim. SAN: learning relationship between convolutional features for multi-scale object detection. In ECCV, pages 328-343. Springer, 2018.","title":"SAN: learning relationship between convolutional features for multi-scale object detection","authors":"Kim","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liao et al., 2018] Minghui Liao, Zhen Zhu, Baoguang Shi, Gui-Song Xia, and Xiang Bai. Rotation-sensitive regression for oriented scene text detection. In CVPR, pages 5909-5918, 2018.","title":"Rotation-sensitive regression for oriented scene text detection","authors":"Liao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2017a] Tsung-Yi Lin, Piotr Dollár, Ross B. Girshick, Kaiming He, Bharath Hariharan, and Serge J. Belongie. Feature pyramid networks for object detection. In CVPR, pages 936-944, 2017.","title":"Feature pyramid networks for object detection","authors":"Lin","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2017b] Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pages 2999-3007, 2017.","title":"Focal loss for dense object detection","authors":"Lin","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2016] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott E. Reed, Cheng-Yang Fu, and Alexander C. Berg. SSD: single shot multibox detector. In ECCV, pages 21-37. Springer, 2016.","title":"SSD: single shot multibox detector","authors":"Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott E Reed, Cheng-Yang Fu, Alexander C Berg","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2018] Zichuan Liu, Guosheng Lin, Sheng Yang, Jiashi Feng, Weisi Lin, and Wang Ling Goh. Learning markov clustering networks for scene text detection. In CVPR, pages 6936-6944, 2018.","title":"Learning markov clustering networks for scene text detection","authors":"Liu","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431-3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Long et al., 2018] Shangbang Long, Jiaqiang Ruan, Wenjie Zhang, Xin He, Wenhao Wu, and Cong Yao. Textsnake: A flexible representation for detecting text of arbitrary shapes. In ECCV, pages 19-35. Springer, 2018.","title":"Textsnake: A flexible representation for detecting text of arbitrary shapes","authors":"Long","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lyu et al., 2018] Pengyuan Lyu, Cong Yao, Wenhao Wu, Shuicheng Yan, and Xiang Bai. Multi-oriented scene text detection via corner localization and region segmentation. In CVPR, pages 7553-7563, 2018.","title":"Multi-oriented scene text detection via corner localization and region segmentation","authors":"Lyu","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Min and Chen, 2018] Shaobo Min and Xuejin Chen. A robust deep attention network to noisy labels in semi- supervised biomedical segmentation. arXiv preprint arXiv:1807.11719, 2018.","title":"A robust deep attention network to noisy labels in semisupervised biomedical segmentation","authors":"Min, Chen ;, Shaobo Min, Xuejin Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2017] Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, and Xiaoou Tang. Residual attention network for image classification. In CVPR, pages 6450-6458, 2017.","title":"Residual attention network for image classification","authors":"Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, Xiaoou Tang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2018] Fangfang Wang, Liming Zhao, Xi Li, Xinchao Wang, and Dacheng Tao. Geometry-aware scene text detection with instance transformation network. In CVPR, pages 1381-1389, 2018.","title":"Geometry-aware scene text detection with instance transformation network","authors":"Fangfang Wang, Liming Zhao, Xi Li, Xinchao Wang, Dacheng Tao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xie et al., 2019] Hongtao Xie, Shancheng Fang, Zheng-Jun Zha, Yating Yang, Yan Li, and Yongdong Zhang. Convolu- tional attention networks for scene text recognition. ACM TOMM, 15(1s):3, 2019.","title":"Convolutional attention networks for scene text recognition","authors":"Hongtao Xie, Shancheng Fang, Zheng-Jun Zha, Yating Yang, Yan Li, Yongdong Zhang","year":"2019","venue":"ACM TOMM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yao et al., 2012] Cong Yao, Xiang Bai, Wenyu Liu, Yi Ma, and Zhuowen Tu. Detecting texts of arbitrary orientations in natural images. In CVPR, pages 1083-1090, 2012.","title":"Detecting texts of arbitrary orientations in natural images","authors":"Cong Yao, Xiang Bai, Wenyu Liu, Yi Ma, Zhuowen Tu","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yao et al., 2014] Cong Yao, Xiang Bai, and Wenyu Liu. A unified framework for multioriented text detection and recognition. IEEE Trans. Image Processing, 23(11):4737- 4749, 2014.","title":"A unified framework for multioriented text detection and recognition","authors":"Cong Yao, Xiang Bai, Wenyu Liu","year":"2014","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zeng et al., 2016] Xingyu Zeng, Wanli Ouyang, Bin Yang, Junjie Yan, and Xiaogang Wang. Gated bi-directional CNN for object detection. In ECCV, pages 354-369, 2016.","title":"Gated bi-directional CNN for object detection","authors":"Xingyu Zeng, Wanli Ouyang, Bin Yang, Junjie Yan, Xiaogang Wang","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2017a] Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. EAST: an efficient and accurate scene text detector. In CVPR, pages 2642-2651, 2017.","title":"EAST: an efficient and accurate scene text detector","authors":"Zhou","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2017b] Yanzhao Zhou, Qixiang Ye, Qiang Qiu, and Jianbin Jiao. Oriented response networks. In CVPR, pages 4961-4970, 2017.","title":"Oriented response networks","authors":"Zhou","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2018] Peng Zhou, Bingbing Ni, Cong Geng, Jianguo Hu, and Yi Xu. Scale-transferrable object detection. In CVPR, pages 528-537, 2018.","title":"Scale-transferrable object detection","authors":"Peng Zhou, Bingbing Ni, Cong Geng, Jianguo Hu, Yi Xu","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}