{"type":"ScholarlyArticle","title":"MSR: Multi-Scale Shape Regression for Scene Text Detection","authors":[{"name":"Chuhui Xue","slug":"chuhui-xue","given":"Chuhui","family":"Xue","affiliation":"School of Computer Science and Engineering , Nanyang Technological University"},{"name":"Shijian Lu","slug":"shijian-lu","given":"Shijian","family":"Lu","affiliation":"School of Computer Science and Engineering , Nanyang Technological University"},{"name":"Wei Zhang","slug":"wei-zhang","given":"Wei","family":"Zhang","affiliation":"School of Control Science and Engineering , Shandong University"}],"abstract":"State-of-the-art scene text detection techniques predict quadrilateral boxes that are prone to localization errors while dealing with straight or curved text lines of different orientations and lengths in scenes. This paper presents a novel multi-scale shape regression network (MSR) that is capable of locating text lines of different lengths, shapes and curvatures in scenes. The proposed MSR detects scene texts by predicting dense text boundary points that inherently capture the location and shape of text lines accurately and are also more tolerant to the variation of text line length as compared with the state of the arts using proposals or segmentation. Additionally, the multi-scale network extracts and fuses features at different scales which demonstrates superb tolerance to the text scale variation. Extensive experiments over several public datasets show that the proposed MSR obtains superior detection performance for both curved and straight text lines of different lengths and orientations.","doi":"10.24963/ijcai.2019/139","url":"https://ijcai.org/proceedings/2019/139","pdf":"https://ijcai.org/proceedings/2019/0139.pdf","markdown":"https://ijcai.org/proceedings/2019/139.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/139","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":139,"pages":{"first":989,"last":995},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Akkiraju et al., 1995] Nataraj Akkiraju, Herbert Edelsbrun- ner, Michael Facello, Ping Fu, EP Mucke, and Carlos Varela. Alpha shapes: definition and software. In Pro- ceedings of the 1st International Computational Geometry Software Workshop, volume 63, page 66, 1995.","title":"Alpha shapes: definition and software","authors":"Nataraj Akkiraju, Herbert Edelsbrunner, Michael Facello, Ping Fu, Carlos Ep Mucke, Varela","year":"1995","venue":"Proceedings of the 1st International Computational Geometry Software Workshop","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ch'ng and Chan, 2017] Chee Kheng Ch'ng and Chee Seng Chan. Total-text: A comprehensive dataset for scene text detection and recognition. In Document Analysis and Recognition (ICDAR), 2017 14th IAPR International Con- ference on, volume 1, pages 935-942, 2017.","title":"Total-text: A comprehensive dataset for scene text detection and recognition","authors":"' Ch, Chan ; Chee Kheng Ng, ' Ch, Chee Ng, Chan Seng","year":"2017","venue":"Document Analysis and Recognition (ICDAR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2018] Dan Deng, Haifeng Liu, Xuelong Li, and Deng Cai. Pixellink: Detecting scene text via instance seg- mentation. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"Pixellink: Detecting scene text via instance segmentation","authors":"Dan Deng, Haifeng Liu, Xuelong Li, Deng Cai","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Girshick, 2015] Ross Girshick. Fast r-cnn. In IEEE inter- national conference on computer vision, 2015.","title":"Fast r-cnn","authors":"Ross Girshick","year":"2015","venue":"IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gupta et al., 2016] A. Gupta, A. Vedaldi, and A. Zisserman. Synthetic data for text localisation in natural images. In IEEE Conference on Computer Vision and Pattern Recog- nition, 2016.","title":"Synthetic data for text localisation in natural images","authors":"A Gupta, A Vedaldi, A Zisserman","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hao et al., 2017] Zekun Hao, Yu Liu, Hongwei Qin, Junjie Yan, Xiu Li, and Xiaolin Hu. Scale-aware face detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 3, 2017.","title":"Scale-aware face detection","authors":"Zekun Hao, Yu Liu, Hongwei Qin, Junjie Yan, Xiu Li, Xiaolin Hu","year":"2017","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2017] Wenhao He, Xu-Yao Zhang, Fei Yin, and Cheng-Lin Liu. Deep direct regression for multi-oriented scene text detection. In IEEE International Conference on Computer Vision, pages 745-753, 2017.","title":"Deep direct regression for multi-oriented scene text detection","authors":"Wenhao He, Xu-Yao Zhang, Fei Yin, Cheng-Lin Liu","year":"2017","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2015] Lichao Huang, Yi Yang, Yafeng Deng, and Yinan Yu. Densebox: Unifying landmark localiza- tion with end to end object detection. arXiv preprint arXiv:1509.04874, 2015.","title":"Densebox: Unifying landmark localization with end to end object detection","authors":"Huang","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Karatzas et al., 2015] Dimosthenis Karatzas, Lluis Gomez- Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bag- danov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, et al. Icdar 2015 competition on robust reading. In 2015 13th Interna- tional Conference on Document Analysis and Recognition (ICDAR), pages 1156-1160, 2015.","title":"Icdar 2015 competition on robust reading","authors":"Dimosthenis Karatzas, Lluis Gomez-Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bagdanov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu","year":"2015","venue":"2015 13th International Conference on Document Analysis and Recognition (ICDAR)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liao et al., 2018] Minghui Liao, Zhen Zhu, Baoguang Shi, Gui-song Xia, and Xiang Bai. Rotation-sensitive regres- sion for oriented scene text detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5909-5918, 2018.","title":"Rotation-sensitive regression for oriented scene text detection","authors":"Liao","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Piotr Dollár, Ross B Gir- shick, Kaiming He, Bharath Hariharan, and Serge J Be- longie. Feature pyramid networks for object detection. In CVPR, volume 1, page 3, 2017.","title":"Feature pyramid networks for object detection","authors":"Tsung-Yi Lin, Piotr Dollár, Ross B Girshick, Kaiming He, Bharath Hariharan, Serge J Be-Longie","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Long et al., 2018] Shangbang Long, Jiaqiang Ruan, Wenjie Zhang, Xin He, Wenhao Wu, and Cong Yao. Textsnake: A flexible representation for detecting text of arbitrary shapes. In The European Conference on Computer Vision (ECCV), 2018.","title":"Textsnake: A flexible representation for detecting text of arbitrary shapes","authors":"Long","year":"2018","venue":"The European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lyu et al., 2018a] Pengyuan Lyu, Minghui Liao, Cong Yao, Wenhao Wu, and Xiang Bai. Mask textspotter: An end- to-end trainable neural network for spotting text with arbi- trary shapes. In The European Conference on Computer Vision (ECCV), 2018.","title":"Mask textspotter: An endto-end trainable neural network for spotting text with arbitrary shapes","authors":"Lyu","year":"2018","venue":"The European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lyu et al., 2018b] Pengyuan Lyu, Cong Yao, Wenhao Wu, Shuicheng Yan, and Xiang Bai. Multi-oriented scene text detection via corner localization and region segmentation. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, pages 7553-7563, 2018.","title":"Multi-oriented scene text detection via corner localization and region segmentation","authors":"Lyu","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234-241, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical image computing and computerassisted intervention","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi et al., 2017] Baoguang Shi, Xiang Bai, and Serge Be- longie. Detecting oriented text in natural images by linking segments. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, pages 3482-3490, 2017.","title":"Detecting oriented text in natural images by linking segments","authors":"Shi","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Singh and Davis, 2018] Bharat Singh and Larry S Davis. An analysis of scale invariance in object detection-snip. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, pages 3578-3587, 2018.","title":"An analysis of scale invariance in object detection-snip","authors":"Bharat Singh, Larry S Davis","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tian et al., 2015] Shangxuan Tian, Yifeng Pan, Chang Huang, Shijian Lu, Kai Yu, and Chew Lim Tan. Text flow: A unified text detection system in natural scene images. In Proceedings of the IEEE international conference on com- puter vision, pages 4651-4659, 2015.","title":"Text flow: A unified text detection system in natural scene images","authors":"Shangxuan Tian, Yifeng Pan, Chang Huang, Shijian Lu, Kai Yu, Chew Lim, Tan","year":"2015","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tian et al., 2016] Zhi Tian, Weilin Huang, Tong He, Pan He, and Yu Qiao. Detecting text in natural image with connectionist text proposal network. In European confer- ence on computer vision, pages 56-72. Springer, 2016.","title":"Detecting text in natural image with connectionist text proposal network","authors":"Weilin Zhi Tian, Tong Huang, Pan He, Yu He, Qiao","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xue et al., 2018] Chuhui Xue, Shijian Lu, and Fangneng Zhan. Accurate scene text detection through border se- mantics awareness and bootstrapping. In European Con- ference on Computer Vision, pages 370-387, 2018.","title":"Accurate scene text detection through border semantics awareness and bootstrapping","authors":"Xue","year":"2018","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yao et al., 2012] Cong Yao, Xiang Bai, Wenyu Liu, Yi Ma, and Zhuowen Tu. Detecting texts of arbitrary orientations in natural images. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 1083-1090, 2012.","title":"Detecting texts of arbitrary orientations in natural images","authors":"Cong Yao, Xiang Bai, Wenyu Liu, Yi Ma, Zhuowen Tu","year":"2012","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yuliang et al., 2017] Liu Yuliang, Jin Lianwen, Zhang Shuaitao, and Zhang Sheng. Detecting curve text in the wild: New dataset and new solution. arXiv preprint arXiv:1712.02170, 2017.","title":"Detecting curve text in the wild: New dataset and new solution","authors":"Liu Yuliang, Jin Lianwen, Zhang Shuaitao, Zhang Sheng","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2017] Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. East: an efficient and accurate scene text detector. In Proc. CVPR, pages 2642-2651, 2017.","title":"East: an efficient and accurate scene text detector","authors":"Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, Jiajun Liang","year":"2017","venue":"Proc. CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}