{"type":"ScholarlyArticle","title":"Omnidirectional Scene Text Detection with Sequential-free Box Discretization","authors":[{"name":"Yuliang Liu","slug":"yuliang-liu","given":"Yuliang","family":"Liu","affiliation":"School of Electronic and Information Engineering , South China University of Technology , China"},{"name":"Sheng Zhang","slug":"sheng-zhang","given":"Sheng","family":"Zhang","affiliation":"School of Electronic and Information Engineering , South China University of Technology , China"},{"name":"Lianwen Jin","slug":"lianwen-jin","given":"Lianwen","family":"Jin","affiliation":"School of Electronic and Information Engineering , South China University of Technology , China"},{"name":"Lele Xie","slug":"lele-xie","given":"Lele","family":"Xie","affiliation":"School of Electronic and Information Engineering , South China University of Technology , China"},{"name":"Yaqiang Wu","slug":"yaqiang-wu","given":"Yaqiang","family":"Wu","affiliation":"Lenovo Inc , China"},{"name":"Zhepeng Wang","slug":"zhepeng-wang","given":"Zhepeng","family":"Wang","affiliation":"Lenovo Inc , China"}],"abstract":"Scene text in the wild is commonly presented with high variant characteristics. Using quadrilateral bounding box to localize the text instance is nearly indispensable for detection methods. However, recent researches reveal that introducing quadrilateral bounding box for scene text detection will bring a label confusion issue which is easily overlooked, and this issue may significantly undermine the detection performance. To address this issue, in this paper, we propose a novel method called Sequential-free Box Discretization (SBD) by discretizing the bounding box into key edges (KE) which can further derive more effective methods to improve detection performance. Experiments showed that the proposed method can outperform state-of-the-art methods in many popular scene text benchmarks, including ICDAR 2015, MLT, and MSRA-TD500. Ablation study also showed that simply integrating the SBD into Mask R-CNN framework, the detection performance can be substantially improved. Furthermore, an experiment on the general object dataset HRSC2016 (multi-oriented ships) showed that our method can outperform recent state-of-the-art methods by a large margin, demonstrating its powerful generalization ability.","doi":"10.24963/ijcai.2019/423","url":"https://ijcai.org/proceedings/2019/423","pdf":"https://ijcai.org/proceedings/2019/0423.pdf","markdown":"https://ijcai.org/proceedings/2019/423.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/423","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":423,"pages":{"first":3052,"last":3058},"keywords":["Machine Learning: Deep Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Deng et al., 2018] Dan Deng, Haifeng Liu, Xuelong Li, and Deng Cai. Pixellink: Detecting scene text via instance segmentation. arXiv preprint arXiv:1801.01315, 2018.","title":"Pixellink: Detecting scene text via instance segmentation","authors":"Dan Deng, Haifeng Liu, Xuelong Li, Deng Cai","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Girshick, 2015] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440-1448, 2015.","title":"Fast r-cnn","authors":"Ross Girshick","year":"2015","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gupta et al., 2016] Ankush Gupta, Andrea Vedaldi, and An- drew Zisserman. Synthetic data for text localisation in nat- ural images. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.","title":"Synthetic data for text localisation in natural images","authors":"Ankush Gupta, Andrea Vedaldi, Andrew Zisserman","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2017a] Kaiming He, Georgia Gkioxari, Piotr Dol- lár, and Ross Girshick. Mask r-cnn. In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 2980-2988. IEEE, 2017.","title":"Mask r-cnn","authors":"He","year":"2017","venue":"Computer Vision (ICCV), 2017 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2017b] Wenhao He, Xu-Yao Zhang, Fei Yin, and Cheng-Lin Liu. Deep direct regression for multi-oriented scene text detection. Proceedings of the IEEE Internation- al Conference on Computer Vision, 2017.","title":"Deep direct regression for multi-oriented scene text detection","authors":"He","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2018] Zheqi He, Yafeng Zhou, Yongtao Wang, Si- wei Wang, Xiaoqing Lu, Zhi Tang, and Ling Cai. An end- to-end quadrilateral regression network for comic panel extraction. In 2018 ACM Multimedia Conference on Mul- timedia Conference, pages 887-895. ACM, 2018.","title":"An endto-end quadrilateral regression network for comic panel extraction","authors":"He","year":"2018","venue":"ACM Multimedia Conference on Multimedia Conference","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hu et al., 2017] Han Hu, Chengquan Zhang, Yuxuan Luo, Yuzhuo Wang, Junyu Han, and Errui Ding. Wordsup: Ex- ploiting word annotations for character based text detec- tion. Proceedings of the IEEE International Conference on Computer Vision, 2017.","title":"Wordsup: Exploiting word annotations for character based text detection","authors":"Han Hu, Chengquan Zhang, Yuxuan Luo, Yuzhuo Wang, Junyu Han, Errui Ding","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kang et al., 2014] Le Kang, Yi Li, and David Doermann. Orientation robust text line detection in natural images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4034-4041, 2014.","title":"Orientation robust text line detection in natural images","authors":"Le Kang, Yi Li, David Doermann","year":"2014","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Karatzas and Gomez-Bigorda, 2015] Dimosthenis Karatzas and et al. Gomez-Bigorda, Lluis. Icdar 2015 competition on robust reading. In Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, pages 1156-1160. IEEE, 2015.","title":"Icdar 2015 competition on robust reading","authors":"Dimosthenis Karatzas","year":"2015","venue":"Document Analysis and Recognition (ICDAR), 2015 13th International Conference on","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liao et al., 2018a] Minghui Liao, Baoguang Shi, and Xiang Bai. Textboxes++: A single-shot oriented scene text detec- tor. IEEE Transactions on Image Processing, 27(8):3676- 3690, 2018.","title":"Textboxes++: A single-shot oriented scene text detector","authors":"Liao","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liao et al., 2018b] Minghui Liao, Zhen Zhu, Baoguang Shi, Gui-song Xia, and Xiang Bai. Rotation-sensitive regres- sion for oriented scene text detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5909-5918, 2018.","title":"Rotation-sensitive regression for oriented scene text detection","authors":"Liao","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu and Jin, 2017] Yuliang Liu and Lianwen Jin. Deep matching prior network: Toward tighter multi-oriented text detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.","title":"Deep matching prior network: Toward tighter multi-oriented text detection","authors":"Jin ; ] Yuliang Liu, Lianwen Liu, Jin","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Zikun Liu, Jingao Hu, Lubin Weng, and Yiping Yang. Rotated region based cnn for ship detec- tion. In Image Processing (ICIP), 2017 IEEE Internation- al Conference on, pages 900-904. IEEE, 2017.","title":"Rotated region based cnn for ship detection","authors":"Liu","year":"2017","venue":"Image Processing (ICIP), 2017 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2018] Xuebo Liu, Ding Liang, Shi Yan, Dagui Chen, Yu Qiao, and Junjie Yan. Fots: Fast oriented tex- t spotting with a unified network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 5676-5685, 2018.","title":"Fots: Fast oriented text spotting with a unified network","authors":"Xuebo Liu, Ding Liang, Shi Yan, Dagui Chen, Yu Qiao, Junjie Yan","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor. Darrell. Fully convolutional networks for seman- tic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3431- 3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lyu et al., 2018] Pengyuan Lyu, Cong Yao, Wenhao Wu, Shuicheng Yan, and Xiang Bai. Multi-oriented scene text detection via corner localization and region segmentation. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, pages 7553-7563, 2018.","title":"Multi-oriented scene text detection via corner localization and region segmentation","authors":"Lyu","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ma et al., 2018] Jianqi Ma, Weiyuan Shao, Hao Ye, Li Wang, Hong Wang, Yingbin Zheng, and Xiangyang Xue. Arbitrary-oriented scene text detection via rotation proposals. IEEE Transactions on Multimedia, 2018. Nayef et al., 2017] Nibal Nayef, Fei Yin, Imen Bizid, Hyunsoo Choi, Yuan Feng, Dimosthenis Karatzas, Zhenbo Luo, Umapada Pal, Christophe Rigaud, Joseph Chazalon, et al. Icdar2017 robust reading challenge on multi-lingual scene text detection and script identification-rrc-mlt. In Document Analysis and Recognition (ICDAR), 2017 14th IAPR International Conference on, volume 1, pages 1454- 1459. IEEE, 2017.","title":"Icdar2017 robust reading challenge on multi-lingual scene text detection and script identification-rrc-mlt","authors":"Ma","year":"2017","venue":"Document Analysis and Recognition (ICDAR)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi et al., 2017a] B. Shi, Yao, M. Liao, Yang M., Xu P., L. Cui, Lu S. Serge Belongie, and Bai X. Icdar2017 com- petition on reading chinese text in the wild (rctw-17). arX- iv preprint arXiv:1708.09585, 2017.","title":"Icdar2017 competition on reading chinese text in the wild (rctw-17)","authors":"Shi","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shi et al., 2017b] Baoguang Shi, Xiang Bai, and Serge Be- longie. Detecting oriented text in natural images by linking segments. Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, 2017.","title":"Detecting oriented text in natural images by linking segments","authors":"Shi","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tian et al., 2016] Zhi Tian, Weilin Huang, Tong He, Pan He, and Yu Qiao. Detecting text in natural image with connectionist text proposal network. In European Confer- ence on Computer Vision, pages 56-72. Springer, 2016.","title":"Detecting text in natural image with connectionist text proposal network","authors":"Weilin Zhi Tian, Tong Huang, Pan He, Yu He, Qiao","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu and Natarajan, 2017] Yue Wu and Prem Natarajan. Self-organized text detection with minimal post- processing via border learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5000-5009, 2017.","title":"Self-organized text detection with minimal postprocessing via border learning","authors":"Yue Wu, Prem Natarajan","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yao et al., 2012] C. Yao, X. Bai, W. Liu, and Y. Ma. De- tecting texts of arbitrary orientations in natural images. In Computer Vision and Pattern Recognition, pages 1083- 1090, 2012.","title":"Detecting texts of arbitrary orientations in natural images","authors":"C Yao, X Bai, W Liu, Y Ma","year":"2012","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yao et al., 2016] Cong Yao, Xiang Bai, Nong Sang, Xinyu Zhou, Shuchang Zhou, and Zhimin Cao. Scene text detec- tion via holistic, multi-channel prediction. arXiv preprint arXiv:1606.09002, 2016.","title":"Scene text detection via holistic, multi-channel prediction","authors":"Cong Yao, Xiang Bai, Nong Sang, Xinyu Zhou, Shuchang Zhou, Zhimin Cao","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yuliang et al., 2017] Liu Yuliang, Jin Lianwen, Zhang Shuaitao, and Zhang Sheng. Detecting curve text in the wild: New dataset and new solution. arXiv preprint arX- iv:1712.02170, 2017.","title":"Detecting curve text in the wild: New dataset and new solution","authors":"Liu Yuliang, Jin Lianwen, Zhang Shuaitao, Zhang Sheng","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2016] Zheng Zhang, Chengquan Zhang, Wei Shen, Cong Yao, Wenyu Liu, and Xiang Bai. Multi- oriented text detection with fully convolutional network- s. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4159-4167, 2016.","title":"Multioriented text detection with fully convolutional networks","authors":"Zheng Zhang, Chengquan Zhang, Wei Shen, Cong Yao, Wenyu Liu, Xiang Bai","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhong et al., 2018] Zhuoyao Zhong, Lei Sun, and Qiang Huo. An anchor-free region proposal network for faster r-cnn based text detection approaches. arXiv preprint arX- iv:1804.09003, 2018.","title":"An anchor-free region proposal network for faster r-cnn based text detection approaches","authors":"Zhong","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2017] Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. East: An efficient and accurate scene text detector. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.","title":"East: An efficient and accurate scene text detector","authors":"Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, Jiajun Liang","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu and Du, 2018] Yixing Zhu and Jun Du. Sliding line point regression for shape robust scene text detection. arX- iv preprint arXiv:1801.09969, 2018.","title":"Yixing Zhu and Jun Du. Sliding line point regression for shape robust scene text detection","authors":"Du Zhu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}