{"type":"ScholarlyArticle","title":"TextFuseNet: Scene Text Detection with Richer Fused Features","authors":[{"name":"Jian Ye","slug":"jian-ye","given":"Jian","family":"Ye","affiliation":"School of Computer Science , Institute of Artificial Intelligence , and National Engineering Research Center for Multimedia Software , Wuhan University , China"},{"name":"Zhe Chen","slug":"zhe-chen","given":"Zhe","family":"Chen","affiliation":"UBTECH Sydney AI Centre , School of Computer Science , Faculty of Engineering , The University of Sydney , Australia"},{"name":"Juhua Liu","slug":"juhua-liu","given":"Juhua","family":"Liu","affiliation":"School of Printing and Packaging , and Institute of Artificial Intelligence , Wuhan University , China"},{"name":"Bo Du","slug":"bo-du","given":"Bo","family":"Du","affiliation":"School of Computer Science , Institute of Artificial Intelligence , and National Engineering Research Center for Multimedia Software , Wuhan University , China"}],"abstract":"Arbitrary shape text detection in natural scenes is an extremely challenging task. Unlike existing text detection approaches that only perceive texts based on limited feature representations, we propose a novel framework, namely TextFuseNet, to exploit the use of richer features fused for text detection. More specifically, we propose to perceive texts from three levels of feature representations, i.e., character-, word- and global-level, and then introduce a novel text representation fusion technique to help achieve robust arbitrary text detection. The multi-level feature representation can adequately describe texts by dissecting them into individual characters while still maintaining their general semantics. TextFuseNet then collects and merges the texts’ features from different levels using a multi-path fusion architecture which can effectively align and fuse different representations. In practice, our proposed TextFuseNet can learn a more adequate description of arbitrary shapes texts, suppressing false positives and producing more accurate detection results. Our proposed framework can also be trained with weak supervision for those datasets that lack character-level annotations. Experiments on several datasets show that the proposed TextFuseNet achieves state-of-the-art performance. Specifically, we achieve an F-measure of 94.3% on ICDAR2013, 92.1% on ICDAR2015, 87.1% on Total-Text and 86.6% on CTW-1500, respectively.","doi":"10.24963/ijcai.2020/72","url":"https://ijcai.org/proceedings/2020/72","pdf":"https://ijcai.org/proceedings/2020/0072.pdf","markdown":"https://ijcai.org/proceedings/2020/72.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/72","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":72,"pages":{"first":516,"last":522},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Machine Learning: Deep Learning: Convolutional networks","Machine Learning: Semi-Supervised Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24505/file_24505.m3u8","length_sec":279,"provider":"hls"}],"references":[{"position":1,"raw":"[Deng et al., 2018] Dan Deng, Haifeng Liu, Xuelong Li, and Deng Cai. Pixellink: Detecting scene text via instance seg- mentation. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"Pixellink: Detecting scene text via instance segmentation","authors":"Dan Deng, Haifeng Liu, Xuelong Li, Deng Cai","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Gao et al., 2019] Mingyu Gao, Yujie Du, Yuxiang Yang, and Jing Zhang. Adaptive anchor box mechanism to improve the accuracy in the object detection system. Multimedia Tools and Applications, 78(19):27383-27402, 2019.","title":"Adaptive anchor box mechanism to improve the accuracy in the object detection system","authors":"Mingyu Gao, Yujie Du, Yuxiang Yang, Jing Zhang","year":"2019","venue":"Multimedia Tools and Applications","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dol- lár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.","title":"Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn","authors":"He","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Liao et al., 2018a] Minghui Liao, Baoguang Shi, and Xiang Bai. Textboxes++: A single-shot oriented scene text detec- tor. IEEE transactions on image processing, 27(8):3676- 3690, 2018.","title":"Textboxes++: A single-shot oriented scene text detector","authors":"Liao","year":"2018","venue":"IEEE transactions on image processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Liao et al., 2018b] Minghui Liao, Zhen Zhu, Baoguang Shi, Gui-song Xia, and Xiang Bai. Rotation-sensitive regres- sion for oriented scene text detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5909-5918, 2018.","title":"Rotation-sensitive regression for oriented scene text detection","authors":"Liao","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Liao et al., 2020] Minghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen, and Xiang Bai. Real-time scene text detection with differentiable binarization. In Thirty-Fourth AAAI Conference on Artificial Intel-ligence, 2020.","title":"Real-time scene text detection with differentiable binarization","authors":"Liao","year":"2020","venue":"Thirty-Fourth AAAI Conference on Artificial Intel-ligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liu et al., 2019a] Yuliang Liu, Lianwen Jin, Shuaitao Zhang, Canjie Luo, and Sheng Zhang. Curved scene text detection via transverse and longitudinal sequence connec- tion. Pattern Recognition, 90:337-345, 2019.","title":"Curved scene text detection via transverse and longitudinal sequence connection","authors":"Liu","year":"2019","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2019b] Zichuan Liu, Guosheng Lin, Sheng Yang, Fayao Liu, Weisi Lin, and Wang Ling Goh. Towards robust curve text detection with conditional spatial expansion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7269-7278, 2019.","title":"Towards robust curve text detection with conditional spatial expansion","authors":"Liu","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Long et al., 2018] Shangbang Long, Jiaqiang Ruan, Wenjie Zhang, Xin He, Wenhao Wu, and Cong Yao. Textsnake: A flexible representation for detecting text of arbitrary shapes. In Proceedings of ECCV, pages 20-36, 2018.","title":"Textsnake: A flexible representation for detecting text of arbitrary shapes","authors":"Long","year":"2018","venue":"Proceedings of ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lyu et al., 2018] Pengyuan Lyu, Minghui Liao, Cong Yao, Wenhao Wu, and Xiang Bai. Mask textspotter: An end- to-end trainable neural network for spotting text with arbi- trary shapes. In Proceedings of ECCV, pages 67-83, 2018.","title":"Mask textspotter: An endto-end trainable neural network for spotting text with arbitrary shapes","authors":"Lyu","year":"2018","venue":"Proceedings of ECCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Qiao et al., 2019] Tingting Qiao, Jing Zhang, Duanqing X- u, and Dacheng Tao. Mirrorgan: Learning text-to-image generation by redescription. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1505-1514, 2019.","title":"Mirrorgan: Learning text-to-image generation by redescription","authors":"Tingting Qiao, Jing Zhang, Duanqing Xu, Dacheng Tao","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Shi et al., 2017] Baoguang Shi, Xiang Bai, and Serge Be- longie. Detecting oriented text in natural images by link- ing segments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2550- 2558, 2017.","title":"Detecting oriented text in natural images by linking segments","authors":"Shi","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Tian et al., 2016] Zhi Tian, Weilin Huang, Tong He, Pan He, and Yu Qiao. Detecting text in natural image with con- nectionist text proposal network. In Proceedings of ECCV, pages 56-72. Springer, 2016.","title":"Detecting text in natural image with connectionist text proposal network","authors":"Tian","year":"2016","venue":"Proceedings of ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tian et al., 2019] Zhuotao Tian, Michelle Shu, Pengyuan Lyu, Ruiyu Li, Chao Zhou, Xiaoyong Shen, and Jiaya Jia. Learning shape-aware embedding for scene text detection. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, pages 4234-4243, 2019.","title":"Learning shape-aware embedding for scene text detection","authors":"Tian","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wang et al., 2017] Zheng Wang, Ruimin Hu, Chen Chen, Yi Yu, Junjun Jiang, Chao Liang, and Shin'ichi Satoh. Person reidentification via discrepancy matrix and matrix metric. IEEE transactions on cybernetics, 48(10):3006- 3020, 2017.","title":"Person reidentification via discrepancy matrix and matrix metric","authors":"Zheng Wang, Ruimin Hu, Chen Chen, Yi Yu, Junjun Jiang, Chao Liang, Shin'ichi Satoh","year":"2017","venue":"IEEE transactions on cybernetics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2019a] Wenhai Wang, Enze Xie, Xiang Li, Wenbo Hou, Tong Lu, Gang Yu, and Shuai Shao. Shape robust text detection with progressive scale expansion net- work. In Proceedings of the IEEE International Confer- ence on Computer Vision, 2019.","title":"Shape robust text detection with progressive scale expansion network","authors":"Wang","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2019b] Wenhai Wang, Enze Xie, Xiaoge Song, Yuhang Zang, Wenjia Wang, Tong Lu, Gang Yu, and Chunhua Shen. Efficient and accurate arbitrary-shaped text detection with pixel aggregation network. In Proceed- ings of the IEEE International Conference on Computer Vision, pages 8440-8449, 2019.","title":"Efficient and accurate arbitrary-shaped text detection with pixel aggregation network","authors":"Wang","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2019c] Xiaobing Wang, Yingying Jiang, Zhen- bo Luo, Cheng-Lin Liu, Hyunsoo Choi, and Sungjin Kim. Arbitrary shape scene text detection with adaptive text re- gion representation. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition, pages 6449-6458, 2019.","title":"Arbitrary shape scene text detection with adaptive text region representation","authors":"Wang","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2019d] Zheng Wang, Junjun Jiang, Yang Wu, Mang Ye, Xiang Bai, and Shin'ichi Satoh. Learning s- parse and identity-preserved hidden attributes for person re-identification. IEEE Transactions on Image Processing, 29(1):2013-2025, 2019.","title":"Learning sparse and identity-preserved hidden attributes for person re-identification","authors":"Wang","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xie et al., 2019] Enze Xie, Yuhang Zang, Shuai Shao, Gang Yu, Cong Yao, and Guangyao Li. Scene text detection with supervised pyramid context network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 9038-9045, 2019.","title":"Scene text detection with supervised pyramid context network","authors":"Xie","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2019] Yongchao Xu, Yukang Wang, Wei Zhou, Yongpan Wang, Zhibo Yang, and Xiang Bai. Textfield: Learning a deep direction field for irregular scene text de- tection. IEEE Transactions on Image Processing, 2019.","title":"Textfield: Learning a deep direction field for irregular scene text detection","authors":"Yongchao Xu, Yukang Wang, Wei Zhou, Yongpan Wang, Zhibo Yang, Xiang Bai","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xue et al., 2019] Chuhui Xue, Shijian Lu, and Wei Zhang. Msr: Multi-scale shape regression for scene text detection. In Proceedings of the 28th International Joint Confer-ence on Artificial Intelligence, 2019.","title":"Msr: Multi-scale shape regression for scene text detection","authors":"Xue","year":"2019","venue":"Proceedings of the 28th International Joint Confer-ence on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2019/139"},{"position":23,"raw":"[Zhang et al., 2019] Chengquan Zhang, Borong Liang, Zuming Huang, Mengyi En, Junyu Han, Errui Ding, and Xinghao Ding. Look more than once: An accurate detector for text of arbitrary shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.","title":"Look more than once: An accurate detector for text of arbitrary shapes","authors":"Zhang","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}