{"type":"ScholarlyArticle","title":"AI-Powered Oracle Bone Inscriptions Recognition and Fragments Rejoining","authors":[{"name":"Chongsheng Zhang","slug":"chongsheng-zhang","given":"Chongsheng","family":"Zhang","affiliation":"Henan University , Kaifeng , China"},{"name":"Ruixing Zong","slug":"ruixing-zong","given":"Ruixing","family":"Zong","affiliation":"Henan University , Kaifeng , China"},{"name":"Shuang Cao","slug":"shuang-cao","given":"Shuang","family":"Cao","affiliation":"Henan University , Kaifeng , China"},{"name":"Yi Men","slug":"yi-men","given":"Yi","family":"Men","affiliation":"Henan University , Kaifeng , China"},{"name":"Bofeng Mo","slug":"bofeng-mo","given":"Bofeng","family":"Mo","affiliation":"Capital Normal University , Beijing , China"}],"abstract":"Oracle Bone Inscriptions (OBI) research is very meaningful for both history and literature. In this paper, we introduce our contributions in AI-Powered Oracle Bone (OB) fragments rejoining and OBI recognition.   (1) We build a real-world dataset OB-Rejoin, and propose an effective OB rejoining algorithm which yields a top-10 accuracy of 98.39%. (2) We design a practical annotation software to facilitate OBI annotation, and build OracleBone-8000, a large-scale dataset with character-level annotations. We adopt deep learning based scene text detection algorithms for OBI localization, which yield an F-score of 89.7%. We propose  a novel deep template matching algorithm for OBI recognition which achieves  an overall accuracy of 80.9%. Since we have been cooperating closely with OBI domain experts,  our effort above helps advance their research. The resources of this work are available at https://github.com/chongshengzhang/OracleBone.","doi":"10.24963/ijcai.2020/779","url":"https://ijcai.org/proceedings/2020/779","pdf":"https://ijcai.org/proceedings/2020/0779.pdf","markdown":"https://ijcai.org/proceedings/2020/779.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/779","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Demos","track":"Demos","number":779,"pages":{"first":5309,"last":5311},"keywords":["Knowledge Representation and Reasoning: general","Machine Learning: general","Computer Vision: general"],"videos":[],"references":[{"position":1,"raw":"[Ding et al., 2008] Hui Ding, Goce Trajcevski, Peter Scheuermann, Xiaoyue Wang, and Eamonn Keogh. Querying and mining of time series data: experimental comparison of representations and distance measures. Proceedings of the VLDB Endowment, 1(2):1542-1552, 2008.","title":"Querying and mining of time series data: experimental comparison of representations and distance measures","authors":"Hui Ding, Goce Trajcevski, Peter Scheuermann, Xiaoyue Wang, Eamonn Keogh","year":"2008","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Franken and van Gemert, 2013] Morris Franken and Jan C. van Gemert. Automatic egyptian hieroglyph recognition by retrieving images as texts. In ACM MM 2013, pages 765-768, 2013.","title":"Automatic egyptian hieroglyph recognition by retrieving images as texts","authors":"Van Franken, ; Gemert, Morris Franken, Jan C Van Gemert","year":"2013","venue":"ACM MM 2013","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Guo et al., 2016] J. Guo, C. Wang, E. Roman-Rangel, H. Chao, and Y. Rui. Building hierarchical representations for oracle character and sketch recognition. IEEE Trans- actions on Image Processing, 25(1):104-118, 2016.","title":"Building hierarchical representations for oracle character and sketch recognition","authors":"J Guo, C Wang, E Roman-Rangel, H Chao, Y Rui","year":"2016","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guo, 1982] Moruo Guo. Collections of oracle-bone inscrip- tions. The Zhong Hua Book Press, 1982.","title":"Collections of oracle-bone inscriptions","authors":"Moruo Guo","year":"1982","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hu et al., 2015] R. Hu, G. Can, C. Pallan Gayol, G. Krem- pel, J. Spotak, G. Vail, S. Marchand-Maillet, J. Odobez, and D. Gatica-Perez. Multimedia analysis and access of ancient maya epigraphy: Tools to support scholars on maya hieroglyphics. IEEE Signal Processing Magazine, 32(4):75-84, 2015.","title":"Multimedia analysis and access of ancient maya epigraphy: Tools to support scholars on maya hieroglyphics","authors":"R Hu, G Can, C Pallan, G Gayol, J Krempel, G Spotak, S Vail, J Marchand-Maillet, D Odobez, Gatica-Perez","year":"2015","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang, 2019] Tianshu Huang. The Fifth Collection of Ora- cle Bone Rejoinings. The Xue Yuan Book Press, 2019.","title":"The Fifth Collection of Oracle Bone Rejoinings","authors":"Tianshu Huang, Huang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liao et al., 2018] Minghui Liao, Baoguang Shi, and Xiang Bai. Textboxes++: A single-shot oriented scene text de- tector. IEEE Trans. Image Processing, 27(8):3676-3690, 2018.","title":"Textboxes++: A single-shot oriented scene text detector","authors":"Liao","year":"2018","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lin, 2008] Hongming Lin. Zui Gu: Research on Oracle Bone Rejoinings. The Taiwan Book Store Press, 2008.","title":"Research on Oracle Bone Rejoinings","authors":"Hongming Zui Lin, Gu","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Long et al., 2018] Shangbang Long, Xin He, and Cong Yao. Scene text detection and recognition: The deep learning era. arXiv, abs/1811.04256, 2018.","title":"Shangbang Long, Xin He, and Cong Yao. Scene text detection and recognition: The deep learning era","authors":"Long","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Men, 2008] Yi Men. Six groups of newly joined huang type oracle bone inscriptions. Cultural Relics of Central China, 2:88-91, 2008.","title":"Yi Men. Six groups of newly joined huang type oracle bone inscriptions","authors":"Men","year":"2008","venue":"Cultural Relics of Central China","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Men, 2012] Yi Men. Ten groups of rejoinings in the huang type oracle bone inscriptions. Huaxia Archaeology, 4:136-137, 2012.","title":"Yi Men. Ten groups of rejoinings in the huang type oracle bone inscriptions","authors":"Men","year":"2012","venue":"Huaxia Archaeology","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Meng, 2017] Lin Meng. Two-stage recognition for ora- cle bone inscriptions. In Image Analysis and Processing -ICIAP 2017 -19th International Conference, Catania, Italy, September 11-15, 2017, Proceedings, Part II, pages 672-682, 2017.","title":"Two-stage recognition for oracle bone inscriptions","authors":"Lin Meng","year":"2017","venue":"Image Analysis and Processing -ICIAP 2017 -19th International Conference","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mo, 2012] Bofeng Mo. Five groups of assemblages ora- cle bone inscription fragments in oracle bone inscription collection of shanghai museum. Chinese Palace Museum Journal, 4:120-124, 2012.","title":"Bofeng Mo. Five groups of assemblages oracle bone inscription fragments in oracle bone inscription collection of shanghai museum","authors":"Mo","year":"2012","venue":"Chinese Palace Museum Journal","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mo, 2016] Bofeng Mo. Six new oracle bone rejoinings in the anonymous type. Archaeology and Cultural Relics, 1:105-108, 2016.","title":"Bofeng Mo. Six new oracle bone rejoinings in the anonymous type","authors":"Mo","year":"2016","venue":"Archaeology and Cultural Relics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shi et al., 2017] Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell., 39(11):2298-2304, 2017.","title":"An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition","authors":"Shi","year":"2017","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shi et al., 2019] Baoguang Shi, Mingkun Yang, Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai. ASTER: An attentional scene text recognizer with flexible rectifica- tion. IEEE transactions on pattern analysis and machine intelligence, 41(9):2035-2048, 2019.","title":"ASTER: An attentional scene text recognizer with flexible rectification","authors":"Shi","year":"2019","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tian et al., 2016] Zhi Tian, Weilin Huang, Tong He, Pan He, and Yu Qiao. Detecting text in natural image with con- nectionist text proposal network. In 14th European Con- ference on Computer Vision (ECCV 2016), pages 56-72, 2016.","title":"Detecting text in natural image with connectionist text proposal network","authors":"Weilin Zhi Tian, Tong Huang, Pan He, Yu He, Qiao","year":"2016","venue":"14th European Conference on Computer Vision (ECCV 2016)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tsai, 1999] Chemao Tsai. Catalog of Oracle Bone Rejoin- ings. The Le Xue Book Press, 1999.","title":"Catalog of Oracle Bone Rejoinings","authors":"Chemao Tsai, Tsai","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zagoruyko and Komodakis, 2015] Sergey Zagoruyko and Nikos Komodakis. Learning to compare image patches via convolutional neural networks. In CVPR 2015, pages 4353-4361, 2015.","title":"Learning to compare image patches via convolutional neural networks","authors":"Komodakis Zagoruyko, Sergey Zagoruyko, Nikos Komodakis","year":"2015","venue":"CVPR 2015","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang and Wang, 2012] Changqing Zhang and Aimin Wang. Method for computer aided rejoining of bones/tortoise shells rubbing. Electronic Design En- gineering, 20(17):1-3, 2012.","title":"Method for computer aided rejoining of bones/tortoise shells rubbing","authors":"Wang ; Zhang, Changqing Zhang, Aimin Wang","year":"2012","venue":"Electronic Design Engineering","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang, 2019] Weijie Zhang. The importance of oracle re- joining in the study of ancient characters and the history of the shang dynasty. Journal of Chinese Writing Systems, 3(1):19-28, 2019.","title":"The importance of oracle rejoining in the study of ancient characters and the history of the shang dynasty","authors":"Weijie Zhang","year":"2019","venue":"Journal of Chinese Writing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou et al., 2017] Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. EAST: an efficient and accurate scene text detector. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), pages 2642-2651, 2017.","title":"EAST: an efficient and accurate scene text detector","authors":"Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, Jiajun Liang","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}