{"type":"ScholarlyArticle","title":"Structure-Aware Handwritten Text Recognition via Graph-Enhanced Cross-Modal Mutual Learning","authors":[{"name":"Ji Gan","slug":"ji-gan","given":"Ji","family":"Gan","affiliation":""},{"name":"Yupeng Zhou","slug":"yupeng-zhou","given":"Yupeng","family":"Zhou","affiliation":""},{"name":"Yanming Zhang","slug":"yanming-zhang","given":"Yanming","family":"Zhang","affiliation":""},{"name":"Jiaxu Leng","slug":"jiaxu-leng","given":"Jiaxu","family":"Leng","affiliation":""},{"name":"Xinbo Gao","slug":"xinbo-gao","given":"Xinbo","family":"Gao","affiliation":""}],"abstract":"Existing handwriting recognition methods only focus on learning visual patterns by modeling low-level relationships of adjacent pixels, while overlooking the intrinsic geometric structures of characters. In this paper, we propose a novel graph-enhanced cross-modal mutual learning network GCM to fully process handwritten text images alongside their corresponding geometric graphs, which consists of one shared cross-modal encoder and two parallel inverse decoders. Specifically, the encoder simultaneously extracts visual and geometric information from the cross-modal inputs, and the decoders fuse the multi-modal features for prediction under the guidance of cross-modal fusion. Moreover, two parallel decoders sequentially aggregate cross-modal features in inverse orders (V→G and G→V) but are enhanced through mutual distillation at each time-step, which involves one-to-one knowledge transfer and fully leverages complementary cross-modal information from both directions. Notably, only one branch of GCM is activated in inference, thus avoiding the increase of the model parameters and computation costs for testing. Experiments show that our method outperforms previous state-of-the-art methods on public benchmarks such as IAM, RIMES, and ICDAR-2013 when no extra training data is utilized.","doi":"10.24963/ijcai.2025/574","url":"https://ijcai.org/proceedings/2025/574","pdf":"https://ijcai.org/proceedings/2025/0574.pdf","markdown":"https://ijcai.org/proceedings/2025/574.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/574","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":574,"pages":{"first":5154,"last":5162},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)","Machine Learning: ML: Classification","Machine Learning: ML: Multi-modal learning","Machine Learning: ML: Attention models"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}