{"type":"ScholarlyArticle","title":"DERI: Cross-Modal ECG Representation Learning with Deep ECG-Report Interaction","authors":[{"name":"Jian Chen","slug":"jian-chen","given":"Jian","family":"Chen","affiliation":""},{"name":"Xiaoru Dong","slug":"xiaoru-dong","given":"Xiaoru","family":"Dong","affiliation":""},{"name":"Wei Wang","slug":"wei-wang","given":"Wei","family":"Wang","affiliation":""},{"name":"Shaorui Zhou","slug":"shaorui-zhou","given":"Shaorui","family":"Zhou","affiliation":""},{"name":"Lequan Yu","slug":"lequan-yu","given":"Lequan","family":"Yu","affiliation":""},{"name":"Xiping Hu","slug":"xiping-hu","given":"Xiping","family":"Hu","affiliation":""}],"abstract":"Electrocardiogram (ECG) is widely used to diagnose cardiac conditions via deep learning methods. Although existing self-supervised learning (SSL) methods have achieved great performance in learning representation for ECG-based cardiac conditions classification, the clinical semantics can not be effectively captured. To overcome this limitation, we proposed to learn cross-modal ECG representations that contain more clinical semantics via a novel framework with \\textbf{D}eep \\textbf{E}CG-\\textbf{R}eport \\textbf{I}nteraction (\\textbf{DERI}). Specifically, we design a novel framework combining multiple alignments and mutual feature reconstructions to learn effective representation of the ECG with the clinical report, which fuses the clinical semantics of the report. An RME-Module inspired by masked modeling is proposed to improve the ECG representation learning. Furthermore, we extend ECG representation learning to report generation with a language model, which is significant for evaluating clinical semantics in the learned representations and even clinical applications. Comprehensive experiments with various settings are conducted on various datasets to show the superior performance of our DERI. Our code is released on https://github.com/cccccj-03/DERI.","doi":"10.24963/ijcai.2025/537","url":"https://ijcai.org/proceedings/2025/537","pdf":"https://ijcai.org/proceedings/2025/0537.pdf","markdown":"https://ijcai.org/proceedings/2025/537.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/537","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":537,"pages":{"first":4824,"last":4832},"keywords":["Data Mining: DM: Mining spatial and/or temporal data","Machine Learning: ML: Classification","Machine Learning: ML: Self-supervised Learning","Machine Learning: ML: Multi-modal learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}