{"type":"ScholarlyArticle","title":"MedualTime: A Dual-Adapter Language Model for Medical Time Series-Text Multimodal Learning","authors":[{"name":"Jiexia Ye","slug":"jiexia-ye","given":"Jiexia","family":"Ye","affiliation":""},{"name":"Weiqi Zhang","slug":"weiqi-zhang","given":"Weiqi","family":"Zhang","affiliation":""},{"name":"Ziyue Li","slug":"ziyue-li","given":"Ziyue","family":"Li","affiliation":""},{"name":"Jia Li","slug":"jia-li","given":"Jia","family":"Li","affiliation":""},{"name":"Meng Zhao","slug":"meng-zhao","given":"Meng","family":"Zhao","affiliation":""},{"name":"Fugee Tsung","slug":"fugee-tsung","given":"Fugee","family":"Tsung","affiliation":""}],"abstract":"The recent rapid advancements in language models (LMs) have garnered attention in medical time series-text multimodal learning.\r\nHowever, existing contrastive learning-based and prompt-based LM approaches tend to be biased, often assigning a primary role to time series modality while treating text modality as secondary. We classify these approaches under a temporal-primary paradigm, which may overlook the unique and critical task-relevant information embedded in text modality like clinical reports, thus failing to fully leverage mutual benefits and complementarity of different modalities.\r\nTo fill this gap, we propose a novel textual-temporal multimodal learning paradigm that enables either modality to serve as the primary while being enhanced by the other, thereby effectively capturing modality-specific information and fostering cross-modal interaction. In specific, we design MedualTime, a language model composed of dual adapters to implement temporal-primary and textual-primary modeling simultaneously. Within each adapter, lightweight adaptation tokens are injected into the top layers of LM to encourage high-level modality fusion. The shared LM pipeline by dual adapters not only achieves adapter alignment but also enables efficient fine-tuning, reducing computational resources. Empirically, MedualTime demonstrates superior performance on medical data, achieving notable improvements of 8% accuracy and 12% F1 in supervised settings. \r\nFurthermore, MedualTime's transferability is validated by \r\nfew-shot transfer experiments from coarse-grained to fine-grained medical data.","doi":"10.24963/ijcai.2025/880","url":"https://ijcai.org/proceedings/2025/880","pdf":"https://ijcai.org/proceedings/2025/0880.pdf","markdown":"https://ijcai.org/proceedings/2025/880.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/880","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":880,"pages":{"first":7913,"last":7921},"keywords":["Machine Learning: ML: Multi-modal learning","Multidisciplinary Topics and Applications: MTA: Health and medicine","Machine Learning: ML: Foundation models","Machine Learning: ML: Classification"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}