{"type":"ScholarlyArticle","title":"FediLoRA: Practical Federated Fine-Tuning of Foundation Models Under Missing-Modality Constraints","authors":[{"name":"Lishan Yang","slug":"lishan-yang","given":"Lishan","family":"Yang","affiliation":""},{"name":"Wei Emma Zhang","slug":"wei-emma-zhang","given":"Wei Emma","family":"Zhang","affiliation":""},{"name":"Nam Kha Nguyen","slug":"nam-kha-nguyen","given":"Nam Kha","family":"Nguyen","affiliation":""},{"name":"Po Hu","slug":"po-hu","given":"Po","family":"Hu","affiliation":""},{"name":"Yanjun Shu","slug":"yanjun-shu","given":"Yanjun","family":"Shu","affiliation":""},{"name":"Weitong Chen","slug":"weitong-chen","given":"Weitong","family":"Chen","affiliation":""},{"name":"Sim Mong Yuan","slug":"sim-mong-yuan","given":"Sim","family":"Mong Yuan","affiliation":""}],"abstract":"Federated Learning with LoRA fine-tuning offers an efficient and privacy-aware solution for institutions to collaboratively leverage their large datasets to train VLLMs. However, participating institutions often possess heterogeneous computational resources, resulting in imbalanced LoRA ranks, which pose a major challenge for effective collaboration. In addition, real-world applications in domains such as healthcare and transportation frequently suffer from missing modalities due to user mistakes or device failures, which significantly degrade global model performance in federated settings. To the best of our knowledge, no prior work has addressed these two challenges simultaneously in federated VLLMs. To tackle these issues, we propose FediLoRA, a lightweight federated LoRA aggregation framework that effectively mitigates the impact of missing modalities in heterogeneous environment. FediLoRA is explicitly motivated by the observation that simple averaging and structured editing can jointly benefit both global and personalized models. Our approach achieves strong performance across multiple general-domain and medical-domain benchmark datasets. Additional experiments on healthcare data further demonstrate that FediLoRA is well-suited for practical, real-world deployment scenarios. Our code is released at https://github.com/gotobcn8/FediLoRA.","doi":"10.24963/ijcai.2026/775","url":"https://ijcai.org/proceedings/2026/775","pdf":"https://ijcai.org/proceedings/2026/0775.pdf","markdown":"https://ijcai.org/proceedings/2026/775.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/775","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"AI and Health","track":"AI and Health","number":775,"pages":{"first":6966,"last":6974},"keywords":["Federated learning: Federated learning","AI4H: Multimodal data","LLM in medicine: LLM in medicine"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}