Proceedings

FedAMB: Adaptive Modality Balancing for Dominance-Robust Multimodal Federated Distillation

Seungjin Han, Juyeob Lee, Sangmin Lee, Eunil Park

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 4330-4338.

https://doi.org/10.24963/ijcai.2026/482

Keywords

  • Machine Learning: Multi-modal learning
  • Machine Learning: Federated learning
  • Machine Learning: Classification

Abstract

Federated knowledge distillation (Fed-KD) exchanges distilled predictions on a shared proxy dataset, often reducing communication and accommodating heterogeneous client architectures. However, in multimodal federated learning (MFL), modality dominance can bias local optimization and contaminate the aggregated global teacher, degrading both multimodal accuracy and robustness to missing modalities, especially under non-IID heterogeneity. We propose Federated Adaptive-Modality Balancing (FedAMB). At the client side, Selective-Modality Regulation (SMR) models dominance as a state-dependent phenomenon and intervenes only when it destabilizes training, strengthening weak modalities without over-regularization. At the server side, Component-wise Modality Distillation (CMD) regulates how aggregated knowledge is transferred to each modality branch, preventing the propagation of fusion-biased teachers while directly improving unimodal representations. Experiments on CREMA-D, AVE, and UR-FUNNY show that FedAMB consistently improves multimodal accuracy and missing-modality robustness.

Seungjin Han, Juyeob Lee, Sangmin Lee, and Eunil Park. 2026. FedAMB: Adaptive Modality Balancing for Dominance-Robust Multimodal Federated Distillation. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 4330–4338. https://doi.org/10.24963/ijcai.2026/482
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards