Three Minds, One Student: Online Multi-Teacher Knowledge Distillation for Multimodal Recommenders

Abstract

Existing multimodal recommendation models using complex fusion mechanisms (e.g., attention) or multi-stage processes (e.g., early or late fusion) integrate different modalities. However, attention-based adaptive fusion is prone to shortcut learning, where dominant collaborative signals (ID) can overshadow other modalities. This dominance affects the entire fusion process: early fusion often amplifies biases driven by identity, while late fusion struggles to extract preference-relevant signals from misaligned modalities, even with alignment regularization. To address these issues, we propose Multi-Teacher Single-Student Online Distillation for Multimodal Recommendation (MTS2-4MM), which reframes the multimodal recommendation task from direct fusion to controllable knowledge transfer. Specifically, we construct multiple teachers to specialize in complementary perspectives, and a unified student distills their guidance via objectives at both the ranking and representation levels. This design explicitly controls modality contributions, improves robustness to modality noise and misalignment. Furthermore, we design a Modality-specific Preference Extractor to explicitly extract user preferences across different modalities equally. Extensive experiments across five real-world datasets demonstrate that MTS2-4MM consistently outperforms state-of-the-art baselines, achieving improvements of up to 7.22%.