{"type":"ScholarlyArticle","title":"Connecting Giants: Synergistic Knowledge Transfer of Large Multimodal Models for Few-Shot Learning","authors":[{"name":"Hao Tang","slug":"hao-tang","given":"Hao","family":"Tang","affiliation":""},{"name":"Shengfeng He","slug":"shengfeng-he","given":"Shengfeng","family":"He","affiliation":""},{"name":"Jing Qin","slug":"jing-qin","given":"Jing","family":"Qin","affiliation":""}],"abstract":"Few-shot learning (FSL) addresses the challenge of classifying novel classes with limited training samples. While some methods leverage semantic knowledge from smaller-scale models to mitigate data scarcity, these approaches often introduce noise and bias due to the data’s inherent simplicity. In this paper, we propose a novel framework, Synergistic Knowledge Transfer (SynTrans), which effectively transfers diverse and complementary knowledge from large multimodal models to empower the off-the-shelf few-shot learner. Specifically, SynTrans employs CLIP as a robust teacher and uses a few-shot vision encoder as a weak student, distilling semantic-aligned visual knowledge via an unsupervised proxy task. Subsequently, a training-free synergistic knowledge mining module facilitates collaboration among large multimodal models to extract high-quality semantic knowledge. Building upon this, a visual-semantic bridging module enables bi-directional knowledge transfer between visual and semantic spaces, transforming explicit visual and implicit semantic knowledge into category-specific classifier weights. Finally, SynTrans introduces a visual weight generator and a semantic weight reconstructor to adaptively construct optimal multimodal FSL classifiers. Experimental results on four FSL datasets demonstrate that SynTrans, even when paired with a simple few-shot vision encoder, significantly outperforms current state-of-the-art methods.","doi":"10.24963/ijcai.2025/693","url":"https://ijcai.org/proceedings/2025/693","pdf":"https://ijcai.org/proceedings/2025/0693.pdf","markdown":"https://ijcai.org/proceedings/2025/693.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/693","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":693,"pages":{"first":6227,"last":6235},"keywords":["Machine Learning: ML: Meta-learning","Machine Learning: ML: Few-shot learning","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning","Computer Vision: CV: Multimodal learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}