{"type":"ScholarlyArticle","title":"Enhancing Mixture of Experts with Independent and Collaborative Learning for Long-Tail Visual Recognition","authors":[{"name":"Yanhao Chen","slug":"yanhao-chen","given":"Yanhao","family":"Chen","affiliation":""},{"name":"Zhongquan Jian","slug":"zhongquan-jian","given":"Zhongquan","family":"Jian","affiliation":""},{"name":"Nianxin Ke","slug":"nianxin-ke","given":"Nianxin","family":"Ke","affiliation":""},{"name":"Shuhao Hu","slug":"shuhao-hu","given":"Shuhao","family":"Hu","affiliation":""},{"name":"Junjie Jiao","slug":"junjie-jiao","given":"Junjie","family":"Jiao","affiliation":""},{"name":"Qingqi Hong","slug":"qingqi-hong","given":"Qingqi","family":"Hong","affiliation":""},{"name":"Qingqiang Wu","slug":"qingqiang-wu","given":"Qingqiang","family":"Wu","affiliation":""}],"abstract":"Deep neural networks (DNNs) face substantial challenges in Long-Tail Visual Recognition (LTVR) due to the inherent class imbalances in real-world data distributions. \r\nThe Mixture of Experts (MoE) framework has emerged as a promising approach to addressing these issues.\r\nHowever, in MoE systems, experts are typically trained to optimize a collective objective, often neglecting the individual optimality of each expert. This individual optimality usually contributes to the overall performance, as the goals of different experts are not mutually exclusive.\r\nWe propose the Independent and Collaborative Learning (ICL) framework to optimize each expert independently while ensuring global optimality. \r\nFirst, Diverse Optimization Learning (DOL) is introduced to enhance expert diversity and individual performance. \r\nThen, we conceptualize experts as parallel circuit branches and introduce Competition and Collaboration Learning (CoL). Competition Learning amplifies the gradients of better-performing experts to preserve individual optimality, and Collaboration Learning encourages collaboration through mutual distillation to enhance optimal knowledge sharing.\r\nICL achieves state-of-the-art accuracy in experiments on CIFAR-100/10-LT, ImageNet-LT, and iNaturalist 2018, respectively. Our code is available at https://github.com/PolarisLight/ICL.","doi":"10.24963/ijcai.2025/93","url":"https://ijcai.org/proceedings/2025/93","pdf":"https://ijcai.org/proceedings/2025/0093.pdf","markdown":"https://ijcai.org/proceedings/2025/93.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/93","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Computer Vision","track":"Main Track","number":93,"pages":{"first":828,"last":836},"keywords":["Machine Learning: ML: Classification","Computer Vision: CV: Representation learning","Machine Learning: ML: Representation learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}