{"type":"ScholarlyArticle","title":"Screening, Rectifying, and Re-Screening: A Unified Framework for Tuning Vision-Language Models with Noisy Labels","authors":[{"name":"Chaowei Fang","slug":"chaowei-fang","given":"Chaowei","family":"Fang","affiliation":""},{"name":"Hangfei Ma","slug":"hangfei-ma","given":"Hangfei","family":"Ma","affiliation":""},{"name":"Zhihao Li","slug":"zhihao-li","given":"Zhihao","family":"Li","affiliation":""},{"name":"De Cheng","slug":"de-cheng","given":"De","family":"Cheng","affiliation":""},{"name":"Yue Zhang","slug":"yue-zhang","given":"Yue","family":"Zhang","affiliation":""},{"name":"Guanbin Li","slug":"guanbin-li","given":"Guanbin","family":"Li","affiliation":""}],"abstract":"Pre-trained vision-language models have shown remarkable potential for downstream tasks. However, their fine-tuning under noisy labels remains an open problem due to challenges like self-confirmation bias and the limitations of conventional small-loss criteria. In this paper, we propose a unified framework to address these issues, consisting of three key steps: Screening, Rectifying, and Re-Screening. First, a dual-level semantic matching mechanism is introduced to categorize samples into clean, ambiguous, and noisy samples by leveraging both macro-level and micro-level textual prompts. Second, we design tailored pseudo-labeling strategies to rectify noisy and ambiguous labels, enabling their effective incorporation into the training process. Finally, a re-screening step, utilizing cross-validation with an auxiliary vision-language model, mitigates self-confirmation bias and enhances the robustness of the framework. Extensive experiments across ten datasets demonstrate that the proposed method significantly outperforms existing approaches for tuning vision-language pre-trained models with noisy labels.","doi":"10.24963/ijcai.2025/568","url":"https://ijcai.org/proceedings/2025/568","pdf":"https://ijcai.org/proceedings/2025/0568.pdf","markdown":"https://ijcai.org/proceedings/2025/568.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/568","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":568,"pages":{"first":5101,"last":5109},"keywords":["Machine Learning: ML: Cost-sensitive learning","Computer Vision: CV: Machine learning for vision","Machine Learning: ML: Classification","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}