{"type":"ScholarlyArticle","title":"Open-Set Test-Time Adaptation in VLMs via Noise-Immune Self-Purification","authors":[{"name":"Mingxu Feng","slug":"mingxu-feng","given":"Mingxu","family":"Feng","affiliation":""},{"name":"Fengqiang Wan","slug":"fengqiang-wan","given":"Fengqiang","family":"Wan","affiliation":""},{"name":"Yang Yang","slug":"yang-yang","given":"Yang","family":"Yang","affiliation":""}],"abstract":"Unlike test-time adaptation (TTA), open-set TTA (OSTTA) aims to robustly adapt to in-distribution (ID) domain shifts while suppressing the detrimental impact of out-of-distribution (OOD) samples encountered at test time. To this end, we propose a novel OSTTA approach named Noise-Immune Self-Purification (NISP). By exploiting the zero-shot priors of pre-trained vision-language models (VLMs), NISP advances from coarse pseudo-labeling to fine-grained, noise-resilient adaptation. Technically, we first introduce a dual-cluster Bayesian Gaussian mixture model to fit VLMs-derived scores, achieving coarse pseudo-ID/OOD separation via posterior-risk thresholding. Subsequently, NISP constructs a noise-immune fine-grained adaptation where the adapter enforces consensus-discrepancy constraints to refine coarse pseudo-labels and suppress noise propagation. We then devise the Jaccard Consistency Score for OOD discrimination. Overall the coarse-to-fine pipeline enables rigorous self-purification and robust online adaptation. Theoretically we show that consensus-discrepancy losses mitigate the deleterious effects of noise. Empirically, NISP achieves state-of-the-art results across multiple OSTTA benchmarks, validating its efficacy. The code is available at https://github.com/njustkmg/IJCAI26-NISP.","doi":"10.24963/ijcai.2026/477","url":"https://ijcai.org/proceedings/2026/477","pdf":"https://ijcai.org/proceedings/2026/0477.pdf","markdown":"https://ijcai.org/proceedings/2026/477.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/477","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Machine Learning","track":"Main Track","number":477,"pages":{"first":4286,"last":4294},"keywords":["Machine Learning: Unsupervised learning","Machine Learning: Open-World/Open-Set/OOD Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}