TEST-V: TEst-time Support-set Tuning for Zero-shot Video Classification

Yan, Rui; Wang, Jin; Qu, Hongyu; Du, Xiaoyu; Zhang, Dong; Tang, Jinhui; Tan, Tieniu

doi:10.24963/ijcai.2025/239

TEST-V: TEst-time Support-set Tuning for Zero-shot Video Classification

Rui Yan, Jin Wang, Hongyu Qu, Xiaoyu Du, Dong Zhang, Jinhui Tang, Tieniu Tan

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 2143-2151.

https://doi.org/10.24963/ijcai.2025/239

Keywords

Computer Vision: CV: Video analysis and understanding
Computer Vision: CV: Vision, language and reasoning

Abstract

Recently, adapting Vision Language Models (VLMs) to zero-shot visual classification by tuning class embedding with a few prompts (Test-time Prompt Tuning, TPT) or replacing class names with generated visual samples (support-set) has shown promising results. However, TPT cannot avoid the semantic gap between modalities while the support-set cannot be tuned. To this end, we draw on each other's strengths and propose a novel framework, namely TEst-time Support-set Tuning for zero-shot Video Classification (TEST-V). It first dilates the support-set with multiple prompts (Multi-prompting Support-set Dilation, MSD) and then erodes the support-set via learnable weights to mine key cues dynamically (Temporal-aware Support-set Erosion, TSE). Specifically, i) MSD expands the support samples for each class based on multiple prompts inquired from LLMs to enrich the diversity of the support-set. ii) TSE tunes the support-set with factorized learnable weights according to the temporal prediction consistency in a self-supervised manner to dig pivotal supporting cues for each class. TEST-V achieves state-of-the-art results across four benchmarks and shows good interpretability.

Rui Yan, Jin Wang, Hongyu Qu, Xiaoyu Du, Dong Zhang, Jinhui Tang, and Tieniu Tan. 2025. TEST-V: TEst-time Support-set Tuning for Zero-shot Video Classification. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 2143–2151. https://doi.org/10.24963/ijcai.2025/239

@inproceedings{ijcai2025p239,
  title     = {TEST-V: TEst-time Support-set Tuning for Zero-shot Video Classification},
  author    = {Yan, Rui and Wang, Jin and Qu, Hongyu and Du, Xiaoyu and Zhang, Dong and Tang, Jinhui and Tan, Tieniu},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {2143--2151},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/239},
  url       = {https://doi.org/10.24963/ijcai.2025/239},
}

TY  - CPAPER
TI  - TEST-V: TEst-time Support-set Tuning for Zero-shot Video Classification
AU  - Yan, Rui
AU  - Wang, Jin
AU  - Qu, Hongyu
AU  - Du, Xiaoyu
AU  - Zhang, Dong
AU  - Tang, Jinhui
AU  - Tan, Tieniu
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 2143
EP  - 2151
DO  - 10.24963/ijcai.2025/239
UR  - https://doi.org/10.24963/ijcai.2025/239
ER  -