Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval

Liu, Weijia; Cao, Jiuxin; Miao, Bo; Fu, Zhiheng; Zhu, Xuelin; Ge, Jiawei; Liu, Bo; Nasim, Mehwish; Mian, Ajmal

doi:10.24963/ijcai.2025/180

Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval

Weijia Liu, Jiuxin Cao, Bo Miao, Zhiheng Fu, Xuelin Zhu, Jiawei Ge, Bo Liu, Mehwish Nasim, Ajmal Mian

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 1612-1620.

https://doi.org/10.24963/ijcai.2025/180

Keywords

Computer Vision: CV: Multimodal learning
Computer Vision: CV: Image and video retrieval

Abstract

Current text-driven Video Moment Retrieval (VMR) methods encode all video clips, including irrelevant ones, disrupting multimodal alignment and hindering optimization. To this end, we propose a denoise-then-retrieve paradigm that explicitly filters text-irrelevant clips from videos and then retrieves the target moment using purified multimodal representations. Following this paradigm, we introduce the Denoise-then-Retrieve Network (DRNet), comprising Text-Conditioned Denoising (TCD) and Text-Reconstruction Feedback (TRF) modules. TCD integrates cross-attention and structured state space blocks to dynamically identify noisy clips and produce a noise mask to purify multimodal video representations. TRF further distills a single query embedding from purified video representations and aligns it with the text embedding, serving as auxiliary supervision for denoising during training. Finally, we perform conditional retrieval using text embeddings on purified video representations for accurate VMR. Experiments on Charades-STA and QVHighlights demonstrate that our approach surpasses state-of-the-art methods on all metrics. Furthermore, our denoise-then-retrieve paradigm is adaptable and can be seamlessly integrated into advanced VMR models to boost performance.

Weijia Liu, Jiuxin Cao, Bo Miao, Zhiheng Fu, Xuelin Zhu, Jiawei Ge, Bo Liu, Mehwish Nasim, and Ajmal Mian. 2025. Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 1612–1620. https://doi.org/10.24963/ijcai.2025/180

@inproceedings{ijcai2025p180,
  title     = {Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval},
  author    = {Liu, Weijia and Cao, Jiuxin and Miao, Bo and Fu, Zhiheng and Zhu, Xuelin and Ge, Jiawei and Liu, Bo and Nasim, Mehwish and Mian, Ajmal},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {1612--1620},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/180},
  url       = {https://doi.org/10.24963/ijcai.2025/180},
}

TY  - CPAPER
TI  - Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval
AU  - Liu, Weijia
AU  - Cao, Jiuxin
AU  - Miao, Bo
AU  - Fu, Zhiheng
AU  - Zhu, Xuelin
AU  - Ge, Jiawei
AU  - Liu, Bo
AU  - Nasim, Mehwish
AU  - Mian, Ajmal
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 1612
EP  - 1620
DO  - 10.24963/ijcai.2025/180
UR  - https://doi.org/10.24963/ijcai.2025/180
ER  -