{"type":"ScholarlyArticle","title":"Modality-Guided Dynamic Graph Fusion and Temporal Diffusion for Self-Supervised RGB-T Tracking","authors":[{"name":"Shenglan Li","slug":"shenglan-li","given":"Shenglan","family":"Li","affiliation":""},{"name":"Rui Yao","slug":"rui-yao","given":"Rui","family":"Yao","affiliation":""},{"name":"Yong Zhou","slug":"yong-zhou","given":"Yong","family":"Zhou","affiliation":""},{"name":"Hancheng Zhu","slug":"hancheng-zhu","given":"Hancheng","family":"Zhu","affiliation":""},{"name":"Kunyang Sun","slug":"kunyang-sun","given":"Kunyang","family":"Sun","affiliation":""},{"name":"Bing Liu","slug":"bing-liu","given":"Bing","family":"Liu","affiliation":""},{"name":"Zhiwen Shao","slug":"zhiwen-shao","given":"Zhiwen","family":"Shao","affiliation":""},{"name":"Jiaqi Zhao","slug":"jiaqi-zhao","given":"Jiaqi","family":"Zhao","affiliation":""}],"abstract":"To reduce the reliance on large-scale annotations, self-supervised RGB-T tracking approaches have garnered significant attention. However, the omission of the object region by erroneous pseudo-label or the introduction of background noise affects the efficiency of modality fusion, while pseudo-label noise triggered by similar object noise can further affect the tracking performance. In this paper, we propose GDSTrack, a novel approach that introduces dynamic graph fusion and temporal diffusion to address the above challenges in self-supervised RGB-T tracking. GDSTrack dynamically fuses the modalities of neighboring frames, treats them as distractor noise, and leverages the denoising capability of a generative model. Specifically, by constructing an adjacency matrix via an Adjacency Matrix Generator (AMG), the proposed Modality-guided Dynamic Graph Fusion (MDGF) module uses a dynamic adjacency matrix to guide graph attention, focusing on and fusing the object’s coherent regions. Temporal Graph-Informed Diffusion (TGID) models MDGF features from neighboring frames as interference, and thus improving robustness against similar-object noise. Extensive experiments conducted on four public RGB-T tracking datasets demonstrate that GDSTrack outperforms the existing state-of-the-art methods. \r\nThe source code is available at https://github.com/LiShenglana/GDSTrack.","doi":"10.24963/ijcai.2025/159","url":"https://ijcai.org/proceedings/2025/159","pdf":"https://ijcai.org/proceedings/2025/0159.pdf","markdown":"https://ijcai.org/proceedings/2025/159.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/159","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Computer Vision","track":"Main Track","number":159,"pages":{"first":1422,"last":1430},"keywords":["Machine Learning: ML: Multi-modal learning","Machine Learning: ML: Unsupervised learning","Computer Vision: CV: Motion and tracking"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}