{"type":"ScholarlyArticle","title":"SparseTT: Visual Tracking with Sparse Transformers","authors":[{"name":"Zhihong Fu","slug":"zhihong-fu","given":"Zhihong","family":"Fu","affiliation":""},{"name":"Zehua Fu","slug":"zehua-fu","given":"Zehua","family":"Fu","affiliation":""},{"name":"Qingjie Liu","slug":"qingjie-liu","given":"Qingjie","family":"Liu","affiliation":""},{"name":"Wenrui Cai","slug":"wenrui-cai","given":"Wenrui","family":"Cai","affiliation":""},{"name":"Yunhong Wang","slug":"yunhong-wang","given":"Yunhong","family":"Wang","affiliation":""}],"abstract":"Transformers have been successfully applied to the visual tracking task and significantly promote tracking performance. The self-attention mechanism designed to model long-range dependencies is the key to the success of Transformers. However, self-attention lacks focusing on the most relevant information in the search regions, making it easy to be distracted by background. In this paper, we relieve this issue with a sparse attention mechanism by focusing the most relevant information in the search regions, which enables a much accurate tracking. Furthermore, we introduce a double-head predictor to boost the accuracy of foreground-background classification and regression of target bounding boxes, which further improve the tracking performance. Extensive experiments show that, without bells and whistles, our method significantly outperforms the state-of-the-art approaches on LaSOT, GOT-10k, TrackingNet, and UAV123, while running at 40 FPS. Notably, the training time of our method is reduced by 75% compared to that of TransT. The source code and models are available at https://github.com/fzh0917/SparseTT.","doi":"10.24963/ijcai.2022/127","url":"https://ijcai.org/proceedings/2022/127","pdf":"https://ijcai.org/proceedings/2022/0127.pdf","markdown":"https://ijcai.org/proceedings/2022/127.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/127","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":127,"pages":{"first":905,"last":912},"keywords":["Computer Vision: Motion and Tracking"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987033","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985176","length_sec":605,"provider":"slideslive"}],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}