{"type":"ScholarlyArticle","title":"TLPG-Tracker: Joint Learning of Target Localization and Proposal Generation for Visual Tracking","authors":[{"name":"Siyuan Li","slug":"siyuan-li","given":"Siyuan","family":"Li","affiliation":"Nanjing University , Nanjing , China"},{"name":"Zhi Zhang","slug":"zhi-zhang","given":"Zhi","family":"Zhang","affiliation":"Zhijiang College of Zhejiang University of Technology , Shaoxing , Zhejiang , China"},{"name":"Ziyu Liu","slug":"ziyu-liu","given":"Ziyu","family":"Liu","affiliation":"Zhejiang University of Technology , Hangzhou , China"},{"name":"Anna Wang","slug":"anna-wang","given":"Anna","family":"Wang","affiliation":"Sichuan University , Chengdu , China"},{"name":"Linglong Qiu","slug":"linglong-qiu","given":"Linglong","family":"Qiu","affiliation":"Zhijiang College of Zhejiang University of Technology , Shaoxing , Zhejiang , China"},{"name":"Feng Du","slug":"feng-du","given":"Feng","family":"Du","affiliation":"Zhejiang University of Technology , Hangzhou , China"}],"abstract":"Target localization and proposal generation are two essential subtasks in generic visual tracking, and it is a challenge to address both the two efficiently. In this paper, we propose an efficient two-stage architecture which makes full use of the complementarity of two subtasks to achieve robust localization and high-quality proposals generation of the target jointly. Specifically, our model performs a novel deformable central correlation operation by an online learning model in both two stages to locate new target centers while generating target proposals in the vicinity of these centers. The proposals are refined in the refinement stage to further improve accuracy and robustness. Moreover, the model benefits from multi-level features aggregation in a neck module and a feature enhancement module. We conduct extensive ablation studies to demonstrate the effectiveness of our proposed methods. Our tracker runs at over 30 FPS and sets a new state-of-the-art on five tracking benchmarks, including LaSOT, VOT2018, TrackingNet, GOT10k, OTB2015.","doi":"10.24963/ijcai.2020/99","url":"https://ijcai.org/proceedings/2020/99","pdf":"https://ijcai.org/proceedings/2020/0099.pdf","markdown":"https://ijcai.org/proceedings/2020/99.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/99","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":99,"pages":{"first":708,"last":715},"keywords":["Machine Learning: Deep Learning","Computer Vision: Motion and Tracking","Machine Learning: Deep Learning: Convolutional networks","Computer Vision: Video: Events, Activities and Surveillance"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25319/file_25319.m3u8","length_sec":232,"provider":"hls"}],"references":[{"position":1,"raw":"[Bai et al., 2018] Shuai Bai, Zhiqun He, Ting-Bing Xu, Zheng Zhu, Yuan Dong, and Hongliang Bai. Multi- hierarchical independent correlation filters for visual tracking. arXiv preprint arXiv:1811.10302, 2018.","title":"Multihierarchical independent correlation filters for visual tracking","authors":"Shuai Bai, Zhiqun He, Ting-Bing Xu, Zheng Zhu, Yuan Dong, Hongliang Bai","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bertinetto et al., 2016] Luca Bertinetto, Jack Valmadre, João F. Henriques, Andrea Vedaldi, and Philip H.S. Torr. Fully-convolutional siamese networks for object tracking. In ECCV, pages 850-865, 2016.","title":"Fully-convolutional siamese networks for object tracking","authors":"Luca Bertinetto, Jack Valmadre, João F Henriques, Andrea Vedaldi, Philip H S Torr","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhat et al., 2018] Goutam Bhat, Joakim Johnander, Martin Danelljan, Fahad Shahbaz Khan, and Michael Felsberg. Unveiling the Power of Deep Tracking. In ECCV, pages 493-509, 2018.","title":"Unveiling the Power of Deep Tracking","authors":"Goutam Bhat, Joakim Johnander, Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bhat et al., 2019] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. Learning Discrimina- tive Model Prediction for Tracking. arXiv preprint arXiv:1904.07220, 2019.","title":"Learning Discriminative Model Prediction for Tracking","authors":"Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bolme et al., 2010] David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui. Visual object tracking using adaptive correlation filters. In CVPR, pages 2544- 2550, 2010.","title":"Visual object tracking using adaptive correlation filters","authors":"Ross David S Bolme, Bruce A Beveridge, Yui Man Draper, Lui","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dai et al., 2017] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. De- formable convolutional networks. In ICCV, pages 764- 773, 2017.","title":"Deformable convolutional networks","authors":"Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Danelljan et al., 2016] Martin Danelljan, Andreas Robin- son, Fahad Shahbaz Khan, and Michael Felsberg. Beyond correlation filters: Learning continuous convolution oper- ators for visual tracking. In ECCV, pages 472-488, 2016.","title":"Beyond correlation filters: Learning continuous convolution operators for visual tracking","authors":"Martin Danelljan, Andreas Robinson, Fahad Shahbaz Khan, Michael Felsberg","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Danelljan et al., 2017] Martin Danelljan, Goutam Bhat, Fa- had Shahbaz Khan, and Michael Felsberg. Eco: Efficient convolution operators for tracking. In CVPR, pages 6931- 6939, 2017.","title":"Eco: Efficient convolution operators for tracking","authors":"Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Danelljan et al., 2019] Martin Danelljan, Goutam Bhat, Fa- had Shahbaz Khan, and Michael Felsberg. ATOM: Accu- rate Tracking by Overlap Maximization. In CVPR, 2019.","title":"ATOM: Accurate Tracking by Overlap Maximization","authors":"Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fan et al., 2019] Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. Lasot: A high-quality benchmark for large-scale single object tracking. In CVPR, pages 5374-5383, 2019.","title":"Lasot: A high-quality benchmark for large-scale single object tracking","authors":"Liting Heng Fan, Fan Lin, Peng Yang, Ge Chu, Sijia Deng, Hexin Yu, Yong Bai, Chunyuan Xu, Haibin Liao, Ling","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2017] Qing Guo, Wei Feng, Ce Zhou, Rui Huang, Liang Wan, and Song Wang. Learning dynamic siamese network for visual object tracking. In ICCV, pages 1763-1771, 2017.","title":"Learning dynamic siamese network for visual object tracking","authors":"Qing Guo, Wei Feng, Ce Zhou, Rui Huang, Liang Wan, Song Wang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Held et al., 2016] David Held, Sebastian Thrun, and Silvio Savarese. Learning to track at 100 fps with deep regression networks. In ECCV, pages 749-765, 2016.","title":"Learning to track at 100 fps with deep regression networks","authors":"David Held, Sebastian Thrun, Silvio Savarese","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2018] Lianghua Huang, Xin Zhao, and Kaiqi Huang. Got-10k: A large high-diversity benchmark for generic object tracking in the wild. arXiv preprint arXiv:1810.11981, 2018.","title":"Got-10k: A large high-diversity benchmark for generic object tracking in the wild","authors":"Huang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jiang et al., 2018] Borui Jiang, Ruixuan Luo, Jiayuan Mao, Tete Xiao, and Yuning Jiang. Acquisition of localization confidence for accurate object detection. In ECCV, pages 784-799, 2018.","title":"Acquisition of localization confidence for accurate object detection","authors":"Jiang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kiani Galoogahi et al., 2017] Hamed Kiani Galoogahi, Ashton Fagg, and Simon Lucey. Learning background- aware correlation filters for visual tracking. In ICCV, pages 1135-1143, 2017.","title":"Learning backgroundaware correlation filters for visual tracking","authors":"Kiani Hamed, Ashton Galoogahi, Simon Fagg, Lucey","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Lei Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kristan et al., 2018] Matej Kristan, Ales Leonardis, Jiri Matas, Michael Felsberg, Roman Pflugfelder, Luka Čehovin, Tomas Vojir, Goutam Bhat, Alan Lukežič, Ab- delrahman Eldesokey, Gustavo Fernandez Dominguez, Alvaro Garcia-Martin, Álvaro Iglesias-Arias, A. Alatan, Abel Gonzalez-Garcia, Alfredo Petrosino, Alireza Memar- moghadam, Andrea Vedaldi, and Andrej Muhič. The sixth visual object tracking vot2018 challenge results. In ECCV workshop, pages 3-53, 2018.","title":"The sixth visual object tracking vot2018 challenge results","authors":"Matej Kristan, Ales Leonardis, Jiri Matas, Michael Felsberg, Roman Pflugfelder, Luka Čehovin, Tomas Vojir, Goutam Bhat, Alan Lukežič, Abdelrahman Eldesokey, Gustavo Fernandez Dominguez, Alvaro Garcia-Martin, Álvaro Iglesias-Arias, A Alatan, Abel Gonzalez-Garcia, Alfredo Petrosino, Alireza Memarmoghadam, Andrea Vedaldi, Andrej Muhič","year":"2018","venue":"ECCV workshop","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2018a] Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu. High performance visual tracking with siamese region proposal network. In CVPR, pages 8971- 8980, 2018.","title":"High performance visual tracking with siamese region proposal network","authors":"Li","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2018b] Zeming Li, Chao Peng, Gang Yu, Xiangyu Zhang, Yangdong Deng, and Jian Sun. DetNet: Design backbone for object detection. In ECCV, pages 334-350, 2018.","title":"DetNet: Design backbone for object detection","authors":"Li","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2019] Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, and Junjie Yan. Siamrpn++: Evolution of siamese visual tracking with very deep networks. In CVPR, pages 4282-4291, 2019.","title":"Siamrpn++: Evolution of siamese visual tracking with very deep networks","authors":"Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, Junjie Yan","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lin et al., 2017] Tsung Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Fea- ture pyramid networks for object detection. In CVPR, pages 936-944, 2017.","title":"Feature pyramid networks for object detection","authors":"Tsung Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, Serge Belongie","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Muller et al., 2018] Matthias Muller, Adel Bibi, Silvio Gi- ancola, Salman Alsubaihi, and Bernard Ghanem. Track- ingnet: A large-scale dataset and benchmark for object tracking in the wild. In ECCV, pages 300-317, 2018.","title":"Trackingnet: A large-scale dataset and benchmark for object tracking in the wild","authors":"Matthias Muller, Adel Bibi, Silvio Giancola, Salman Alsubaihi, Bernard Ghanem","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nam and Han, 2016] Hyeonseob Nam and Bohyung Han. Learning multi-domain convolutional neural networks for visual tracking. In CVPR, pages 4293-4302, 2016.","title":"Hyeonseob Nam and Bohyung Han. Learning multi-domain convolutional neural networks for visual tracking","authors":"Nam, Han","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Song et al., 2018] Yibing Song, Chao Ma, Xiaohe Wu, Li- jun Gong, Linchao Bao, Wangmeng Zuo, Chunhua Shen, Rynson WH Lau, and Ming-Hsuan Yang. Vital: Visual tracking via adversarial learning. In CVPR, pages 8990- 8999, 2018.","title":"Vital: Visual tracking via adversarial learning","authors":"Yibing Song, Chao Ma, Xiaohe Wu, Lijun Gong, Linchao Bao, Wangmeng Zuo, Chunhua Shen, Rynson Wh Lau, Ming-Hsuan Yang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun et al., 2018] Chong Sun, Dong Wang, Huchuan Lu, and Ming-Hsuan Yang. Correlation tracking via joint discrim- ination and reliability learning. In CVPR, pages 489-497, 2018.","title":"Correlation tracking via joint discrimination and reliability learning","authors":"Chong Sun, Dong Wang, Huchuan Lu, Ming-Hsuan Yang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tu et al., 2019] Zhengzheng Tu, Ajian Zhou, Bo Jiang, and Bin Luo. Visual object tracking via graph convolutional representation. In ICMEW, pages 234-239, 2019.","title":"Visual object tracking via graph convolutional representation","authors":"Tu","year":"2019","venue":"ICMEW","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Valmadre et al., 2017] Jack Valmadre, Luca Bertinetto, Joao Henriques, Andrea Vedaldi, and Philip HS Torr. End- to-end representation learning for correlation filter based tracking. In CVPR, pages 2805-2813, 2017.","title":"Endto-end representation learning for correlation filter based tracking","authors":"Valmadre","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2019a] Guangting Wang, Chong Luo, Zhiwei Xiong, and Wenjun Zeng. SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object Tracking. arXiv preprint arXiv:1904.04452, 2019.","title":"SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object Tracking","authors":"Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2019b] Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, and Philip HS Torr. Fast online object tracking and segmentation: A unifying approach. In CVPR, pages 1328-1338, 2019.","title":"Fast online object tracking and segmentation: A unifying approach","authors":"Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wu et al., 2015] Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9):1834- 1848, 2015.","title":"Object tracking benchmark","authors":"Yi Wu, Jongwoo Lim, Ming-Hsuan Yang","year":"2015","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2019] Tianyang Xu, Zhenhua Feng, Xiaojun Wu, and Josef Kittler. Learning adaptive discriminative cor- relation filters via temporal consistency preserving spa- tial feature selection for robust visual object tracking. IEEE Transactions on Image Processing, 28(11):5596- 5609, 2019.","title":"Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking","authors":"Tianyang Xu, Zhenhua Feng, Xiaojun Wu, Josef Kittler","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2019] Ze Yang, Shaohui Liu, Han Hu, Li- wei Wang, and Stephen Lin. RepPoints: Point Set Representation for Object Detection. arXiv preprint arXiv:1904.11490, 2019.","title":"RepPoints: Point Set Representation for Object Detection","authors":"Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, Stephen Lin","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2018] Yunhua Zhang, Lijun Wang, Jinqing Qi, Dong Wang, Mengyang Feng, and Huchuan Lu. Struc- tured siamese network for real-time visual tracking. In ECCV, pages 351-366, 2018.","title":"Structured siamese network for real-time visual tracking","authors":"Zhang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhu et al., 2018] Zheng Zhu, Qiang Wang, Bo Li, Wei Wu, Junjie Yan, and Weiming Hu. Distractor-aware siamese networks for visual object tracking. In ECCV, pages 103- 119, 2018.","title":"Distractor-aware siamese networks for visual object tracking","authors":"Zheng Zhu, Qiang Wang, Bo Li, Wei Wu, Junjie Yan, Weiming Hu","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhu et al., 2019] Xizhou Zhu, Dazhi Cheng, Zheng Zhang, Stephen Lin, and Jifeng Dai. An empirical study of spatial attention mechanisms in deep networks. arXiv preprint arXiv:1904.05873, 2019.","title":"An empirical study of spatial attention mechanisms in deep networks","authors":"Xizhou Zhu, Dazhi Cheng, Zheng Zhang, Stephen Lin, Jifeng Dai","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}