{"type":"ScholarlyArticle","title":"Do not Lose the Details: Reinforced Representation Learning for High Performance Visual Tracking","authors":[{"name":"Qiang Wang","slug":"qiang-wang","given":"Qiang","family":"Wang","affiliation":"University of Chinese Academy of Sciences; National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Mengdan Zhang","slug":"mengdan-zhang","given":"Mengdan","family":"Zhang","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Junliang Xing","slug":"junliang-xing","given":"Junliang","family":"Xing","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Jin Gao","slug":"jin-gao","given":"Jin","family":"Gao","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Weiming Hu","slug":"weiming-hu","given":"Weiming","family":"Hu","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Steve Maybank","slug":"steve-maybank","given":"Steve","family":"Maybank","affiliation":"Department of Computer Science and Information Systems , Birkbeck College , University of London"}],"abstract":"This work presents a novel end-to-end trainable CNN model for high performance visual object tracking. It learns both low-level fine-grained representations and a high-level semantic embedding space in a mutual reinforced way, and a multi-task learning strategy is proposed to perform the correlation analysis on representations from both levels. In particular, a fully convolutional encoder-decoder network is designed to reconstruct the original visual features from the semantic projections to preserve all the geometric information. Moreover, the correlation filter layer working on the fine-grained representations leverages a global context constraint for accurate object appearance modeling. The correlation filter in this layer is updated online efficiently without network fine-tuning. Therefore, the proposed tracker benefits from two complementary effects: the adaptability of the fine-grained correlation analysis and the generalization capability of the semantic embedding. Extensive experimental evaluations on four popular benchmarks demonstrate its state-of-the-art performance.","doi":"10.24963/ijcai.2018/137","url":"https://ijcai.org/proceedings/2018/137","pdf":"https://ijcai.org/proceedings/2018/0137.pdf","markdown":"https://ijcai.org/proceedings/2018/137.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/137","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":137,"pages":{"first":985,"last":991},"keywords":["Computer Vision: Computer Vision","Computer Vision: Motion and Tracking"],"videos":[],"references":[{"position":1,"raw":"[Bertinetto et al., 2016] Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. Fully- convolutional siamese networks for object tracking. In ECCV workshop, pages 850-865, 2016.","title":"Fullyconvolutional siamese networks for object tracking","authors":"Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, Philip Hs Torr","year":"2016","venue":"ECCV workshop","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Danelljan et al., 2014a] Martin Danelljan, Gustav Häger, Fahad Khan, and Michael Felsberg. Accurate scale estimation for ro- bust visual tracking. In BMVC, pages 65.1-65.11, 2014.","title":"Accurate scale estimation for robust visual tracking","authors":"Danelljan","year":"2014","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Danelljan et al., 2014b] Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg, and Joost Van de Weijer. Adaptive color at- tributes for real-time visual tracking. In CVPR, pages 1090-1097, 2014.","title":"Adaptive color attributes for real-time visual tracking","authors":"Danelljan","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Danelljan et al., 2015a] Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg. Convolutional features for correlation filter based visual tracking. In ICCV workshop, pages 58-66, 2015.","title":"Convolutional features for correlation filter based visual tracking","authors":"Danelljan","year":"2015","venue":"ICCV workshop","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Danelljan et al., 2015b] Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg. Learning spatially reg- ularized correlation filters for visual tracking. In ICCV, pages 4310-4318, 2015.","title":"Learning spatially regularized correlation filters for visual tracking","authors":"Danelljan","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan and Ling, 2017] Heng Fan and Haibin Ling. Parallel tracking and verifying: A framework for real-time and high accuracy vi- sual tracking. In ICCV, pages 5487-5495, 2017.","title":"Parallel tracking and verifying: A framework for real-time and high accuracy visual tracking","authors":"Heng Fan, Haibin Ling","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Girshick, 2015] Ross Girshick. Fast R-CNN. In ICCV, pages 1440-1448, 2015.","title":"Fast R-CNN","authors":"Ross Girshick","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Held et al., 2016] David Held, Sebastian Thrun, and Silvio Savarese. Learning to track at 100 fps with deep regression net- works. In ECCV, pages 749-765, 2016.","title":"Learning to track at 100 fps with deep regression networks","authors":"David Held, Sebastian Thrun, Silvio Savarese","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Henriques et al., 2015] João F Henriques, Rui Caseiro, Pedro Mar- tins, and Jorge Batista. High-speed tracking with kernelized cor- relation filters. TPAMI, 37(3):583-596, 2015.","title":"High-speed tracking with kernelized correlation filters","authors":"Rui João F Henriques, Pedro Caseiro, Jorge Martins, Batista","year":"2015","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hong et al., 2015a] Seunghoon Hong, Tackgeun You, Suha Kwak, and Bohyung Han. Online tracking by learning discriminative saliency map with convolutional neural network. In ICML, pages 597-606, 2015.","title":"Online tracking by learning discriminative saliency map with convolutional neural network","authors":"Hong","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hong et al., 2015b] Zhibin Hong, Zhe Chen, Chaohui Wang, X- ue Mei, Danil Prokhorov, and Dacheng Tao. Multi-store track- er (muster): A cognitive psychology inspired approach to object tracking. In CVPR, pages 749-758, 2015.","title":"Multi-store tracker (muster): A cognitive psychology inspired approach to object tracking","authors":"Hong","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kalal et al., 2010] Zdenek Kalal, Jiri Matas, and Krystian Mikola- jczyk. Pn learning: Bootstrapping binary classifiers by structural constraints. In CVPR, pages 49-56, 2010.","title":"Pn learning: Bootstrapping binary classifiers by structural constraints","authors":"Zdenek Kalal, Jiri Matas, Krystian Mikolajczyk","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kiani Galoogahi et al., 2017] Hamed Kiani Galoogahi, Ashton Fagg, and Simon Lucey. Learning background-aware correlation filters for visual tracking. In ICCV, pages 1144 -1152, 2017.","title":"Learning background-aware correlation filters for visual tracking","authors":"Kiani Hamed, Ashton Galoogahi, Simon Fagg, Lucey","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kristan et al., 2015] M. Kristan, J. Matas, A. Leonardis, M. Fels- berg, L. ˇCehovin, and G. Fern ´. The visual object tracking vot2015 challenge results. In ICCV workshop, pages 564-586, 2015.","title":"The visual object tracking vot2015 challenge results","authors":"M Kristan, J Matas, A Leonardis, M Felsberg, L, G Fern","year":"2015","venue":"ICCV workshop","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kristan et al., 2017] Matej Kristan, Ales Leonardis, Jiri Matas, Michael Felsberg, Roman Pflugfelder, and Luka Cehovin Zajc. The visual object tracking vot2017 challenge results. In ICCV workshop, pages 1949-1972, 2017.","title":"The visual object tracking vot2017 challenge results","authors":"Matej Kristan, Ales Leonardis, Jiri Matas, Michael Felsberg, Roman Pflugfelder, Luka Cehovin, Zajc","year":"2017","venue":"ICCV workshop","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Ge- offrey E Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li and Zhu, 2014] Yang Li and Jianke Zhu. A scale adaptive kernel correlation filter tracker with feature integration. In ECCV, pages 254-265, 2014.","title":"A scale adaptive kernel correlation filter tracker with feature integration","authors":"Zhu ; ] Li, Li Yang, Jianke Zhu","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431-3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Long","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lukezic et al., 2017] Alan Lukezic, Tomas Vojir, Luka Ce- hovin Zajc, Jiri Matas, and Matej Kristan. Discriminative cor- relation filter with channel and spatial reliability. In CVPR, pages 4847-4856, 2017.","title":"Discriminative correlation filter with channel and spatial reliability","authors":"Alan Lukezic, Tomas Vojir, Cehovin Luka, Jiri Zajc, Matej Matas, Kristan","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ma et al., 2015a] Chao Ma, Jia-Bin Huang, Xiaokang Yang, and Ming-Hsuan Yang. Hierarchical convolutional features for visual tracking. In ICCV, pages 3074-3082, 2015.","title":"Hierarchical convolutional features for visual tracking","authors":"Ma","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ma et al., 2015b] Chao Ma, Xiaokang Yang, Chongyang Zhang, and Ming-Hsuan Yang. Long-term correlation tracking. In CVPR, pages 5388-5396, 2015.","title":"Long-term correlation tracking","authors":"Ma","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mueller et al., 2017] Matthias Mueller, Neil Smith, and Bernard Ghanem. Context-aware correlation filter tracking. In CVPR, pages 1396-1404, 2017.","title":"Context-aware correlation filter tracking","authors":"Matthias Mueller, Neil Smith, Bernard Ghanem","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nam and Han, 2016] Hyeonseob Nam and Bohyung Han. Learn- ing multi-domain convolutional neural networks for visual track- ing. In CVPR, pages 4293-4302, 2016.","title":"Hyeonseob Nam and Bohyung Han. Learning multi-domain convolutional neural networks for visual tracking","authors":"Nam, Han","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ross et al., 2008] David A Ross, Jongwoo Lim, Ruei-Sung Lin, and Ming-Hsuan Yang. Incremental learning for robust visual tracking. IJCV, 77(1-3):125-141, 2008.","title":"Incremental learning for robust visual tracking","authors":"Jongwoo David A Ross, Ruei-Sung Lim, Ming-Hsuan Lin, Yang","year":"2008","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexan- der C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recog- nition Challenge. IJCV, 115(3):211-252, 2015.","title":"ImageNet Large Scale Visual Recognition Challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C Berg, Li Fei-Fei","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale im- age recognition. In ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tao et al., 2016] Ran Tao, Efstratios Gavves, and Arnold W M S- meulders. Siamese instance search for tracking. In CVPR, pages 1420-1429, 2016.","title":"Siamese instance search for tracking","authors":"Ran Tao, Efstratios Gavves, Arnold W M Smeulders","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Valmadre et al., 2017] Jack Valmadre, Luca Bertinetto, João F Henriques, Andrea Vedaldi, and Philip HS Torr. End-to-end representation learning for correlation filter based tracking. In CVPR, pages 5000-5008, 2017.","title":"End-to-end representation learning for correlation filter based tracking","authors":"Valmadre","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang and Yeung, 2013] Naiyan Wang and Dit-Yan Yeung. Learn- ing a deep compact image representation for visual tracking. In NIPS, pages 809-817, 2013.","title":"Learning a deep compact image representation for visual tracking","authors":"Wang, Yeung, Naiyan Wang, Dit-Yan Yeung","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2015] Lijun Wang, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. Visual tracking with fully convolutional net- works. In CVPR, pages 3119-3127, 2015.","title":"Visual tracking with fully convolutional networks","authors":"Lijun Wang, Wanli Ouyang, Xiaogang Wang, Huchuan Lu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2017] Qiang Wang, Jin Gao, Junliang Xing, Mengdan Zhang, and Weiming Hu. DCFNet: Discriminant correlation fil- ters network for visual tracking. In arXiv:1704.04057, 2017.","title":"DCFNet: Discriminant correlation filters network for visual tracking","authors":"Qiang Wang, Jin Gao, Junliang Xing, Mengdan Zhang, Weiming Hu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2018] Qiang Wang, Zhu Teng, Junliang Xing, Jin Gao, Weiming Hu, and Steve Maybank. Learning attentions: Residual attentional siamese network for high performance on- line visual tracking. In CVPR, 2018.","title":"Learning attentions: Residual attentional siamese network for high performance online visual tracking","authors":"Qiang Wang, Zhu Teng, Junliang Xing, Jin Gao, Weiming Hu, Steve Maybank","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wu et al., 2013] Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Online object tracking: A benchmark. In CVPR, pages 2411- 2418, 2013.","title":"Online object tracking: A benchmark","authors":"Yi Wu, Jongwoo Lim, Ming-Hsuan Yang","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wu et al., 2015] Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Object tracking benchmark. TPAMI, 37(9):1834-1848, 2015.","title":"Object tracking benchmark","authors":"Yi Wu, Jongwoo Lim, Ming-Hsuan Yang","year":"2015","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2014] Jianming Zhang, Shugao Ma, and Stan Sclaroff. Meem: robust tracking via multiple experts using en- tropy minimization. In ECCV, pages 188-203, 2014.","title":"Meem: robust tracking via multiple experts using entropy minimization","authors":"Jianming Zhang, Shugao Ma, Stan Sclaroff","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2015] Mengdan Zhang, Junliang Xing, Jin Gao, and Weiming Hu. Robust visual tracking using joint scale-spatial cor- relation filters. In ICIP, pages 1468-1472, 2015.","title":"Robust visual tracking using joint scale-spatial correlation filters","authors":"Zhang","year":"2015","venue":"ICIP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}