{"type":"ScholarlyArticle","title":"Unsupervised Embedding and Association Network for Multi-Object Tracking","authors":[{"name":"Yu-Lei Li","slug":"yu-lei-li","given":"Yu-Lei","family":"Li","affiliation":"School of Informatics , Xiamen University"}],"abstract":"How to generate robust trajectories of multiple objects without using any manual identity annotation? Recently, identity embedding features from Re-ID models are adopted to associate targets into trajectories.\r\nHowever, most previous methods equipped with embedding features heavily rely on manual identity annotations, which bring a high cost for the multi-object tracking (MOT) task.\r\nTo address the above problem, we present an unsupervised embedding and association network (UEANet) for learning discriminative embedding features with pseudo identity labels. \r\nSpecifically, we firstly generate the pseudo identity labels by adopting a Kalman filter tracker to associate multiple targets into trajectories and assign a unique identity label to each trajectory. \r\nSecondly, we train the transformer-based identity embedding branch and MLP-based data association branch of UEANet with these pseudo labels, and UEANet extracts branch-dependent features for the unsupervised MOT task.\r\nExperimental results show that UEANet confirms the outstanding ability to suppress IDS and achieves comparable performance compared with state-of-the-art methods on three MOT datasets.","doi":"10.24963/ijcai.2022/157","url":"https://ijcai.org/proceedings/2022/157","pdf":"https://ijcai.org/proceedings/2022/0157.pdf","markdown":"https://ijcai.org/proceedings/2022/157.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/157","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":157,"pages":{"first":1123,"last":1129},"keywords":["Computer Vision: Motion and Tracking"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987074","length_sec":73,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985217","length_sec":510,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bernardin and Stiefelhagen, 2008] Keni Bernardin and Rainer Stiefelhagen. Evaluating multiple object tracking performance: the clear mot metrics. EURASIP Journal on Image and Video Processing, 2008:1-10, 2008.","title":"Evaluating multiple object tracking performance: the clear mot metrics","authors":"Stiefelhagen Bernardin, Keni Bernardin, Rainer Stiefelhagen","year":"2008","venue":"EURASIP Journal on Image and Video Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dendorfer et al., 2020] Patrick Dendorfer, Hamid Rezatofighi, Anton Milan, Javen Shi, Daniel Cre- mers, Ian Reid, Stefan Roth, Konrad Schindler, and Laura Leal-Taixé. Mot20: A benchmark for multi object track- ing in crowded scenes. arXiv preprint arXiv:2003.09003, 2020.","title":"Mot20: A benchmark for multi object tracking in crowded scenes","authors":"Patrick Dendorfer, Hamid Rezatofighi, Anton Milan, Javen Shi, Daniel Cremers, Ian Reid, Stefan Roth, Konrad Schindler, Laura Leal-Taixé","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ding et al., 2021] Xiaohan Ding, Chunlong Xia, Xiangyu Zhang, Xiaojie Chu, Jungong Han, and Guiguang Ding. Repmlp: re-parameterizing convolutions into fully- connected layers for image recognition. arXiv preprint arXiv:2105.01883, 2021.","title":"Repmlp: re-parameterizing convolutions into fullyconnected layers for image recognition","authors":"Xiaohan Ding, Chunlong Xia, Xiangyu Zhang, Xiaojie Chu, Jungong Han, Guiguang Ding","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guo et al., 2021] Song Guo, Jingya Wang, Xinchao Wang, and Dacheng Tao. Online multiple object tracking with cross-task synergy. In CVPR, pages 8136-8145, 2021.","title":"Online multiple object tracking with cross-task synergy","authors":"Song Guo, Jingya Wang, Xinchao Wang, Dacheng Tao","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Li et al., 2019] Xiang Li, Xiaolin Hu, and Jian Yang. Spatial group-wise enhance: Improving semantic fea- ture learning in convolutional networks. arXiv preprint arXiv:1905.09646, 2019.","title":"Spatial group-wise enhance: Improving semantic feature learning in convolutional networks","authors":"Xiang Li, Xiaolin Hu, Jian Yang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Liu et al., 2021] Huajun Liu, Fuqiang Liu, Xinyi Fan, and Dong Huang. Polarized self-attention: towards high-quality pixel-wise regression. arXiv preprint arXiv:2107.00782, 2021.","title":"Polarized self-attention: towards high-quality pixel-wise regression","authors":"Huajun Liu, Fuqiang Liu, Xinyi Fan, Dong Huang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liu et al., 2022] Qiankun Liu, Dongdong Chen, Qi Chu, Lu Yuan, Bin Liu, Lei Zhang, and Nenghai Yu. Online multi-object tracking with unsupervised re-identification learning and occlusion estimation. Neurocomputing, 2022.","title":"Online multi-object tracking with unsupervised re-identification learning and occlusion estimation","authors":"Qiankun Liu, Dongdong Chen, Qi Chu, Lu Yuan, Bin Liu, Lei Zhang, Nenghai Yu","year":"2022","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Milan et al., 2016] Anton Milan, Laura Leal-Taixé, Ian Reid, Stefan Roth, and Konrad Schindler. Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831, 2016.","title":"Mot16: A benchmark for multi-object tracking","authors":"Anton Milan, Laura Leal-Taixé, Ian Reid, Stefan Roth, Konrad Schindler","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Peng et al., 2020] Jinlong Peng, Changan Wang, Fangbin Wan, Yang Wu, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Yanwei Fu. Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking. In ECCV, vol- ume 12349, pages 145-161, 2020.","title":"Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking","authors":"Jinlong Peng, Changan Wang, Fangbin Wan, Yang Wu, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, Yanwei Fu","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Shan et al., 2020] Chaobing Shan, Chunbo Wei, Bing Deng, Jianqiang, Huang, Xian-Sheng Hua, Xiaoliang Cheng, and Kewei Liang. Tracklets predicting based adap- tive graph tracking. arXiv preprint arXiv:2010.09015, 2020.","title":"Tracklets predicting based adaptive graph tracking","authors":"Shan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Sun et al., 2020] Peize Sun, Yi Jiang, Rufeng Zhang, Enze Xie, Jinkun Cao, Xinting Hu, Tao Kong, Ze- huan Yuan, Changhu Wang, and Ping Luo. Transtrack: Multiple-object tracking with transformer. arXiv preprint arXiv:2012.15460, 2020.","title":"Transtrack: Multiple-object tracking with transformer","authors":"Peize Sun, Yi Jiang, Rufeng Zhang, Enze Xie, Jinkun Cao, Xinting Hu, Tao Kong, Zehuan Yuan, Changhu Wang, Ping Luo","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Sun et al., 2021] ShiJie Sun, Naveed Akhtar, HuanSheng Song, Ajmal Mian, and Mubarak Shah. Deep affinity net- work for multiple object tracking. IEEE transactions on pattern analysis and machine intelligence, 43(1):104-119, 2021.","title":"Deep affinity network for multiple object tracking","authors":"Sun","year":"2021","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Tolstikhin et al., 2021] Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. arXiv preprint arXiv:2105.01601, 2021.","title":"Mlp-mixer: An all-mlp architecture for vision","authors":"Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Touvron et al., 2021] Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Gautier Izacard, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, et al. Resmlp: Feedforward net- works for image classification with data-efficient training. arXiv preprint arXiv:2105.03404, 2021.","title":"Resmlp: Feedforward networks for image classification with data-efficient training","authors":"Touvron","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2021a] Qiang Wang, Yun Zheng, Pan Pan, and Yinghui Xu. Joint object detection and multi-object track- ing with graph neural networks. In ICRA, pages 1-1, 2021.","title":"Joint object detection and multi-object tracking with graph neural networks","authors":"Wang","year":"2021","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2021b] Qiang Wang, Yun Zheng, Pan Pan, and Yinghui Xu. Multiple object tracking with correlation learning. In CVPR, pages 3876-3886, 2021.","title":"Multiple object tracking with correlation learning","authors":"Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wu et al., 2021] Jialian Wu, Jiale Cao, Liangchen Song, Yu Wang, Ming Yang, and Junsong Yuan. Track to detect and segment: An online multi-object tracker. In CVPR, pages 12352-12361, 2021.","title":"Track to detect and segment: An online multi-object tracker","authors":"Wu","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2020] Yihong Xu, Aljosa Osep, Yutong Ban, Radu Horaud, Laura Leal-Taixé, and Xavier Alameda- Pineda. How to train your deep multi-object tracker. In CVPR, pages 6786-6795, 2020.","title":"How to train your deep multi-object tracker","authors":"Yihong Xu, Aljosa Osep, Yutong Ban, Radu Horaud, Laura Leal-Taixé, Xavier Alameda-Pineda","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2021] Yihong Xu, Yutong Ban, Guillaume Delorme, Chuang Gan, Daniela Rus, and Xavier Alameda-Pineda. Transcenter: Transformers with dense queries for multiple-object tracking. arXiv preprint arXiv:2103.15145, 2021.","title":"Transcenter: Transformers with dense queries for multiple-object tracking","authors":"Yihong Xu, Yutong Ban, Guillaume Delorme, Chuang Gan, Daniela Rus, Xavier Alameda-Pineda","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang and Yang, 2021] Qing-Long Zhang and Yu-Bin Yang. Sa-net: Shuffle attention for deep convolutional neural networks. In ICASSP, pages 2235-2239, 2021.","title":"Sa-net: Shuffle attention for deep convolutional neural networks","authors":"Yang ; Qing-Long Zhang, Yu-Bin Zhang, Yang","year":"2021","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2021a] Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object track- ing by associating every detection box. arXiv preprint arXiv:2110.06864, 2021.","title":"Bytetrack: Multi-object tracking by associating every detection box","authors":"Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2021b] Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu. Fairmot: On the fairness of detection and re-identification in multiple ob- ject tracking. International Journal of Computer Vision, 129(11):3069-3087, 2021.","title":"Fairmot: On the fairness of detection and re-identification in multiple object tracking","authors":"Zhang","year":"2021","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2019] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. Objects as points. arXiv preprint arXiv:1904.07850, 2019.","title":"Objects as points","authors":"Zhou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zou, 2020] Chao Liang Zhipeng Zhang Yi Lu Xue Zhou Bing Li Xiyong Ye Jianxiao Zou. Rethinking the compe- tition between detection and reid in multi-object tracking. arXiv preprint arXiv:2010.12138, 2020.","title":"Rethinking the competition between detection and reid in multi-object tracking","authors":"Chao Liang, Zhipeng Zhang, Yi Lu, Xue Zhou, Bing Li, Xiyong Ye, Jianxiao Zou","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}