{"type":"ScholarlyArticle","title":"OSP2B: One-Stage Point-to-Box Network for 3D Siamese Tracking","authors":[{"name":"Jiahao Nie","slug":"jiahao-nie","given":"Jiahao","family":"Nie","affiliation":"School of Electronics and Information , Hangzhou Dianzi University , China"},{"name":"Zhiwei He","slug":"zhiwei-he","given":"Zhiwei","family":"He","affiliation":"School of Electronics and Information , Hangzhou Dianzi University , China"},{"name":"Yuxiang Yang","slug":"yuxiang-yang","given":"Yuxiang","family":"Yang","affiliation":"School of Electronics and Information , Hangzhou Dianzi University , China"},{"name":"Zhengyi Bao","slug":"zhengyi-bao","given":"Zhengyi","family":"Bao","affiliation":"School of Electronics and Information , Hangzhou Dianzi University , China"},{"name":"Mingyu Gao","slug":"mingyu-gao","given":"Mingyu","family":"Gao","affiliation":"School of Electronics and Information , Hangzhou Dianzi University , China"},{"name":"Jing Zhang","slug":"jing-zhang","given":"Jing","family":"Zhang","affiliation":"School of Computer Science , The University of Sydney , Australia"}],"abstract":"Two-stage point-to-box network acts as a critical role in the recent popular 3D Siamese tracking paradigm, which first generates proposals and then predicts corresponding proposal-wise scores. However, such a network suffers from tedious hyper-parameter tuning and task misalignment, limiting the tracking performance. Towards these concerns, we propose a simple yet effective one-stage point-to-box network for point cloud-based 3D single object tracking. It synchronizes 3D proposal generation and center-ness score prediction by a parallel predictor without tedious hyper-parameters. To guide a task-aligned score ranking of proposals, a center-aware focal loss is proposed to supervise the training of the center-ness branch, which enhances the network's discriminative ability to distinguish proposals of different quality. Besides, we design a binary target classifier to identify target-relevant points. By integrating the derived classification scores with the center-ness scores, the resulting network can effectively suppress interference proposals and further mitigate task misalignment. Finally, we present a novel one-stage Siamese tracker OSP2B equipped with the designed network. Extensive experiments on challenging benchmarks including KITTI and Waymo SOT Dataset show that our OSP2B achieves leading performance with a considerable real-time speed.","doi":"10.24963/ijcai.2023/143","url":"https://ijcai.org/proceedings/2023/143","pdf":"https://ijcai.org/proceedings/2023/0143.pdf","markdown":"https://ijcai.org/proceedings/2023/143.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/143","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":143,"pages":{"first":1285,"last":1293},"keywords":["Computer Vision: CV: 3D computer vision","Robotics: ROB: Robotics and vision","Computer Vision: CV: Motion and tracking"],"videos":[],"references":[{"position":1,"raw":"[Asvadi et al., 2016] Alireza Asvadi, Pedro Girao, Paulo Peixoto, and Urbano Nunes. 3d object tracking using rgb and lidar data. In ITSC, pages 1255-1260, 2016.","title":"3d object tracking using rgb and lidar data","authors":"Alireza Asvadi, Pedro Girao, Paulo Peixoto, Urbano Nunes","year":"2016","venue":"ITSC","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bertinetto et al., 2016] Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. Fully-convolutional siamese networks for object tracking. In European conference on computer vision, pages 850- 865. Springer, 2016.","title":"Fully-convolutional siamese networks for object tracking","authors":"Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, Philip Hs Torr","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bibi et al., 2016] Adel Bibi, Tianzhu Zhang, and Bernard Ghanem. 3d part-based sparse tracker with automatic syn- chronization and registration. In CVPR, pages 1439-1448, 2016.","title":"3d part-based sparse tracker with automatic synchronization and registration","authors":"Adel Bibi, Tianzhu Zhang, Bernard Ghanem","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang, Xiaoyun Yang, and Huchuan Lu. Transformer tracking. In CVPR, pages 8126-8135, 2021.","title":"Transformer tracking","authors":"Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang, Xiaoyun Yang, Huchuan Lu","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cui et al., 2021] Yubo Cui, Zheng Fang, Jiayao Shan, Zuoxu Gu, and Sifan Zhou. 3d object tracking with trans- former. In BMCV, 2021.","title":"3d object tracking with transformer","authors":"Yubo Cui, Zheng Fang, Jiayao Shan, Zuoxu Gu, Sifan Zhou","year":"2021","venue":"BMCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fang et al., 2020] Zhang Fang, Suo Zhou, Yu Cui, and Sac Scherer. 3d-siamrpn: An end-to-end learning method for real-time 3d single object tracking using raw point cloud. IEEE Sensors Journal, 21(4):1019-1026, 2020.","title":"3d-siamrpn: An end-to-end learning method for real-time 3d single object tracking using raw point cloud","authors":"Zhang Fang, Suo Zhou, Yu Cui, Sac Scherer","year":"2020","venue":"IEEE Sensors Journal","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Geiger et al., 2012] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, pages 3354- 3361, 2012.","title":"Are we ready for autonomous driving? the kitti vision benchmark suite","authors":"Andreas Geiger, Philip Lenz, Raquel Urtasun","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Giancola et al., 2019] Silvio Giancola, Jesus Zarzar, and Bernard Ghanem. Leveraging Shape Completion for 3D Siamese Tracking. In CVPR, pages 1359-1368, 2019.","title":"Leveraging Shape Completion for 3D Siamese Tracking","authors":"Silvio Giancola, Jesus Zarzar, Bernard Ghanem","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2020] Dongyan Guo, Jun Wang, Ying Cui, Zhenhua Wang, and Shengyong Chen. Siamcar: Siamese fully convolutional classification and regression for visual tracking. In CVPR, pages 6269-6277, 2020.","title":"Siamcar: Siamese fully convolutional classification and regression for visual tracking","authors":"Dongyan Guo, Jun Wang, Ying Cui, Zhenhua Wang, Shengyong Chen","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2022] Zhiyang Guo, Yunyao Mao, Wengang Zhou, Min Wang, and Houqiang Li. Cmt: Context- matching-guided transformer for 3d tracking in point clouds. In European Conference on Computer Vision, pages 95-111, 2022.","title":"Cmt: Contextmatching-guided transformer for 3d tracking in point clouds","authors":"Zhiyang Guo, Yunyao Mao, Wengang Zhou, Min Wang, Houqiang Li","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hui et al., 2021] Le Hui, Lingpeng Wang, Mingmei Cheng, Jin Xie, and Jian Yang. 3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds. In NIPS, pages 28714-28727, 2021.","title":"3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds","authors":"Hui","year":"2021","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hui et al., 2022] Le Hui, Lingpeng Wang, Linghua Tang, Kaihao Lan, Jin Xie, and Jian Yang. 3D Siamese Trans- former Network for Single Object Tracking on Point Clouds. In ECCV, 2022.","title":"3D Siamese Transformer Network for Single Object Tracking on Point Clouds","authors":"Le Hui, Lingpeng Wang, Linghua Tang, Kaihao Lan, Jin Xie, Jian Yang","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Javed et al., 2022] Sajid Javed, Martin Danelljan, Fa- had Shahbaz Khan, Muhammad Haris Khan, Michael Felsberg, and Jiri Matas. Visual Object Tracking with Dis- criminative Filters and Siamese Networks: A Survey and Outlook. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, pages 1-20, 2022.","title":"Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook","authors":"Sajid Javed, Martin Danelljan, Fahad Shahbaz Khan, Muhammad Haris Khan, Michael Felsberg, Jiri Matas","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kart et al., 2019] Ugur Kart, Alan Lukezic, Matej Kristan, Joni-Kristian Kamarainen, and Jiri Matas. Object tracking by reconstruction with view-specific discriminative corre- lation filters. In ICCV, pages 1339-1348, 2019.","title":"Object tracking by reconstruction with view-specific discriminative correlation filters","authors":"Ugur Kart, Alan Lukezic, Matej Kristan, Joni-Kristian Kamarainen, Jiri Matas","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2018] Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu. High performance visual tracking with siamese region proposal network. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 8971-8980, 2018.","title":"High performance visual tracking with siamese region proposal network","authors":"Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, Xiaolin Hu","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In CVPR, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2018] Ye Liu, Xiao-Yuan Jing, Jianhui Nie, Hao Gao, Jun Liu, and Guo-Ping Jiang. Context-aware three- dimensional mean-shift with occlusion handling for robust object tracking in rgb-d videos. TMM, 21(3):664-677, 2018.","title":"Context-aware threedimensional mean-shift with occlusion handling for robust object tracking in rgb-d videos","authors":"Ye Liu, Xiao-Yuan Jing, Jianhui Nie, Hao Gao, Jun Liu, Guo-Ping Jiang","year":"2018","venue":"TMM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nie et al., 2022a] Jiahao Nie, Zhiwei He, Yuxiang Yang, Mingyu Gao, and Zhekang Dong. Learning localization- aware target confidence for siamese visual tracking. TMM, 2022.","title":"Learning localizationaware target confidence for siamese visual tracking","authors":"Nie","year":"2022","venue":"TMM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nie et al., 2022b] Jiahao Nie, Zhiwei He, Yuxiang Yang, Mingyu Gao, and Jing Zhang. Glt-t: Global-local trans- former voting for 3d single object tracking in point clouds, 2022.","title":"Glt-t: Global-local transformer voting for 3d single object tracking in point clouds","authors":"Nie","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nie et al., 2022c] Jiahao Nie, Han Wu, Zhiwei He, Mingyu Gao, and Zhekang Dong. Spreading fine-grained prior knowledge for accurate tracking. TCSVT, 2022.","title":"Spreading fine-grained prior knowledge for accurate tracking","authors":"Nie","year":"2022","venue":"TCSVT","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pi et al., 2022] Zhixiong Pi, Weitao Wan, Chong Sun, Changxin Gao, Nong Sang, and Chen Li. Hierarchical feature embedding for visual tracking. In European Con- ference on Computer Vision, pages 428-445, 2022.","title":"Hierarchical feature embedding for visual tracking","authors":"Pi","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pieropan et al., 2015] Alessandro Pieropan, Niklas Bergström, Masatoshi Ishikawa, and Hedvig Kjell- ström. Robust 3d tracking of unknown objects. In ICRA, pages 2410-2417, 2015.","title":"Robust 3d tracking of unknown objects","authors":"Alessandro Pieropan, Niklas Bergström, Masatoshi Ishikawa, Hedvig Kjellström","year":"2015","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Qi et al., 2017a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652-660, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Qi","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Qi et al., 2017b] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. NIPS, 30, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Qi","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Qi et al., 2019] Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas. Deep hough voting for 3d object de- tection in point clouds. In ICCV, pages 9277-9286, 2019.","title":"Deep hough voting for 3d object detection in point clouds","authors":"Or Charles R Qi, Kaiming Litany, Leonidas J He, Guibas","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Qi et al., 2020] Haozhe Qi, Chen Feng, Zhiguo Cao, Feng Zhao, and Yang Xiao. P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds. In CVPR, pages 6328- 6337, 2020.","title":"P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds","authors":"Haozhe Qi, Chen Feng, Zhiguo Cao, Feng Zhao, Yang Xiao","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Shan et al., 2021] Jiayao Shan, Sifan Zhou, Zheng Fang, and Yubo Cui. PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Clouds. In IROS, pages 1310-1316, 2021.","title":"PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Clouds","authors":"Shan","year":"2021","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sun et al., 2020] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. 3d siamese transformer network for single object tracking on point clouds. In CVPR, pages 2446-2454, 2020.","title":"3d siamese transformer network for single object tracking on point clouds","authors":"Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2021] Zhoutao Wang, Qian Xie, Yu-Kun Lai, Jing Wu, Kun Long, and Jun Wang. MLVSNet: Multi- level Voting Siamese Network for 3D Visual Tracking. In ICCV, pages 3081-3090, 2021.","title":"MLVSNet: Multilevel Voting Siamese Network for 3D Visual Tracking","authors":"Zhoutao Wang, Qian Xie, Yu-Kun Lai, Jing Wu, Kun Long, Jun Wang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2013] Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Online object tracking: A benchmark. In CVPR, pages 2411-2418, 2013.","title":"Online object tracking: A benchmark","authors":"Yi Wu, Jongwoo Lim, Ming-Hsuan Yang","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xu et al., 2022] Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vitpose: Simple vision transformer baselines for human pose estimation. arXiv preprint arXiv:2204.12484, 2022.","title":"Vitpose: Simple vision transformer baselines for human pose estimation","authors":"Yufei Xu, Jing Zhang, Qiming Zhang, Dacheng Tao","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang and Tao, 2020] Jing Zhang and Dacheng Tao. Em- powering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intel- ligence of things. IoT, 8(10):7789-7817, 2020.","title":"Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things","authors":"Tao ; ] Jing Zhang, Dacheng Zhang, Tao","year":"2020","venue":"IoT","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2020] Zhipeng Zhang, Houwen Peng, Jian- long Fu, Bing Li, and Weiming Hu. Ocean: Object-aware anchor-free tracking. In European Conference on Com- puter Vision, pages 771-787, 2020.","title":"Ocean: Object-aware anchor-free tracking","authors":"Zhipeng Zhang, Houwen Peng, Jianlong Fu, Bing Li, Weiming Hu","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2022a] Qiming Zhang, Yufei Xu, Jing Zhang, and Dacheng Tao. Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and be- yond. arXiv preprint arXiv:2202.10108, 2022.","title":"Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2022b] Qiming Zhang, Yufei Xu, Jing Zhang, and Dacheng Tao. Vsa: Learning varied-size window at- tention in vision transformers. In European conference on computer vision, 2022.","title":"Vsa: Learning varied-size window attention in vision transformers","authors":"Zhang","year":"2022","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zheng et al., 2021] Chaoda Zheng, Xu Yan, Jiantao Gao, Weibing Zhao, Wei Zhang, Zhen Li, and Shuguang Cui. Box-Aware Feature Enhancement for Single Object Track- ing on Point Clouds. In ICCV, pages 13179-13188, 2021.","title":"Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds","authors":"Zheng","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zheng et al., 2022] Chaoda Zheng, Xu Yan, Haiming Zhang, Baoyuan Wang, Shenghui Cheng, Shuguang Cui, and Zhen Li. Beyond 3d siamese tracking: A motion- centric paradigm for 3d single object tracking in point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8111- 8120, 2022.","title":"Beyond 3d siamese tracking: A motioncentric paradigm for 3d single object tracking in point clouds","authors":"Zheng","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhou et al., 2022] Changqing Zhou, Zhipeng Luo, Yueru Luo, Tianrui Liu, Liang Pan, Zhongang Cai, Haiyu Zhao, and Shijian Lu. PTTR: Relational 3D Point Cloud Object Tracking with Transformer. In CVPR, pages 8531-8540, 2022.","title":"PTTR: Relational 3D Point Cloud Object Tracking with Transformer","authors":"Changqing Zhou, Zhipeng Luo, Yueru Luo, Tianrui Liu, Liang Pan, Zhongang Cai, Haiyu Zhao, Shijian Lu","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}