{"type":"ScholarlyArticle","title":"DCDet: Dynamic Cross-based 3D Object Detector","authors":[{"name":"Shuai Liu","slug":"shuai-liu","given":"Shuai","family":"Liu","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University"},{"name":"Boyang Li","slug":"boyang-li","given":"Boyang","family":"Li","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University"},{"name":"Zhiyu Fang","slug":"zhiyu-fang","given":"Zhiyu","family":"Fang","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University"},{"name":"Kai Huang","slug":"kai-huang","given":"Kai","family":"Huang","affiliation":"School of Computer Science and Engineering , Sun Yat-sen University"}],"abstract":"Recently, significant progress has been made in the research of 3D object detection. However, most prior studies have focused on the utilization of center-based or anchor-based label assignment schemes. Alternative label assignment strategies remain unexplored in 3D object detection. We find that the center-based label assignment often fails to generate sufficient positive samples for training, while the anchor-based label assignment tends to encounter an imbalanced issue when handling objects with different scales. To solve these issues, we introduce a dynamic cross label assignment (DCLA) scheme, which dynamically assigns positive samples for each object from a cross-shaped region, thus providing sufficient and balanced positive samples for training. Furthermore, to address the challenge of accurately regressing objects with varying scales, we put forth a rotation-weighted Intersection over Union (RWIoU) metric to replace the widely used L1 metric in regression loss. Extensive experiments demonstrate the generality and effectiveness of our DCLA and RWIoU-based regression loss. The Code is available at https://github.com/Say2L/DCDet.git.","doi":"10.24963/ijcai.2024/124","url":"https://ijcai.org/proceedings/2024/124","pdf":"https://ijcai.org/proceedings/2024/0124.pdf","markdown":"https://ijcai.org/proceedings/2024/124.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/124","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":124,"pages":{"first":1119,"last":1127},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)","Computer Vision: CV: 3D computer vision"],"videos":[],"references":[{"position":1,"raw":"[Deng et al., 2021] Jiajun Deng, Shaoshuai Shi, Peiwei Li, Wengang Zhou, Yanyong Zhang, and Houqiang Li. Voxel r-cnn: Towards high performance voxel-based 3d object detection. In AAAI, 2021.","title":"Voxel r-cnn: Towards high performance voxel-based 3d object detection","authors":"Jiajun Deng, Shaoshuai Shi, Peiwei Li, Wengang Zhou, Yanyong Zhang, Houqiang Li","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Fan et al., 2022a] Lue Fan, Ziqi Pang, Tianyuan Zhang, Yu- Xiong Wang, Hang Zhao, Feng Wang, Naiyan Wang, and Zhaoxiang Zhang. Embracing single stride 3d object de- tector with sparse transformer. In CVPR, 2022.","title":"Embracing single stride 3d object detector with sparse transformer","authors":"Fan","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fan et al., 2022b] Lue Fan, Feng Wang, Naiyan Wang, and ZHAO-XIANG ZHANG. Fully sparse 3d object detection. In NeurIPS, 2022.","title":"Fully sparse 3d object detection","authors":"Fan","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ge et al., 2020] Runzhou Ge, Zhuangzhuang Ding, Yihan Hu, Yu Wang, Sijia Chen, Li Huang, and Yuan Li. Afdet: Anchor free one stage 3d object detection. arXiv preprint arXiv:2006.12671, 2020.","title":"Afdet: Anchor free one stage 3d object detection","authors":"Ge","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ge et al., 2021] Zheng Ge, Songtao Liu, Feng Wang, Zem- ing Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430, 2021.","title":"Yolox: Exceeding yolo series in 2021","authors":"Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, Jian Sun","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Geiger et al., 2012] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012.","title":"Are we ready for autonomous driving? the kitti vision benchmark suite","authors":"Andreas Geiger, Philip Lenz, Raquel Urtasun","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2022] Chenhang He, Ruihuang Li, Shuai Li, and Lei Zhang. Voxel set transformer: A set-to-set approach to 3d object detection from point clouds. In CVPR, 2022.","title":"Voxel set transformer: A set-to-set approach to 3d object detection from point clouds","authors":"He","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2022] Yihan Hu, Zhuangzhuang Ding, Runzhou Ge, Wenxin Shao, Li Huang, Kun Li, and Qiang Liu. Afdetv2: Rethinking the necessity of the second stage for object detection from point clouds. In AAAI, 2022.","title":"Afdetv2: Rethinking the necessity of the second stage for object detection from point clouds","authors":"Yihan Hu, Zhuangzhuang Ding, Runzhou Ge, Wenxin Shao, Li Huang, Kun Li, Qiang Liu","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lang et al., 2019] Alex H Lang, Sourabh Vora, Holger Cae- sar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Point- pillars: Fast encoders for object detection from point clouds. In CVPR, 2019.","title":"Pointpillars: Fast encoders for object detection from point clouds","authors":"Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, Oscar Beijbom","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2021] Zhichao Li, Feng Wang, and Naiyan Wang. Lidar r-cnn: An efficient and universal 3d object detector. In CVPR, 2021.","title":"Lidar r-cnn: An efficient and universal 3d object detector","authors":"Zhichao Li, Feng Wang, Naiyan Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2023] Jinyu Li, Chenxu Luo, and Xiaodong Yang. Pillarnext: Rethinking network designs for 3d object de- tection in lidar point clouds. In CVPR, 2023.","title":"Pillarnext: Rethinking network designs for 3d object detection in lidar point clouds","authors":"Jinyu Li, Chenxu Luo, Xiaodong Yang","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Qi et al., 2017a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Qi","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Qi et al., 2017b] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in a metric space. In NeurIPS, 2017. [Rezatofighi et al., 2019] Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In CVPR, 2019.","title":"Generalized intersection over union: A metric and a loss for bounding box regression","authors":"Qi","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sheng et al., 2022] Hualian Sheng, Sijia Cai, Na Zhao, Bing Deng, Jianqiang Huang, Xian-Sheng Hua, Min-Jian Zhao, and Gim Hee Lee. Rethinking iou-based optimiza- tion for single-stage 3d object detection. In ECCV, 2022.","title":"Rethinking iou-based optimization for single-stage 3d object detection","authors":"Hualian Sheng, Sijia Cai, Na Zhao, Bing Deng, Jianqiang Huang, Xian-Sheng Hua, Min-Jian Zhao, Gim Hee, Lee","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shi et al., 2019] Shaoshuai Shi, Xiaogang Wang, and Hong- sheng Li. Pointrcnn: 3d object proposal generation and detection from point cloud. In CVPR, 2019.","title":"Pointrcnn: 3d object proposal generation and detection from point cloud","authors":"Shaoshuai Shi, Xiaogang Wang, Hongsheng Li","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shi et al., 2020a] Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn: Point-voxel feature set abstraction for 3d ob- ject detection. In CVPR, 2020.","title":"Pv-rcnn: Point-voxel feature set abstraction for 3d object detection","authors":"Shi","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi et al., 2020b] Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network. TPAMI, 2020.","title":"From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network","authors":"Shi","year":"2020","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shi et al., 2022] Guangsheng Shi, Ruifeng Li, and Chao Ma. Pillarnet: Real-time and high-performance pillar- based 3d object detection. In ECCV, 2022.","title":"Pillarnet: Real-time and high-performance pillarbased 3d object detection","authors":"Guangsheng Shi, Ruifeng Li, Chao Ma","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi et al., 2023] Shaoshuai Shi, Li Jiang, Jiajun Deng, Zhe Wang, Chaoxu Guo, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn++: Point-voxel feature set ab- straction with local vector representation for 3d object de- tection. IJCV, 2023.","title":"Pv-rcnn++: Point-voxel feature set abstraction with local vector representation for 3d object detection","authors":"Shaoshuai Shi, Li Jiang, Jiajun Deng, Zhe Wang, Chaoxu Guo, Jianping Shi, Xiaogang Wang, Hongsheng Li","year":"2023","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2020] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, 2020.","title":"Scalability in perception for autonomous driving: Waymo open dataset","authors":"Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun et al., 2022] Pei Sun, Mingxing Tan, Weiyue Wang, Chenxi Liu, Fei Xia, Zhaoqi Leng, and Dragomir Anguelov. Swformer: Sparse window transformer for 3d object detection in point clouds. In ECCV, 2022.","title":"Swformer: Sparse window transformer for 3d object detection in point clouds","authors":"Pei Sun, Mingxing Tan, Weiyue Wang, Chenxi Liu, Fei Xia, Zhaoqi Leng, Dragomir Anguelov","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Team, 2020] OpenPCDet Development Team. Open- pcdet: An open-source toolbox for 3d object detec- tion from point clouds. https://github.com/open-mmlab/ OpenPCDet, 2020.","title":"OpenPCDet Development Team. Openpcdet: An open-source toolbox for 3d object detection from point clouds","authors":null,"year":"2020","venue":null,"doi":null,"url":"https://github.com/open-mmlab/OpenPCDet,2020","internal_url":null},{"position":24,"raw":"[Tian et al., 2019] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object de- tection. In ICCV, 2019.","title":"Fcos: Fully convolutional one-stage object detection","authors":"Chunhua Zhi Tian, Hao Shen, Tong Chen, He","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2021] Qi Wang, Jian Chen, Jianqiang Deng, and Xinfang Zhang. 3d-centernet: 3d object detection network for point clouds with center estimation priority. Pattern Recognition, 2021.","title":"3d-centernet: 3d object detection network for point clouds with center estimation priority","authors":"Qi Wang, Jian Chen, Jianqiang Deng, Xinfang Zhang","year":"2021","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2023] Haiyang Wang, Chen Shi, Shaoshuai Shi, Meng Lei, Sen Wang, Di He, Bernt Schiele, and Li- wei Wang. Dsvt: Dynamic sparse voxel transformer with rotated sets. In CVPR, 2023.","title":"Dsvt: Dynamic sparse voxel transformer with rotated sets","authors":"Haiyang Wang, Chen Shi, Shaoshuai Shi, Meng Lei, Sen Wang, Di He, Bernt Schiele, Liwei Wang","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2022] Qiangeng Xu, Yiqi Zhong, and Ulrich Neu- mann. Behind the curtain: Learning occluded shapes for 3d object detection. In AAAI, 2022.","title":"Behind the curtain: Learning occluded shapes for 3d object detection","authors":"Xu","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yan et al., 2018] Yan Yan, Yuxing Mao, and Bo Li. Sec- ond: Sparsely embedded convolutional detection. Sensors, 2018.","title":"Second: Sparsely embedded convolutional detection","authors":"Yan Yan, Yuxing Mao, Bo Li","year":"2018","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2020] Zetong Yang, Yanan Sun, Shu Liu, and Jiaya Jia. 3dssd: Point-based 3d single stage object detec- tor. In CVPR, 2020.","title":"3dssd: Point-based 3d single stage object detector","authors":"Yang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yin et al., 2021] Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and track- ing. In CVPR, 2021.","title":"Center-based 3d object detection and tracking","authors":"Xingyi Tianwei Yin, Philipp Zhou, Krahenbuhl","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2020] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li. Bridging the gap between anchor-based and anchor-free detection via adaptive train- ing sample selection. In CVPR, 2020.","title":"Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection","authors":"Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, Stan Z Li","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2022a] Yi-Fan Zhang, Weiqiang Ren, Zhang Zhang, Zhen Jia, Liang Wang, and Tieniu Tan. Focal and efficient iou loss for accurate bounding box regression. Neurocomputing, 2022.","title":"Focal and efficient iou loss for accurate bounding box regression","authors":"Zhang","year":"2022","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2022b] Yifan Zhang, Qingyong Hu, Guoquan Xu, Yanxin Ma, Jianwei Wan, and Yulan Guo. Not all points are equal: Learning highly efficient point-based de- tectors for 3d lidar point clouds. In CVPR, 2022.","title":"Not all points are equal: Learning highly efficient point-based detectors for 3d lidar point clouds","authors":"Zhang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zheng et al., 2020] Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren. Distance-iou loss: Faster and better learning for bounding box regres- sion. In AAAI, 2020.","title":"Distance-iou loss: Faster and better learning for bounding box regression","authors":"Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, Dongwei Ren","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zheng et al., 2021] Wu Zheng, Weiliang Tang, Sijin Chen, Li Jiang, and Chi-Wing Fu. Cia-ssd: Confident iou-aware single-stage object detector from point cloud. In AAAI, 2021.","title":"Cia-ssd: Confident iou-aware single-stage object detector from point cloud","authors":"Weiliang Wu Zheng, Sijin Tang, Li Chen, Chi-Wing Jiang, Fu","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhou and Tuzel, 2018] Yin Zhou and Oncel Tuzel. Voxel- net: End-to-end learning for point cloud based 3d object detection. In CVPR, 2018.","title":"Voxelnet: End-to-end learning for point cloud based 3d object detection","authors":"Tuzel Zhou, Yin Zhou, Oncel Tuzel","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhou et al., 2019a] Dingfu Zhou, Jin Fang, Xibin Song, Chenye Guan, Junbo Yin, Yuchao Dai, and Ruigang Yang. Iou loss for 2d/3d object detection. In 3DV, 2019.","title":"Iou loss for 2d/3d object detection","authors":"Zhou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhou et al., 2019b] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. Objects as points. arXiv preprint arXiv:1904.07850, 2019.","title":"Objects as points","authors":"Zhou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhou et al., 2022] Zixiang Zhou, Xiangchen Zhao, Yu Wang, Panqu Wang, and Hassan Foroosh. Cen- terformer: Center-based transformer for 3d object detection. In ECCV, 2022.","title":"Centerformer: Center-based transformer for 3d object detection","authors":"Zhou","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhu et al., 2020] Benjin Zhu, Jianfeng Wang, Zhengkai Jiang, Fuhang Zong, Songtao Liu, Zeming Li, and Jian Sun. Autoassign: Differentiable label assignment for dense object detection. arXiv preprint arXiv:2007.03496, 2020.","title":"Autoassign: Differentiable label assignment for dense object detection","authors":"Zhu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}