{"type":"ScholarlyArticle","title":"Dual Relation Knowledge Distillation for Object Detection","authors":[{"name":"Zhen-Liang Ni","slug":"zhen-liang-ni","given":"Zhen-Liang","family":"Ni","affiliation":"Institute of Automation , Chinese Academy of Sciences"},{"name":"Fukui Yang","slug":"fukui-yang","given":"Fukui","family":"Yang","affiliation":"Department of Computer Vision Technology (VIS) , Baidu Inc."},{"name":"Shengzhao Wen","slug":"shengzhao-wen","given":"Shengzhao","family":"Wen","affiliation":"Department of Computer Vision Technology (VIS) , Baidu Inc."},{"name":"Gang Zhang","slug":"gang-zhang","given":"Gang","family":"Zhang","affiliation":"Department of Computer Vision Technology (VIS) , Baidu Inc."}],"abstract":"Knowledge distillation is an effective method for model compression. However, it is still a challenging topic to apply knowledge distillation to detection tasks. There are two key points resulting in poor distillation performance for detection tasks. One is the serious imbalance between foreground and background features, another one is that small object lacks enough feature representation. To solve the above issues, we propose a new distillation method named dual relation knowledge distillation (DRKD), including pixel-wise relation distillation and instance-wise relation distillation. The pixel-wise relation distillation embeds pixel-wise features in the graph space and applies graph convolution to capture the global pixel relation. By distilling the global pixel relation, the student detector can learn the relation between foreground and background features, and avoid the difficulty of distilling features directly for the feature imbalance issue. Besides, we find that instance-wise relation supplements valuable knowledge beyond independent features for small objects. Thus, the instance-wise relation distillation is designed, which calculates the similarity of different instances to obtain a relation matrix. More importantly, a relation filter module is designed to highlight valuable instance relations. The proposed dual relation knowledge distillation is general and can be easily applied for both one-stage and two-stage detectors. Our method achieves state-of-the-art performance, which improves Faster R-CNN based on ResNet50 from 38.4% to 41.6% mAP and improves RetinaNet based on ResNet50 from 37.4% to 40.3% mAP on COCO 2017.","doi":"10.24963/ijcai.2023/142","url":"https://ijcai.org/proceedings/2023/142","pdf":"https://ijcai.org/proceedings/2023/0142.pdf","markdown":"https://ijcai.org/proceedings/2023/142.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/142","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":142,"pages":{"first":1276,"last":1284},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)"],"videos":[],"references":[{"position":1,"raw":"[Cai and Vasconcelos, 2018] Zhaowei Cai and Nuno Vas- concelos. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 6154-6162, 2018.","title":"Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection","authors":"Vasconcelos Cai","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2017] Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker. Learning effi- cient object detection models with knowledge distillation. Advances in neural information processing systems, 30, 2017.","title":"Learning efficient object detection models with knowledge distillation","authors":"Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, Manmohan Chandraker","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019] Yunpeng Chen, Marcus Rohrbach, Zhicheng Yan, Yan Shuicheng, Jiashi Feng, and Yannis Kalantidis. Graph-based global reasoning networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 433-442, 2019.","title":"Graph-based global reasoning networks","authors":"Yunpeng Chen, Marcus Rohrbach, Zhicheng Yan, Yan Shuicheng, Jiashi Feng, Yannis Kalantidis","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dai et al., 2021] Xing Dai, Zeren Jiang, Zhao Wu, Yiping Bao, Zhicheng Wang, Si Liu, and Erjin Zhou. General in- stance distillation for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 7842-7851, 2021.","title":"General instance distillation for object detection","authors":"Xing Dai, Zeren Jiang, Zhao Wu, Yiping Bao, Zhicheng Wang, Si Liu, Erjin Zhou","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du et al., 2021] Zhixing Du, Rui Zhang, Ming-Fang Chang, Xishan Zhang, Shaoli Liu, Tianshi Chen, and Yunji Chen. Distilling object detectors with feature richness. In Neural Information Processing Systems, 2021.","title":"Distilling object detectors with feature richness","authors":"Zhixing Du, Rui Zhang, Ming-Fang Chang, Xishan Zhang, Shaoli Liu, Tianshi Chen, Yunji Chen","year":"2021","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Duan et al., 2019] Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 6569-6578, 2019.","title":"Centernet: Keypoint triplets for object detection","authors":"Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, Qi Tian","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fu et al., 2019] Jun Fu, Jing Liu, Haijie Tian, Yong Li, Y- ongjun Bao, Zhiwei Fang, and Hanqing Lu. Dual atten- tion network for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), June 2019.","title":"Dual attention network for scene segmentation","authors":"Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, Hanqing Lu","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gong et al., 2019] Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4852-4861, 2019.","title":"Differentiable soft quantization: Bridging full-precision and low-bit neural networks","authors":"Gong","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo et al., 2021] Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, and Chang Xu. Distilling object detectors via decoupled features. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2154-2164, 2021.","title":"Distilling object detectors via decoupled features","authors":"Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, Chang Xu","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Han et al., 2015] Song Han, Jeff Pool, John Tran, and William J Dally. Learning both weights and connections for efficient neural network. In NIPS, 2015.","title":"Learning both weights and connections for efficient neural network","authors":"Han","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Comput- er Vision and Pattern Recognition (CVPR), June 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Heo et al., 2019] Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi. A comprehensive overhaul of feature distillation. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 1921-1930, 2019.","title":"A comprehensive overhaul of feature distillation","authors":"Heo","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Law and Deng, 2018] Hei Law and Jia Deng. Cornernet: Detecting objects as paired keypoints. In Proceedings of the European conference on computer vision (ECCV), pages 734-750, 2018.","title":"Cornernet: Detecting objects as paired keypoints","authors":"Deng ; ] Hei Law, Jia Law, Deng","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In European conference on com- puter vision, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense ob- ject detection. In Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2020] Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao. Hrank: Filter pruning using high-rank fea- ture map. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1529- 1538, 2020.","title":"Hrank: Filter pruning using high-rank feature map","authors":"Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, Ling Shao","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2016] Wei Liu, Dragomir Anguelov, Dumitru Er- han, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference on computer vision, pages 21-37. Springer, 2016.","title":"Ssd: Single shot multibox detector","authors":"Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C Berg","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lu et al., 2019] Xin Lu, Buyu Li, Yuxin Yue, Quanquan Li, and Junjie Yan. Grid r-cnn. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7363-7372, 2019.","title":"Grid r-cnn","authors":"Xin Lu, Buyu Li, Yuxin Yue, Quanquan Li, Junjie Yan","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Redmon and Farhadi, 2018] Joseph Redmon and Ali Farha- di. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.","title":"Yolov3: An incremental improvement","authors":"Joseph Redmon, Ali Farhadi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. Advances in neural information processing systems, 28:91-99, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2020] Ruoyu Sun, Fuhui Tang, Xiaopeng Zhang, Hongkai Xiong, and Qi Tian. Distilling object detectors with task adaptive regularization. arXiv preprint arX- iv:2006.13108, 2020.","title":"Distilling object detectors with task adaptive regularization","authors":"Ruoyu Sun, Fuhui Tang, Xiaopeng Zhang, Hongkai Xiong, Qi Tian","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tang et al., 2022] Sanli Tang, Zhongyu Zhang, Zhanzhan Cheng, Jing Lu, Yunlu Xu, Yi Niu, and Fan He. Distill- ing object detectors with global knowledge. In European Conference on Computer Vision, 2022.","title":"Distilling object detectors with global knowledge","authors":"Sanli Tang, Zhongyu Zhang, Zhanzhan Cheng, Jing Lu, Yunlu Xu, Yi Niu, Fan He","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tian et al., 2019] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object de- tection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9627-9636, 2019.","title":"Fcos: Fully convolutional one-stage object detection","authors":"Chunhua Zhi Tian, Hao Shen, Tong Chen, He","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2018] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7794-7803, 2018.","title":"Non-local neural networks","authors":"Xiaolong Wang, Ross Girshick, Abhinav Gupta, Kaiming He","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2019] Tao Wang, Li Yuan, Xiaopeng Zhang, and Jiashi Feng. Distilling object detectors with fine- grained feature imitation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4933-4942, 2019.","title":"Distilling object detectors with finegrained feature imitation","authors":"Tao Wang, Li Yuan, Xiaopeng Zhang, Jiashi Feng","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2017] Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), July 2017.","title":"Aggregated residual transformations for deep neural networks","authors":"Xie","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2019] Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. Reppoints: Point set repre- sentation for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9657-9666, 2019.","title":"Reppoints: Point set representation for object detection","authors":"Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, Stephen Lin","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2022] Zhendong Yang, Zhe Li, Xiaohu Jiang, Yuan Gong, Zehuan Yuan, Danpei Zhao, and Chun Yuan. Focal and global knowledge distillation for detectors. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4643-4652, 2022.","title":"Focal and global knowledge distillation for detectors","authors":"Zhendong Yang, Zhe Li, Xiaohu Jiang, Yuan Gong, Zehuan Yuan, Danpei Zhao, Chun Yuan","year":"2022","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Ma, 2021] Linfeng Zhang and Kaisheng Ma. Improve object detection with feature-based knowledge distillation: Towards accurate and efficient detectors. In International Conference on Learning Representations, 2021.","title":"Improve object detection with feature-based knowledge distillation: Towards accurate and efficient detectors","authors":"Ma ; Zhang, Linfeng Zhang, Kaisheng Ma","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2020] Hongkai Zhang, Hong Chang, Bing- peng Ma, Naiyan Wang, and Xilin Chen. Dynamic r-cnn: Towards high quality object detection via dynamic train- ing. In European Conference on Computer Vision, pages 260-275. Springer, 2020.","title":"Dynamic r-cnn: Towards high quality object detection via dynamic training","authors":"Hongkai Zhang, Hong Chang, Bingpeng Ma, Naiyan Wang, Xilin Chen","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2021] Xiangguo Zhang, Haotong Qin, Yifu D- ing, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, and Xianglong Liu. Diversifying sample generation for accurate data-free quantization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15658-15667, 2021.","title":"Diversifying sample generation for accurate data-free quantization","authors":"Zhang","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}