{"type":"ScholarlyArticle","title":"D3ETR: Decoder Distillation for Detection Transformer","authors":[{"name":"Xiaokang Chen","slug":"xiaokang-chen","given":"Xiaokang","family":"Chen","affiliation":"National Key Laboratory of General Artificial Intelligence , School of IST , Peking University"},{"name":"Jiahui Chen","slug":"jiahui-chen","given":"Jiahui","family":"Chen","affiliation":"Beihang University"},{"name":"Yan Liu","slug":"yan-liu","given":"Yan","family":"Liu","affiliation":"The Chinese University of Hong Kong"},{"name":"Jiaxiang Tang","slug":"jiaxiang-tang","given":"Jiaxiang","family":"Tang","affiliation":"National Key Laboratory of General Artificial Intelligence , School of IST , Peking University"},{"name":"Gang Zeng","slug":"gang-zeng","given":"Gang","family":"Zeng","affiliation":"National Key Laboratory of General Artificial Intelligence , School of IST , Peking University"}],"abstract":"Although various knowledge distillation (KD) methods for CNN-based detectors have been proven effective in improving small students, build- ing baselines and recipes for DETR-based detec- tors remains a challenge. This paper concentrates on the transformer decoder of DETR-based detec- tors and explores KD methods suitable for them. However, the random order of the decoder outputs poses a challenge for knowledge distillation as it provides no direct correspondence between the pre- dictions of the teacher and the student. To this end, we propose MixMatcher that aligns the de- coder outputs of DETR-based teacher and student, by mixing two teacher-student matching strategies for combined advantages. The first strategy, Adap- tive Matching, applies bipartite matching to adap- tively match the outputs of the teacher and the stu- dent in each decoder layer. The second strategy, Fixed Matching, fixes the correspondence between the outputs of the teacher and the student with the same object queries as input, which alleviates in- stability of bipartite matching in Adaptive Match- ing. Using both strategies together produces bet- ter results than using either strategy alone. Based on MixMatcher, we devise Decoder Distillation for DEtection TRansformer (D3ETR), which dis- tills knowledge in decoder predictions and attention maps from the teacher to student. D3ETR shows superior performance on various DETR-based de- tectors with different backbones. For instance, D3ETR improves Conditional DETR-R50-C5 by 8.3 mAP under 12 epochs training setting with Conditional DETR-R101-C5 serving as the teacher. The code will be released.","doi":"10.24963/ijcai.2024/74","url":"https://ijcai.org/proceedings/2024/74","pdf":"https://ijcai.org/proceedings/2024/0074.pdf","markdown":"https://ijcai.org/proceedings/2024/74.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/74","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":74,"pages":{"first":668,"last":676},"keywords":["Computer Vision: CV: Applications","Computer Vision: CV: Recognition (object detection, categorization)"],"videos":[],"references":[{"position":1,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020.","title":"End-to-end object detection with transformers","authors":"Carion","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2017] Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker. Learning effi- cient object detection models with knowledge distillation. Advances in neural information processing systems, 30, 2017.","title":"Learning efficient object detection models with knowledge distillation","authors":"Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, Manmohan Chandraker","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021] Pengguang Chen, Shu Liu, Hengshuang Zhao, and Jiaya Jia. Distilling knowledge via knowledge review. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5008- 5017, 2021.","title":"Distilling knowledge via knowledge review","authors":"Chen","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2022a] Qiang Chen, Xiaokang Chen, Jian Wang, Haocheng Feng, Junyu Han, Errui Ding, Gang Zeng, and Jingdong Wang. Group detr: Fast detr training with group-wise one-to-many assignment. 2022.","title":"Group detr: Fast detr training with group-wise one-to-many assignment","authors":"Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022b] Qiang Chen, Jian Wang, Chuchu Han, Shan Zhang, Zexian Li, Xiaokang Chen, Jiahui Chen, Xiaodi Wang, Shuming Han, Gang Zhang, Haocheng Feng, Kun Yao, Junyu Han, Errui Ding, and Jingdong Wang. Group detr v2: Strong object detector with encoder- decoder pretraining. 2022.","title":"Group detr v2: Strong object detector with encoderdecoder pretraining","authors":"Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2022c] Xiaokang Chen, Fangyun Wei, Gang Zeng, and Jingdong Wang. Conditional detr v2: Efficient detection transformer with box queries. arXiv preprint arXiv:2207.08914, 2022.","title":"Conditional detr v2: Efficient detection transformer with box queries","authors":"Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cho and Hariharan, 2019] Jang Hyun Cho and Bharath Har- iharan. On the efficacy of knowledge distillation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4794-4802, 2019.","title":"On the efficacy of knowledge distillation","authors":"Hariharan ; Cho, Hyun Jang, Bharath Cho, Hariharan","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dai et al., 2021] Xing Dai, Zeren Jiang, Zhao Wu, Yiping Bao, Zhicheng Wang, Si Liu, and Erjin Zhou. General in- stance distillation for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 7842-7851, 2021.","title":"General instance distillation for object detection","authors":"Xing Dai, Zeren Jiang, Zhao Wu, Yiping Bao, Zhicheng Wang, Si Liu, Erjin Zhou","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dong et al., 2022] Na Dong, Yongqiang Zhang, Mingli Ding, and Gim Hee Lee. Incremental-detr: Incremen- tal few-shot object detection via self-supervised learning. arXiv preprint arXiv:2205.04042, 2022.","title":"Incremental-detr: Incremental few-shot object detection via self-supervised learning","authors":"Na Dong, Yongqiang Zhang, Mingli Ding, Gim Hee, Lee","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Glorot and Bengio, 2010] Xavier Glorot and Yoshua Ben- gio. Understanding the difficulty of training deep feed- forward neural networks. In AISTATS, 2010.","title":"Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks","authors":"Bengio Glorot","year":"2010","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2021] Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, and Chang Xu. Distilling object detectors via decoupled features. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2154-2164, 2021.","title":"Distilling object detectors via decoupled features","authors":"Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, Chang Xu","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hao et al., 2020] Miao Hao, Yitao Liu, Xiangyu Zhang, and Jian Sun. Labelenc: A new intermediate supervision method for object detection. In European Conference on Computer Vision, pages 529-545. Springer, 2020.","title":"Labelenc: A new intermediate supervision method for object detection","authors":"Miao Hao, Yitao Liu, Xiangyu Zhang, Jian Sun","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hinton et al., 2015] Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural net- work. arXiv: Machine Learning, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey E Hinton, Oriol Vinyals, Jeffrey Dean","year":"2015","venue":"arXiv: Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2022] Ding Jia, Yuhui Yuan, Haodi He, Xiaopei Wu, Haojun Yu, Weihong Lin, Lei Sun, Chao Zhang, and Han Hu. Detrs with hybrid matching. arXiv preprint arXiv:2207.13080, 2022.","title":"Detrs with hybrid matching","authors":"Ding Jia, Yuhui Yuan, Haodi He, Xiaopei Wu, Haojun Yu, Weihong Lin, Lei Sun, Chao Zhang, Han Hu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jiahao et al., 2022] Chang Jiahao, Wang Shuo, Xu Guangkai, Chen Zehui, Yang Chenhongyi, and Zhao Feng. Detrdistill: A simple knowledge distillation framework for detr-families. In ICLR2023 submission, 2022.","title":"Detrdistill: A simple knowledge distillation framework for detr-families","authors":"Chang Jiahao, Wang Shuo, Xu Guangkai, Chen Zehui, Yang Chenhongyi, Zhao Feng","year":"2022","venue":"ICLR2023 submission","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kuhn, 1955] Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 1955.","title":"The hungarian method for the assignment problem","authors":"Harold W Kuhn","year":"1955","venue":"Naval research logistics quarterly","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2017] Quanquan Li, Shengying Jin, and Junjie Yan. Mimicking very efficient network for object detec- tion. In Proceedings of the ieee conference on computer vision and pattern recognition, pages 6356-6364, 2017.","title":"Mimicking very efficient network for object detection","authors":"Li","year":"2017","venue":"Proceedings of the ieee conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2022] Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. Dn-detr: Accelerate detr training by introducing query denoising. In CVPR, 2022.","title":"Dn-detr: Accelerate detr training by introducing query denoising","authors":"Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, Lei Zhang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin et al., 2014a] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Com- mon objects in context. In ECCV, 2014.","title":"Microsoft COCO: Common objects in context","authors":"Lin","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lin et al., 2014b] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In ECCV, 2014.","title":"Microsoft COCO: common objects in context","authors":"Lin","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2022] Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint arXiv:2201.12329, 2022.","title":"Dab-detr: Dynamic anchor boxes are better queries for detr","authors":"Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Loshchilov and Hutter, 2017] Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. In ICLR, 2017.","title":"Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam","authors":"Hutter Loshchilov","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Meng et al., 2021] Depu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng, Houqiang Li, Yuhui Yuan, Lei Sun, and Jing- dong Wang. Conditional detr for fast training convergence. In ICCV, pages 3651-3660, 2021.","title":"Conditional detr for fast training convergence","authors":"Depu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng, Houqiang Li, Yuhui Yuan, Lei Sun, Jingdong Wang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Park et al., 2019] Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3967-3976, 2019.","title":"Relational knowledge distillation","authors":"Park","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rezatofighi et al., 2019] Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian D. Reid, and Silvio Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In CVPR, 2019.","title":"Generalized intersection over union: A metric and a loss for bounding box regression","authors":"Hamid Rezatofighi, Nathan Tsoi, Junyoung Gwak, Amir Sadeghian, Ian D Reid, Silvio Savarese","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Romero et al., 2014] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014.","title":"Fitnets: Hints for thin deep nets","authors":"Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sandler et al., 2018] Mark Sandler, Andrew Howard, Men- glong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR, 2018.","title":"Mobilenetv2: Inverted residuals and linear bottlenecks","authors":"Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Song et al., 2022] Hwanjun Song, Deqing Sun, Sanghyuk Chun, Varun Jampani, Dongyoon Han, Byeongho Heo, Wonjae Kim, and Ming-Hsuan Yang. Vidt: An efficient and effective fully transformer-based object detector. In ICLR, 2022.","title":"Vidt: An efficient and effective fully transformer-based object detector","authors":"Song","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Tian et al., 2019] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive representation distillation. arXiv preprint arXiv:1910.10699, 2019.","title":"Contrastive representation distillation","authors":"Yonglong Tian, Dilip Krishnan, Phillip Isola","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tung and Mori, 2019] Frederick Tung and Greg Mori. Similarity-preserving knowledge distillation. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 1365-1374, 2019.","title":"Similarity-preserving knowledge distillation","authors":"Frederick Tung, Greg Mori","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2022] Yingming Wang, Xiangyu Zhang, Tong Yang, and Jian Sun. Anchor detr: Query design for transformer-based detector. In AAAI, 2022.","title":"Anchor detr: Query design for transformer-based detector","authors":"Wang","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang et al., 2021] Jing Yang, Brais Martinez, Adrian Bulat, Georgios Tzimiropoulos, et al. Knowledge distillation via softmax regression representation learning. International Conference on Learning Representations (ICLR), 2021.","title":"Knowledge distillation via softmax regression representation learning","authors":"Jing Yang, Brais Martinez, Adrian Bulat, Georgios Tzimiropoulos","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yang et al., 2022a] Zhendong Yang, Zhe Li, Xiaohu Jiang, Yuan Gong, Zehuan Yuan, Danpei Zhao, and Chun Yuan. Focal and global knowledge distillation for detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4643-4652, 2022.","title":"Focal and global knowledge distillation for detectors","authors":"Yang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2022b] Zhendong Yang, Zhe Li, Mingqi Shao, Dachuan Shi, Zehuan Yuan, and Chun Yuan. Masked generative distillation. arXiv preprint arXiv:2205.01529, 2022.","title":"Masked generative distillation","authors":"Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yim et al., 2017] Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4133-4141, 2017.","title":"A gift from knowledge distillation: Fast optimization, network minimization and transfer learning","authors":"Yim","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zagoruyko and Komodakis, 2016] Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Im- proving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928, 2016.","title":"Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer","authors":"Komodakis Zagoruyko, Sergey Zagoruyko, Nikos Komodakis","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang and Ma, 2020] Linfeng Zhang and Kaisheng Ma. Improve object detection with feature-based knowledge distillation: Towards accurate and efficient detectors. In International Conference on Learning Representations, 2020.","title":"Improve object detection with feature-based knowledge distillation: Towards accurate and efficient detectors","authors":"Ma ; Zhang, Linfeng Zhang, Kaisheng Ma","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang et al., 2018] Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. Deep mutual learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4320-4328, 2018.","title":"Deep mutual learning","authors":"Ying Zhang, Tao Xiang, Timothy M Hospedales, Huchuan Lu","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2022a] Gongjie Zhang, Zhipeng Luo, Yingchen Yu, Kaiwen Cui, and Shijian Lu. Acceler- ating detr convergence via semantic-aligned matching. In CVPR, pages 949-958, 2022.","title":"Accelerating detr convergence via semantic-aligned matching","authors":"Zhang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang et al., 2022b] Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising an- chor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022.","title":"Dino: Detr with improved denoising anchor boxes for end-to-end object detection","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhang et al., 2022c] Peizhen Zhang, Zijian Kang, Tong Yang, Xiangyu Zhang, Nanning Zheng, and Jian Sun. Lgd: label-guided self-distillation for object detection. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3309-3317, 2022.","title":"Lgd: label-guided self-distillation for object detection","authors":"Zhang","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhao et al., 2022] Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang. Decoupled knowledge dis- tillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11953- 11962, 2022.","title":"Decoupled knowledge distillation","authors":"Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, Jiajun Liang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zheng et al., 2021] Nanning Zheng, Jian Sun, Xiangyu Zhang, Zijian Kang, and Peizhen Zhang. Instance- conditional knowledge distillation for object detection. 2021.","title":"Instanceconditional knowledge distillation for object detection","authors":"Nanning Zheng, Jian Sun, Xiangyu Zhang, Zijian Kang, Peizhen Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Zhou et al., 2021] Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou, Guoli Wang, Junsong Yuan, and Qian Zhang. Rethinking soft labels for knowledge distilla- tion: A bias-variance tradeoff perspective. arXiv preprint arXiv:2102.00650, 2021.","title":"Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective","authors":"Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou, Guoli Wang, Junsong Yuan, Qian Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zhu et al., 2020] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: deformable transformers for end-to-end object detection. CoRR, abs/2010.04159, 2020.","title":"Deformable DETR: deformable transformers for end-to-end object detection","authors":"Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}