{"type":"ScholarlyArticle","title":"Conflict-Alleviated Gradient Descent for Adaptive Object Detection","authors":[{"name":"Wenxu Shi","slug":"wenxu-shi","given":"Wenxu","family":"Shi","affiliation":"School of Microelectronics and Communication Engineering , Chongqing University , China"},{"name":"Bochuan Zheng","slug":"bochuan-zheng","given":"Bochuan","family":"Zheng","affiliation":"School of Computer Science , China West Normal University , China"}],"abstract":"Unsupervised domain adaptive object detection (DAOD) aims to adapt detectors from a labeled source domain to an unlabelled target domain. Existing DAOD works learn feature representations with both class discriminative and domain invariant by jointly  minimizing the loss across domain alignment and detection tasks. However, joint resolution of different tasks may lead to conflicts, with one contributing factor being gradient conflicts during optimization. If left untouched, such disagreement may degrade adaptation performance. In this work, we propose an efficient optimization strategy named Conflict-Alleviated  Gradient descent (CAGrad) which aims to  alleviate the conflict between two tasks (i.e., alignment and classification). Particularly, we alter the gradients by projecting each onto the normal plane of the other. The projection operation changes conflicting gradients from obtuse angles to acute angles, thus alleviating the conflict and achieving gradient harmonization. We further validate our theoretical analysis and methods on several domain adaptive object detection tasks, including cross-camera, weather, scene, and synthetic to real-world adaptation. Extensive experiments on multiple DAOD benchmarks demonstrate the effectiveness and superiority of our CAGrad.","doi":"10.24963/ijcai.2024/137","url":"https://ijcai.org/proceedings/2024/137","pdf":"https://ijcai.org/proceedings/2024/0137.pdf","markdown":"https://ijcai.org/proceedings/2024/137.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/137","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":137,"pages":{"first":1236,"last":1244},"keywords":["Machine Learning: ML: Unsupervised learning","Machine Learning: ML: Optimization","Computer Vision: CV: Recognition (object detection, categorization)"],"videos":[],"references":[{"position":1,"raw":"[Cai and Vasconcelos, 2018] Zhaowei Cai and Nuno Vas- concelos. Cascade r-cnn: Delving into high quality object detection. In CVPR, pages 6154-6162, 2018.","title":"Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection","authors":"Vasconcelos Cai","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213-229, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In CVPR, pages 3339-3348, 2018.","title":"Domain adaptive faster r-cnn for object detection in the wild","authors":"Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, Luc Van Gool","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Chaoqi Chen, Zebiao Zheng, Xinghao Ding, Yue Huang, and Qi Dou. Harmonizing transferabil- ity and discriminability for adapting object detectors. In CVPR, pages 8869-8878, 2020.","title":"Harmonizing transferability and discriminability for adapting object detectors","authors":"Chen","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cordts et al., 2016a] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene under- standing. In CVPR, pages 3213-3223, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Cordts","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cordts et al., 2016b] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene under- standing. In CVPR, pages 3213-3223, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Cordts","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cui et al., 2020] Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. Towards dis- criminability and diversity: Batch nuclear-norm maxi- mization under label insufficient situations. In CVPR, pages 3941-3950, 2020.","title":"Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations","authors":"Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, Qi Tian","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Deng et al., 2021] Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan. Unbiased mean teacher for cross-domain ob- ject detection. In CVPR, pages 4091-4101, 2021.","title":"Unbiased mean teacher for cross-domain object detection","authors":"Jinhong Deng, Wen Li, Yuhua Chen, Lixin Duan","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ganin and Lempitsky, 2015] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backprop- agation. In ICML, pages 1180-1189, 2015.","title":"Unsupervised domain adaptation by backpropagation","authors":"Yaroslav Ganin, Victor Lempitsky","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The jour- nal of machine learning research, 17(1):2096-2030, 2016.","title":"Domain-adversarial training of neural networks","authors":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Mario Franc ¸ois Laviolette, Victor Marchand, Lempitsky","year":"2016","venue":"The journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Geiger et al., 2012] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, pages 3354- 3361, 2012.","title":"Are we ready for autonomous driving? the kitti vision benchmark suite","authors":"Andreas Geiger, Philip Lenz, Raquel Urtasun","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He and Zhang, 2019] Zhenwei He and Lei Zhang. Multi- adversarial faster-rcnn for unrestricted object detection. In ICCV, pages 6668-6677, 2019.","title":"Multiadversarial faster-rcnn for unrestricted object detection","authors":"Zhenwei He, Lei Zhang","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He and Zhang, 2020] Zhenwei He and Lei Zhang. Domain adaptive object detection via asymmetric tri-way faster- rcnn. In ECCV, pages 309-324, 2020.","title":"Domain adaptive object detection via asymmetric tri-way fasterrcnn","authors":"Zhenwei He, Lei Zhang","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In CVPR, pages 2961-2969, 2017.","title":"Kaiming He, Georgia Gkioxari","authors":"He","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[He et al., 2022] Zhenwei He, Lei Zhang, Yi Yang, and Xinbo Gao. Partial alignment for object detection in the wild. IEEE Transactions on Circuits and Systems for Video Technology, 32(8):5238-5251, 2022.","title":"Partial alignment for object detection in the wild","authors":"Zhenwei He, Lei Zhang, Yi Yang, Xinbo Gao","year":"2022","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[He et al., 2023] Zhenwei He, Lei Zhang, Xinbo Gao, and David Zhang. Multi-adversarial faster-rcnn with paradigm teacher for unrestricted object detection. International Journal of Computer Vision, 131(3):680-700, 2023.","title":"Multi-adversarial faster-rcnn with paradigm teacher for unrestricted object detection","authors":"Zhenwei He, Lei Zhang, Xinbo Gao, David Zhang","year":"2023","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Huang et al., 2022] Wei-Jie Huang, Yu-Lin Lu, Shih-Yao Lin, Yusheng Xie, and Yen-Yu Lin. Aqt: Adversarial query transformers for domain adaptive object detection. In IJCAI, pages 972-979. International Joint Conferences on Artificial Intelligence, 2022.","title":"Aqt: Adversarial query transformers for domain adaptive object detection","authors":"Wei-Jie Huang, Yu-Lin Lu, Shih-Yao Lin, Yusheng Xie, Yen-Yu Lin","year":"2022","venue":"International Joint Conferences on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2022/136"},{"position":19,"raw":"[Johnson-Roberson et al., 2016] Matthew Johnson- Roberson, Charles Barto, Rounak Mehta, Sharath Nittur Sridhar, Karl Rosaen, and Ram Vasudevan. Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks? arXiv preprint arXiv:1610.01983, 2016.","title":"Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks","authors":"Matthew Johnson-Roberson, Charles Barto, Rounak Mehta, Nittur Sharath, Karl Sridhar, Ram Rosaen, Vasudevan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kang et al., 2019] Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In CVPR, pages 4893-4902, 2019.","title":"Contrastive adaptation network for unsupervised domain adaptation","authors":"Guoliang Kang, Lu Jiang, Yi Yang, Alexander G Hauptmann","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lang et al., 2023] Qinghai Lang, Lei Zhang, Wenxu Shi, Weijie Chen, and Shiliang Pu. Exploring implicit domain- invariant features for domain adaptive object detection. IEEE Transactions on Circuits and Systems for Video Technology, 33(4):1816-1826, 2023.","title":"Exploring implicit domaininvariant features for domain adaptive object detection","authors":"Lang","year":"2023","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2016] Wei Liu, Dragomir Anguelov, Dumitru Er- han, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In ECCV, pages 21-37. Springer, 2016.","title":"Ssd: Single shot multibox detector","authors":"Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C Berg","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu et al., 2023] Yaoyao Liu, Bernt Schiele, Andrea Vedaldi, and Christian Rupprecht. Continual detection transformer for incremental object detection. In CVPR, pages 23799-23808, 2023.","title":"Continual detection transformer for incremental object detection","authors":"Liu","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pan and Yang, 2009] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowl- edge and data engineering, 22(10):1345-1359, 2009.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2009","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Redmon and Farhadi, 2018] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.","title":"Yolov3: An incremental improvement","authors":"Joseph Redmon, Ali Farhadi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Redmon et al., 2016] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Uni- fied, real-time object detection. In CVPR, pages 779-788, 2016.","title":"You only look once: Unified, real-time object detection","authors":"Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. Advances in neural information processing systems, 28:91-99, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sakaridis et al., 2018a] Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene understanding with synthetic data. International Journal of Computer Vi- sion, 126:973-992, 2018.","title":"Semantic foggy scene understanding with synthetic data","authors":"Sakaridis","year":"2018","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sakaridis et al., 2018b] Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene understanding with synthetic data. International Journal of Computer Vi- sion, 126:973-992, 2018.","title":"Semantic foggy scene understanding with synthetic data","authors":"Sakaridis","year":"2018","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tian et al., 2019] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object de- tection. In iccv, pages 9627-9636, 2019.","title":"Fcos: Fully convolutional one-stage object detection","authors":"Chunhua Zhi Tian, Hao Shen, Tong Chen, He","year":"2019","venue":"iccv","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2021a] Wen Wang, Yang Cao, Jing Zhang, Fengxiang He, Zheng-Jun Zha, Yonggang Wen, and Dacheng Tao. Exploring sequence feature alignment for domain adaptive detection transformers. In ACM MM, pages 1730-1738, 2021.","title":"Exploring sequence feature alignment for domain adaptive detection transformers","authors":"Wang","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2021b] Wen Wang, Yang Cao, Jing Zhang, Fengxiang He, Zheng-Jun Zha, Yonggang Wen, and Dacheng Tao. Exploring sequence feature alignment for domain adaptive detection transformers. In ACM MM, pages 1730-1738, 2021.","title":"Exploring sequence feature alignment for domain adaptive detection transformers","authors":"Wang","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wu et al., 2021] Aming Wu, Rui Liu, Yahong Han, Linchao Zhu, and Yi Yang. Vector-decomposed disentanglement for domain-invariant object detection. In ICCV, pages 9342-9351, 2021.","title":"Vector-decomposed disentanglement for domain-invariant object detection","authors":"Aming Wu, Rui Liu, Yahong Han, Linchao Zhu, Yi Yang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wu et al., 2022] Aming Wu, Yahong Han, Linchao Zhu, and Yi Yang. Instance-invariant domain adaptive object detection via progressive disentanglement. IEEE TPAMI, 44(8):4178-4193, 2022.","title":"Instance-invariant domain adaptive object detection via progressive disentanglement","authors":"Aming Wu, Yahong Han, Linchao Zhu, Yi Yang","year":"2022","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2020] Minghao Xu, Hang Wang, Bingbing Ni, Qi Tian, and Wenjun Zhang. Cross-domain detection via graph-induced prototype alignment. In CVPR, pages 12355-12364, 2020.","title":"Cross-domain detection via graph-induced prototype alignment","authors":"Minghao Xu, Hang Wang, Bingbing Ni, Qi Tian, Wenjun Zhang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yu et al., 2020] Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driv- ing dataset for heterogeneous multitask learning. In CVPR, pages 2636-2645, 2020.","title":"Bdd100k: A diverse driving dataset for heterogeneous multitask learning","authors":"Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, Trevor Darrell","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhu et al., 2020] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020.","title":"Deformable detr: Deformable transformers for end-to-end object detection","authors":"Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}