{"type":"ScholarlyArticle","title":"Augmenting Anchors by the Detector Itself","authors":[{"name":"Xiaopei Wan","slug":"xiaopei-wan","given":"Xiaopei","family":"Wan","affiliation":""},{"name":"Guoqiu Li","slug":"guoqiu-li","given":"Guoqiu","family":"Li","affiliation":"Tsinghua University"},{"name":"Yujiu Yang","slug":"yujiu-yang","given":"Yujiu","family":"Yang","affiliation":"Tsinghua University"},{"name":"Zhenhua Guo","slug":"zhenhua-guo","given":"Zhenhua","family":"Guo","affiliation":"Alibaba Group"}],"abstract":"Usually, it is difficult to determine the scale and aspect ratio of anchors for anchor-based object detection methods. Current state-of-the-art object detectors either determine anchor parameters according to objects' shape and scale in a dataset, or avoid this problem by utilizing anchor-free methods, however, the former scheme is dataset-specific and the latter methods could not get better performance than the former ones. In this paper, we propose a novel anchor augmentation method named AADI, which means Augmenting Anchors by the Detector Itself. AADI is not an anchor-free method, instead, it can convert the scale and aspect ratio of anchors from a continuous space to a discrete space, which greatly alleviates the problem of anchors' designation. Furthermore, AADI is a learning-based anchor augmentation method, but it does not add any parameters or hyper-parameters, which is beneficial for research and downstream tasks. Extensive experiments on COCO dataset demonstrate the effectiveness of AADI, specifically, AADI achieves significant performance boosts on many state-of-the-art object detectors (eg. at least +2.4 box AP on Faster R-CNN, +2.2 box AP on Mask R-CNN, and +0.9 box AP on Cascade Mask R-CNN). We hope that this simple and cost-efficient method can be widely used in object detection. Code and models are available at https://github.com/WanXiaopei/aadi.","doi":"10.24963/ijcai.2022/191","url":"https://ijcai.org/proceedings/2022/191","pdf":"https://ijcai.org/proceedings/2022/0191.pdf","markdown":"https://ijcai.org/proceedings/2022/191.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/191","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":191,"pages":{"first":1371,"last":1377},"keywords":["Computer Vision: Recognition (object detection, categorization)"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986665","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984808","length_sec":299,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Cai and Vasconcelos, 2018] Zhaowei Cai and Nuno Vas- concelos. Cascade r-cnn: Delving into high quality object detection. In CVPR, 2018.","title":"Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection","authors":"Vasconcelos Cai","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV. Springer, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2020] Yihong Chen, Zheng Zhang, Yue Cao, Liwei Wang, Stephen Lin, and Han Hu. Reppoints v2: Verification meets regression for object detection. In NeurIPS, 2020.","title":"Reppoints v2: Verification meets regression for object detection","authors":"Yihong Chen, Zheng Zhang, Yue Cao, Liwei Wang, Stephen Lin, Han Hu","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dai et al., 2017] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. De- formable convolutional networks. In ICCV, 2017.","title":"Deformable convolutional networks","authors":"Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2009] J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"J Deng, W Dong, R Socher, L Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan and Ling, 2019] Heng Fan and Haibin Ling. Siamese cascaded region proposal networks for real-time visual tracking. In CVPR, 2019.","title":"Siamese cascaded region proposal networks for real-time visual tracking","authors":"Heng Fan, Haibin Ling","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Feng et al., 2021] Chengjian Feng, Yujie Zhong, Yu Gao, Matthew R Scott, and Weilin Huang. Tood: Task-aligned one-stage object detection. In ICCV, 2021.","title":"Tood: Task-aligned one-stage object detection","authors":"Chengjian Feng, Yujie Zhong, Yu Gao, Matthew R Scott, Weilin Huang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gidaris and Komodakis, 2016] Spyros Gidaris and Nikos Komodakis. Attend refine repeat: Active box pro- posal generation via in-out localization. arXiv preprint arXiv:1606.04446, 2016.","title":"Attend refine repeat: Active box proposal generation via in-out localization","authors":"Komodakis Gidaris, Spyros Gidaris, Nikos Komodakis","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Girshick et al., 2014] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014.","title":"Rich feature hierarchies for accurate object detection and semantic segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Girshick, 2015] Ross Girshick. Fast r-cnn. In ICCV, 2015.","title":"Fast r-cnn","authors":"Ross Girshick","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017.","title":"Kaiming He, Georgia Gkioxari","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang et al., 2018] Borui Jiang, Ruixuan Luo, Jiayuan Mao, Tete Xiao, and Yuning Jiang. Acquisition of localization confidence for accurate object detection. In ECCV, 2018.","title":"Acquisition of localization confidence for accurate object detection","authors":"Jiang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kim and Lee, 2020] Kang Kim and Hee Seok Lee. Prob- abilistic anchor assignment with iou prediction for object detection. In ECCV, 2020.","title":"Probabilistic anchor assignment with iou prediction for object detection","authors":"Kang Kim, Hee Seok, Lee","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2019] Hongyang Li, Yu Liu, Wanli Ouyang, and Xiaogang Wang. Zoom out-and-in network with map at- tention decision for region proposal and object detection. International Journal of Computer Vision, 127(3):225- 238, 2019.","title":"Zoom out-and-in network with map attention decision for region proposal and object detection","authors":"Hongyang Li, Yu Liu, Wanli Ouyang, Xiaogang Wang","year":"2019","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In ECCV. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Fea- ture pyramid networks for object detection. In CVPR, 2017.","title":"Feature pyramid networks for object detection","authors":"Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, Serge Belongie","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103.14030, 2021.","title":"Swin transformer: Hierarchical vision transformer using shifted windows","authors":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Micikevicius et al., 2017] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv:1710.03740, 2017.","title":"Mixed precision training","authors":"Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Paszke et al., 2017] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De- Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.","title":"Automatic differentiation in pytorch","authors":"Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De-Vito, Zeming Lin, Alban Desmaison, Luca Antiga, Adam Lerer","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ren et al., 2016] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6):1137-1149, 2016.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2021] Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. Sparse r-cnn: End- to-end object detection with learnable proposals. In CVPR, 2021.","title":"Sparse r-cnn: Endto-end object detection with learnable proposals","authors":"Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tian et al., 2019] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object de- tection. In ICCV, 2019.","title":"Fcos: Fully convolutional one-stage object detection","authors":"Chunhua Zhi Tian, Hao Shen, Tong Chen, He","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2019] Jiaqi Wang, Kai Chen, Shuo Yang, Chen Change Loy, and Dahua Lin. Region proposal by guided anchoring. In CVPR, 2019.","title":"Region proposal by guided anchoring","authors":"Jiaqi Wang, Kai Chen, Shuo Yang, Chen Change Loy, Dahua Lin","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2021] Jianfeng Wang, Lin Song, Zeming Li, Hongbin Sun, Jian Sun, and Nanning Zheng. End-to- end object detection with fully convolutional network. In CVPR, 2021.","title":"End-toend object detection with fully convolutional network","authors":"Jianfeng Wang, Lin Song, Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu and He, 2018] Yuxin Wu and Kaiming He. Group nor- malization. In ECCV, 2018.","title":"Group normalization","authors":"Yuxin Wu, Kaiming He","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2019] Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2. https://github.com/facebookresearch/detectron2, 2019.","title":"Detectron2","authors":"Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, Ross Girshick","year":"2019","venue":null,"doi":null,"url":"https://github.com/facebookresearch/detectron2","internal_url":null},{"position":27,"raw":"[Yang et al., 2019] Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. Reppoints: Point set represen- tation for object detection. In ICCV, 2019.","title":"Reppoints: Point set representation for object detection","authors":"Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, Stephen Lin","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yu and Koltun, 2015] Fisher Yu and Vladlen Koltun. Multi- scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, 2015.","title":"Multiscale context aggregation by dilated convolutions","authors":"Koltun ; Fisher Yu, Vladlen Yu, Koltun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2020] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li. Bridging the gap between anchor-based and anchor-free detection via adaptive train- ing sample selection. In CVPR, 2020.","title":"Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection","authors":"Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, Stan Z Li","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhu et al., 2020] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020.","title":"Deformable detr: Deformable transformers for end-to-end object detection","authors":"Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}