{"type":"ScholarlyArticle","title":"Corner Affinity: A Robust Grouping Algorithm to Make Corner-guided Detector Great Again","authors":[{"name":"Haoran Wei","slug":"haoran-wei","given":"Haoran","family":"Wei","affiliation":"University of Chinese Academy of Sciences"},{"name":"Chenglong Liu","slug":"chenglong-liu","given":"Chenglong","family":"Liu","affiliation":"University of Chinese Academy of Sciences"},{"name":"Ping Guo","slug":"ping-guo","given":"Ping","family":"Guo","affiliation":"Intel Labs China"},{"name":"Yangguang Zhu","slug":"yangguang-zhu","given":"Yangguang","family":"Zhu","affiliation":"University of Chinese Academy of Sciences"},{"name":"Jiamei Fu","slug":"jiamei-fu","given":"Jiamei","family":"Fu","affiliation":"University of Chinese Academy of Sciences"},{"name":"Bing Wang","slug":"bing-wang","given":"Bing","family":"Wang","affiliation":"University of Chinese Academy of Sciences"},{"name":"Peng Wang","slug":"peng-wang","given":"Peng","family":"Wang","affiliation":"Intel Labs China"}],"abstract":"Corner-guided detector enjoys potential ability to yield precise  bounding boxes. However, unreliable corner pairs, generated by heuristic grouping guidance, hinder the development of this detector. In this paper, we propose a novel corner grouping algorithm, termed as Corner Affinity, to significantly boost the reliability and robustness of corner grouping. The proposed Corner Affinity is a couple of two interactional factors, namely, 1) the structure affinity (SA), applying to generate preliminary corner pairs through the corresponding object's shallow construction information. 2) the contexts affinity (CA), running as optimizing corner pairs via embedding deeper semantic features of affiliated instances.  Equipped with the Corner Affinity, a detector can produce high-quality bounding boxes upon preferable paired corner keypoints. Experimental results show the superiority of our design on multiple benchmark datasets. Specifically, for CornerNet baseline, the proposed Corner Affinity brings AP boostings of 5.8% on COCO, 35.8% on Citypersons, and 17.2% on UCAS-AOD without bells and whistles.","doi":"10.24963/ijcai.2022/203","url":"https://ijcai.org/proceedings/2022/203","pdf":"https://ijcai.org/proceedings/2022/0203.pdf","markdown":"https://ijcai.org/proceedings/2022/203.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/203","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":203,"pages":{"first":1458,"last":1464},"keywords":["Computer Vision: Segmentation","Computer Vision: Recognition (object detection, categorization)"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986446","length_sec":38,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984589","length_sec":297,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bochkovskiy et al., 2020] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020.","title":"Yolov4: Optimal speed and accuracy of object detection","authors":"Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai and Vasconcelos, 2018] Zhaowei Cai and Nuno Vas- concelos. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 6154-6162, 2018.","title":"Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection","authors":"Vasconcelos Cai","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision, pages 213-229. Springer, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene under- standing. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 3213-3223, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2020] Zhiwei Dong, Guoxuan Li, Yue Liao, Fei Wang, Pengju Ren, and Chen Qian. Centripetal- net: Pursuing high-quality keypoint pairs for object de- tection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10519- 10528, 2020.","title":"Centripetalnet: Pursuing high-quality keypoint pairs for object detection","authors":"Zhiwei Dong, Guoxuan Li, Yue Liao, Fei Wang, Pengju Ren, Chen Qian","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Duan et al., 2019] Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 6569-6578, 2019.","title":"Centernet: Keypoint triplets for object detection","authors":"Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, Qi Tian","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duan et al., 2020] Kaiwen Duan, Lingxi Xie, Honggang Qi, Song Bai, Qingming Huang, and Qi Tian. Corner pro- posal network for anchor-free, two-stage object detection. In Computer Vision-ECCV 2020: 16th European Con- ference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 399-416. Springer, 2020.","title":"Corner proposal network for anchor-free, two-stage object detection","authors":"Kaiwen Duan, Lingxi Xie, Honggang Qi, Song Bai, Qingming Huang, Qi Tian","year":"2020","venue":"Computer Vision-ECCV 2020: 16th European Conference","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Law and Deng, 2018] Hei Law and Jia Deng. Cornernet: Detecting objects as paired keypoints. In Proceedings of the European Conference on Computer Vision (ECCV), pages 734-750, 2018.","title":"Cornernet: Detecting objects as paired keypoints","authors":"Deng ; ] Hei Law, Jia Law, Deng","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In European conference on com- puter vision, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin et al., 2017a] Tsung-Yi Lin, Piotr Dollár, Ross Gir- shick, Kaiming He, Bharath Hariharan, and Serge Be- longie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117-2125, 2017.","title":"Feature pyramid networks for object detection","authors":"Lin","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lin et al., 2017b] Tsung-Yi Lin, Priya Goyal, Ross Gir- shick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Lin","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2016] Wei Liu, Dragomir Anguelov, Dumitru Er- han, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference on computer vision, pages 21-37. Springer, 2016.","title":"Ssd: Single shot multibox detector","authors":"Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C Berg","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2021] Fanfan Liu, Haoran Wei, Wenzhe Zhao, Guozhen Li, Jingquan Peng, and Zihao Li. Wb-detr: Transformer-based detector without backbone. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 2979-2987, 2021.","title":"Wb-detr: Transformer-based detector without backbone","authors":"Fanfan Liu, Haoran Wei, Wenzhe Zhao, Guozhen Li, Jingquan Peng, Zihao Li","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Newell et al., 2017] Alejandro Newell, Zhiao Huang, and Jia Deng. Associative embedding: End-to-end learning for joint detection and grouping. In Advances in neural information processing systems, pages 2277-2287, 2017.","title":"Associative embedding: End-to-end learning for joint detection and grouping","authors":"Alejandro Newell, Zhiao Huang, Jia Deng","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Redmon and Farhadi, 2017] Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7263-7271, 2017.","title":"Yolo9000: better, faster, stronger","authors":"Joseph Redmon, Ali Farhadi","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. In Advances in neural information processing systems, pages 91-99, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2021] Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. Sparse r-cnn: End- to-end object detection with learnable proposals. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14454-14463, 2021.","title":"Sparse r-cnn: Endto-end object detection with learnable proposals","authors":"Sun","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tian et al., 2019] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object de- tection. In Proceedings of the IEEE international confer- ence on computer vision, pages 9627-9636, 2019.","title":"Fcos: Fully convolutional one-stage object detection","authors":"Chunhua Zhi Tian, Hao Shen, Tong Chen, He","year":"2019","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wei et al., 2020] Haoran Wei, Yue Zhang, Bing Wang, Yang Yang, Hao Li, and Hongqi Wang. X-linenet: Detecting air- craft in remote sensing images by a pair of intersecting line segments. IEEE Transactions on Geoscience and Remote Sensing, 59(2):1645-1659, 2020.","title":"X-linenet: Detecting aircraft in remote sensing images by a pair of intersecting line segments","authors":"Haoran Wei, Yue Zhang, Bing Wang, Yang Yang, Hao Li, Hongqi Wang","year":"2020","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2019] Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. Reppoints: Point set represen- tation for object detection. In Proceedings of the IEEE In- ternational Conference on Computer Vision, pages 9657- 9666, 2019.","title":"Reppoints: Point set representation for object detection","authors":"Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, Stephen Lin","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhou et al., 2019] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. Objects as points. arXiv preprint arXiv:1904.07850, 2019.","title":"Objects as points","authors":"Zhou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhu et al., 2015] Haigang Zhu, Xiaogang Chen, Weiqun Dai, Kun Fu, Qixiang Ye, and Jianbin Jiao. Orientation robust object detection in aerial images using deep convo- lutional neural network. In 2015 IEEE International Con- ference on Image Processing (ICIP), pages 3735-3739. IEEE, 2015.","title":"Orientation robust object detection in aerial images using deep convolutional neural network","authors":"Haigang Zhu, Xiaogang Chen, Weiqun Dai, Kun Fu, Qixiang Ye, Jianbin Jiao","year":"2015","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}