{"type":"ScholarlyArticle","title":"Learning Spatial Similarity Distribution for Few-shot Object Counting","authors":[{"name":"Yuanwu Xu","slug":"yuanwu-xu","given":"Yuanwu","family":"Xu","affiliation":"School of Artificial Intelligence , Nanjing University of Science and Technology"},{"name":"Feifan Song","slug":"feifan-song","given":"Feifan","family":"Song","affiliation":"School of Artificial Intelligence , Nanjing University of Science and Technology"},{"name":"Haofeng Zhang","slug":"haofeng-zhang","given":"Haofeng","family":"Zhang","affiliation":"School of Artificial Intelligence , Nanjing University of Science and Technology"}],"abstract":"Few-shot object counting aims to count the number of objects in a query image that belong to the same class as the given exemplar images. Existing methods compute the similarity between the query image and exemplars in the 2D spatial domain and perform regression to obtain the counting number. However, these methods overlook the rich information about the spatial distribution of similarity on the exemplar images, leading to significant impact on matching accuracy. To address this issue, we propose a network learning Spatial Similarity Distribution (SSD) for few-shot object counting, which preserves the spatial structure of exemplar features and calculates a 4D similarity pyramid point-to-point between the query features and exemplar features, capturing the complete distribution information for each point in the 4D similarity space. We propose a Similarity Learning Module (SLM) which applies the efficient center-pivot 4D convolutions on the similarity pyramid to map different similarity distributions to distinct predicted density values, thereby obtaining accurate count. Furthermore, we also introduce a Feature Cross Enhancement (FCE) module that enhances query and exemplar features mutually to improve the accuracy of feature matching. Our approach outperforms state-of-the-art methods on multiple datasets, including FSC-147 and CARPK. Code is available at https://github.com/CBalance/SSD.","doi":"10.24963/ijcai.2024/167","url":"https://ijcai.org/proceedings/2024/167","pdf":"https://ijcai.org/proceedings/2024/0167.pdf","markdown":"https://ijcai.org/proceedings/2024/167.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/167","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":167,"pages":{"first":1507,"last":1515},"keywords":["Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning","Computer Vision: CV: Recognition (object detection, categorization)"],"videos":[],"references":[{"position":1,"raw":"[Abousamra et al., 2021] Shahira Abousamra, Minh Hoai, Dimitris Samaras, and Chao Chen. Localization in the crowd with topological constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 872-881, 2021.","title":"Localization in the crowd with topological constraints","authors":"Shahira Abousamra, Minh Hoai, Dimitris Samaras, Chao Chen","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arteta et al., 2016] Carlos Arteta, Victor Lempitsky, and Andrew Zisserman. Counting in the wild. In Proceedings of the European Conference on Computer Vision, pages 483-498, 2016.","title":"Counting in the wild","authors":"Arteta","year":"2016","venue":"Proceedings of the European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2022] Zhi-Qi Cheng, Qi Dai, Hong Li, Jingkuan Song, Xiao Wu, and Alexander G Hauptmann. Rethinking spatial invariance of convolutional networks for object counting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19638-19648, 2022.","title":"Rethinking spatial invariance of convolutional networks for object counting","authors":"Zhi-Qi Cheng, Qi Dai, Hong Li, Jingkuan Song, Xiao Wu, Alexander G Hauptmann","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Du et al., 2023] Zhipeng Du, Jiankang Deng, and Miaojing Shi. Domain-general crowd counting in unseen scenarios. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 37, pages 561-570, 2023.","title":"Domain-general crowd counting in unseen scenarios","authors":"Zhipeng Du, Jiankang Deng, Miaojing Shi","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In Proceedings of the Interna- tional Conference on Machine Learning, pages 1126- 1135, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"Proceedings of the International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dol- lár, and Ross Girshick. Mask r-cnn. In 2017 IEEE Inter- national Conference on Computer Vision (ICCV), pages 2980-2988, 2017.","title":"Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn","authors":"He","year":"2017","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hsieh et al., 2017] Meng-Ru Hsieh, Yen-Liang Lin, and Winston H Hsu. Drone-based object counting by spatially regularized regional proposal network. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 4145-4153, 2017.","title":"Drone-based object counting by spatially regularized regional proposal network","authors":"Meng-Ru Hsieh, Yen-Liang Lin, Winston H Hsu","year":"2017","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liang et al., 2023] Dingkang Liang, Jiahao Xie, Zhikang Zou, Xiaoqing Ye, Wei Xu, and Xiang Bai. Crowdclip: Unsupervised crowd counting via vision-language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2893-2903, 2023.","title":"Crowdclip: Unsupervised crowd counting via vision-language model","authors":"Dingkang Liang, Jiahao Xie, Zhikang Zou, Xiaoqing Ye, Wei Xu, Xiang Bai","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin and Chan, 2023] Wei Lin and Antoni B Chan. Opti- mal transport minimization: Crowd localization on den- sity maps for semi-supervised counting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 21663-21673, 2023.","title":"Optimal transport minimization: Crowd localization on density maps for semi-supervised counting","authors":"Wei Lin, Antoni B Chan","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lin et al., 2021] Hui Lin, Xiaopeng Hong, and Yabin Wang. Object counting: You only need to look at one. arXiv preprint arXiv:2112.05993, 2021.","title":"Object counting: You only need to look at one","authors":"Lin","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2022] Wei Lin, Kunlin Yang, Xinzhu Ma, Junyu Gao, Lingbo Liu, Shinan Liu, Jun Hou, Shuai Yi, and Antoni Chan. Scale-prior deformable convolution for exemplar-guided class-agnostic counting. In Proceedings of the British Machine Vision Conference, 2022.","title":"Scale-prior deformable convolution for exemplar-guided class-agnostic counting","authors":"Wei Lin, Kunlin Yang, Xinzhu Ma, Junyu Gao, Lingbo Liu, Shinan Liu, Jun Hou, Shuai Yi, Antoni Chan","year":"2022","venue":"Proceedings of the British Machine Vision Conference","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2022] Chang Liu, Yujie Zhong, Andrew Zisser- man, and Weidi Xie. Countr: Transformer-based gener- alised visual counting. In Proceedings of the British Ma- chine Vision Conference, 2022.","title":"Countr: Transformer-based generalised visual counting","authors":"Chang Liu, Yujie Zhong, Andrew Zisserman, Weidi Xie","year":"2022","venue":"Proceedings of the British Machine Vision Conference","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Loshchilov and Hutter, 2017] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.","title":"Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization","authors":"Hutter Loshchilov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lu et al., 2019] Erika Lu, Weidi Xie, and Andrew Zisser- man. Class-agnostic counting. In Proceedings of the Asian Conference on Computer Vision, pages 669-684, 2019.","title":"Class-agnostic counting","authors":"Erika Lu, Weidi Xie, Andrew Zisserman","year":"2019","venue":"Proceedings of the Asian Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma et al., 2019] Zhiheng Ma, Xing Wei, Xiaopeng Hong, and Yihong Gong. Bayesian loss for crowd count es- timation with point supervision. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6142-6151, 2019.","title":"Bayesian loss for crowd count estimation with point supervision","authors":"Zhiheng Ma, Xing Wei, Xiaopeng Hong, Yihong Gong","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Min et al., 2021] Juhong Min, Dahyun Kang, and Minsu Cho. Hypercorrelation squeeze for few-shot segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6941-6952, 2021.","title":"Hypercorrelation squeeze for few-shot segmentation","authors":"Min","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ranjan et al., 2021] Viresh Ranjan, Udbhav Sharma, Thu Nguyen, and Minh Hoai. Learning to count everything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3394-3403, 2021.","title":"Learning to count everything","authors":"Udbhav Viresh Ranjan, Thu Sharma, Minh Nguyen, Hoai","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rocco et al., 2018] Ignacio Rocco, Mircea Cimpoi, Relja Arandjelović, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Neighbourhood consensus networks. Advances in neural information processing systems, 31, 2018.","title":"Neighbourhood consensus networks","authors":"Ignacio Rocco, Mircea Cimpoi, Relja Arandjelović, Akihiko Torii, Tomas Pajdla, Josef Sivic","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi et al., 2022] Min Shi, Hao Lu, Chen Feng, Chengxin Liu, and Zhiguo Cao. Represent, compare, and learn: A similarity-aware framework for class-agnostic counting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9529-9538, 2022.","title":"Represent, compare, and learn: A similarity-aware framework for class-agnostic counting","authors":"Shi","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shu et al., 2022] Weibo Shu, Jia Wan, Kay Chen Tan, Sam Kwong, and Antoni B Chan. Crowd counting in the fre- quency domain. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 19618-19627, 2022.","title":"Crowd counting in the frequency domain","authors":"Weibo Shu, Jia Wan, Kay Chen Tan, Sam Kwong, Antoni B Chan","year":"2022","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Stewart et al., 2016] Russell Stewart, Mykhaylo Andriluka, and Andrew Y Ng. End-to-end people detection in crowded scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2325- 2333, 2016.","title":"End-to-end people detection in crowded scenes","authors":"Russell Stewart, Mykhaylo Andriluka, Andrew Y Ng","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ðukić et al., 2023] Nikola Ðukić, Alan Lukežič, Vitjan Za- vrtanik, and Matej Kristan. A low-shot object counting network with iterative prototype adaptation. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 18872-18881, 2023.","title":"A low-shot object counting network with iterative prototype adaptation","authors":"Nikola Ðukić, Alan Lukežič, Vitjan Zavrtanik, Matej Kristan","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wan et al., 2021] Jia Wan, Ziquan Liu, and Antoni B Chan. A generalized loss function for crowd counting and local- ization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1974- 1983, 2021.","title":"A generalized loss function for crowd counting and localization","authors":"Jia Wan, Ziquan Liu, Antoni B Chan","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2020] Qi Wang, Junyu Gao, Wei Lin, and Xue- long Li. Nwpu-crowd: A large-scale benchmark for crowd counting and localization. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 43(6):2141-2149, 2020.","title":"Nwpu-crowd: A large-scale benchmark for crowd counting and localization","authors":"Qi Wang, Junyu Gao, Wei Lin, Xuelong Li","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu and He, 2018] Yuxin Wu and Kaiming He. Group nor- malization. In Proceedings of the European conference on computer vision (ECCV), pages 3-19, 2018.","title":"Group normalization","authors":"He Wu, Yuxin Wu, Kaiming He","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang and Ramanan, 2019] Gengshan Yang and Deva Ra- manan. Volumetric correspondence networks for optical flow. Advances in neural information processing systems, 32, 2019.","title":"Volumetric correspondence networks for optical flow","authors":"Ramanan Yang","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2021] Shuo-Diao Yang, Hung-Ting Su, Win- ston H Hsu, and Wen-Chin Chen. Class-agnostic few-shot object counting. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 870-878, 2021.","title":"Class-agnostic few-shot object counting","authors":"Shuo-Diao Yang, Hung-Ting Su, Winston H Hsu, Wen-Chin Chen","year":"2021","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[You et al., 2023] Zhiyuan You, Kai Yang, Wenhan Luo, Xin Lu, Lei Cui, and Xinyi Le. Few-shot object counting with similarity-aware feature enhancement. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 6315-6324, 2023.","title":"Few-shot object counting with similarity-aware feature enhancement","authors":"Zhiyuan You, Kai Yang, Wenhan Luo, Xin Lu, Lei Cui, Xinyi Le","year":"2023","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}