Proceedings

Multi-Granularity Graph-Convolution-Based Method for Weakly Supervised Person Search

Download BibTeX Cite

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. Main Track. Pages 1326-1334.

https://doi.org/10.24963/ijcai.2024/147

Abstract

One-step Weakly Supervised Person Search (WSPS) jointly performs pedestrian detection and person Re-IDentification (ReID) only with bounding box annotations, which makes the traditional person ReID problem more suitable and efficient for real-world applications. However, this task is very challenging due to the following reasons: 1) large feature gap between person ReID and general object detection tasks when learning shared representations; 2) difficult pseudo identity estimation for each person image with unrefined raw detection and dramatic scale changes. To address above issues, we propose a multi-granularity graph convolution framework to jointly optimize the aligned task features, as well as to assist the pseudo label estimation. Specifically, the multi-granularity feature alignment module (MFA) in the designed two-branch framework, employs cluster-level bi-directional interaction of various granularity information to narrow down the large feature gap. Further, upon the MFA module, we introduce the multi-granularity graph-convolution-based pseudo-label estimation module, to enhance feature representations for distinguishing diverse identities. Extensive experimental results demonstrate the effectiveness of the proposed method, and show superior performances to state-of-the art methods by a large margin on CUHK-SYSU and PRW datasets.

  1. [Cao et al., 2022] Jiale Cao, Yanwei Pang, Rao Muhammad Anwer, Hisham Cholakkal, Jin Xie, Mubarak Shah, and Fahad Shahbaz Khan. Pstr: End-to-end one-step person search with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9458-9467, 2022.
  2. [Chen et al., 2018] Di Chen, Shanshan Zhang, Wanli Ouyang, Jian Yang, and Ying Tai. Person search via a mask-guided two-stream cnn model. In Proceedings of the european conference on computer vision (ECCV), pages 734-750, 2018.
  3. [Chen et al., 2019] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155, 2019.
  4. [Chen et al., 2020] Di Chen, Shanshan Zhang, Jian Yang, and Bernt Schiele. Norm-aware embedding for efficient person search. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 12615-12624, 2020.
  5. [Dong et al., 2020a] Wenkai Dong, Zhaoxiang Zhang, Chunfeng Song, and Tieniu Tan. Bi-directional interac- tion network for person search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2839-2848, 2020.
  6. [Dong et al., 2020b] Wenkai Dong, Zhaoxiang Zhang, Chunfeng Song, and Tieniu Tan. Instance guided pro- posal network for person search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2585-2594, 2020.
  7. [Feng et al., 2023] Cheng Feng, Dezhi Han, and Chongqing Chen. Dthn: Dual-transformer head end-to-end person search network. Computers, Materials & Continua, 77(1), 2023.
  8. [Han et al., 2019] Chuchu Han, Jiacheng Ye, Yunshan Zhong, Xin Tan, Chi Zhang, Changxin Gao, and Nong Sang. Re-id driven localization refinement for person search. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9814-9823, 2019.
  9. [Han et al., 2021a] Byeong-Ju Han, Kuhyeun Ko, and Jae- Young Sim. End-to-end trainable trident person search network using adaptive gradient propagation. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 925-933, 2021.
  10. [Han et al., 2021b] Chuchu Han, Kai Su, Dongdong Yu, Zehuan Yuan, Changxin Gao, Nong Sang, Yi Yang, and Changhu Wang. Weakly supervised person search with region siamese networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12006-12015, 2021.
  11. [Hansun, 2013] Seng Hansun. A new approach of moving average method in time series analysis. In 2013 confer- ence on new media studies (CoNMedia), pages 1-4. IEEE, 2013.
  12. [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
  13. [Jia et al., 2022] Chengyou Jia, Minnan Luo, Caixia Yan, Xiaojun Chang, and Qinghua Zheng. Cgua: Context- guided and unpaired-assisted weakly supervised person search. arXiv preprint arXiv:2203.14307, 2022.
  14. [Ke et al., 2022] Xiao Ke, Hao Liu, Wenzhong Guo, Baitao Chen, Yuhang Cai, and Weibin Chen. Joint sample en- hancement and instance-sensitive feature learning for ef- ficient person search. IEEE Transactions on Circuits and Systems for Video Technology, 32(11):7924-7937, 2022.
  15. [Kim et al., 2021] Hanjae Kim, Sunghun Joung, Ig-Jae Kim, and Kwanghoon Sohn. Prototype-guided saliency fea- ture learning for person search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4865-4874, 2021.
  16. [Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  17. [Lan et al., 2018] Xu Lan, Xiatian Zhu, and Shaogang Gong. Person search by multi-scale matching. In Proceedings of the European conference on computer vision (ECCV), pages 536-552, 2018.
  18. [Lee et al., 2022] Sanghoon Lee, Youngmin Oh, Donghyeon Baek, Junghyup Lee, and Bumsub Ham. Oimnet++: Prototypical normalization and localization-aware learn- ing for person search. In European Conference on Com- puter Vision, pages 621-637. Springer, 2022.
  19. [Li and Miao, 2021] Zhengjia Li and Duoqian Miao. Se- quential end-to-end network for efficient person search. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pages 2011-2019, 2021.
  20. [Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748, 2018.
  21. [Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. Advances in neural information processing systems, 28, 2015.
  22. [Tang et al., 2024] Zongheng Tang, Yulu Gao, Tianrui Hui, Fengguang Peng, and Si Liu. Mi3c: Mining intra-and inter-image context for person search. Pattern Recogni- tion, 148:110169, 2024.
  23. [Wang et al., 2020] Cheng Wang, Bingpeng Ma, Hong Chang, Shiguang Shan, and Xilin Chen. Tcts: A task- consistent two-stage framework for person search. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11952-11961, 2020.
  24. [Wang et al., 2023] Benzhi Wang, Yang Yang, Jinlin Wu, Guo-jun Qi, and Zhen Lei. Self-similarity driven scale- invariant learning for weakly supervised person search. arXiv preprint arXiv:2302.12986, 2023.
  25. [Wang et al., 2024] Jiabei Wang, Yanwei Pang, Jiale Cao, Hanqing Sun, Zhuang Shao, and Xuelong Li. Deep intra- image contrastive learning for weakly supervised one-step person search. Pattern Recognition, 147:110047, 2024.
  26. [Xiao et al., 2017] Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, and Xiaogang Wang. Joint detection and identi- fication feature learning for person search. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3415-3424, 2017.
  27. [Xin et al., 2023] Xiaoqi Xin, Dezhi Han, and Mingming Cui. Daaps: A deformable-attention-based anchor-free person search model. Computers, Materials & Continua, 77(2), 2023.
  28. [Yan et al., 2019] Yichao Yan, Qiang Zhang, Bingbing Ni, Wendong Zhang, Minghao Xu, and Xiaokang Yang. Learning context graph for person search. In Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, pages 2158-2167, 2019.
  29. [Yan et al., 2021] Yichao Yan, Jinpeng Li, Jie Qin, Song Bai, Shengcai Liao, Li Liu, Fan Zhu, and Ling Shao. Anchor- free person search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7690-7699, 2021.
  30. [Yan et al., 2022a] Tianyi Yan, Kuan Zhu, Guibo Zhu, Ming Tang, Jinqiao Wang, et al. Plug-and-play pseudo label cor- rection network for unsupervised person re-identification. arXiv preprint arXiv:2206.06607, 2022.
  31. [Yan et al., 2022b] Yichao Yan, Jinpeng Li, Shengcai Liao, Jie Qin, Bingbing Ni, Ke Lu, and Xiaokang Yang. Explor- ing visual context for weakly supervised person search. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 36, pages 3027-3035, 2022.
  32. [Yang et al., 2021] Wenjie Yang, Houjing Huang, Xiaotang Chen, and Kaiqi Huang. Bottom-up foreground-aware fea- ture fusion for practical person search. IEEE Transactions on Circuits and Systems for Video Technology, 32(1):262- 274, 2021.
  33. [Yu et al., 2022] Rui Yu, Dawei Du, Rodney LaLonde, Daniel Davila, Christopher Funk, Anthony Hoogs, and Brian Clipp. Cascade transformers for end-to-end person search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7267- 7276, 2022.
  34. [Zhang et al., 2020] Yaobin Zhang, Weihong Deng, Mei Wang, Jiani Hu, Xian Li, Dongyue Zhao, and Dongchao Wen. Global-local gcn: Large-scale label noise cleans- ing for face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7731-7740, 2020.
  35. [Zhang et al., 2023a] Shizhou Zhang, Qingchun Yang, De Cheng, Yinghui Xing, Guoqiang Liang, Peng Wang, and Yanning Zhang. Ground-to-aerial person search: Benchmark dataset and approach. In Proceedings of the 31st ACM International Conference on Multimedia, pages 789-799, 2023.
  36. [Zhang et al., 2023b] Yuqi Zhang, Qi Qian, Hongsong Wang, Chong Liu, Weihua Chen, and Fan Wan. Graph convolution based efficient re-ranking for visual retrieval. IEEE Transactions on Multimedia, 2023.
  37. [Zhang et al., 2024] Pengcheng Zhang, Xiaohan Yu, Xiao Bai, Chen Wang, Jin Zheng, and Xin Ning. Joint dis- criminative representation learning for end-to-end person search. Pattern Recognition, 147:110053, 2024.
  38. [Zheng et al., 2017] Liang Zheng, Hengheng Zhang, Shaoyan Sun, Manmohan Chandraker, Yi Yang, and Qi Tian. Person re-identification in the wild. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pages 1367-1376, 2017.
  39. [Zhong et al., 2019] Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H Li, and Ge Li. Graph con- volutional label noise cleaner: Train a plug-and-play ac- tion classifier for anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1237-1246, 2019.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards