{"type":"ScholarlyArticle","title":"A Similarity Inference Metric for RGB-Infrared Cross-Modality Person Re-identification","authors":[{"name":"Mengxi Jia","slug":"mengxi-jia","given":"Mengxi","family":"Jia","affiliation":"School of Electronic and Computer Engineering , Peking University , China"},{"name":"Yunpeng Zhai","slug":"yunpeng-zhai","given":"Yunpeng","family":"Zhai","affiliation":"School of Electronic and Computer Engineering , Peking University , China"},{"name":"Shijian Lu","slug":"shijian-lu","given":"Shijian","family":"Lu","affiliation":"Nanyang Technological University , Singapore"},{"name":"Siwei Ma","slug":"siwei-ma","given":"Siwei","family":"Ma","affiliation":"School of Electronics Engineering and Computer Science , Peking University , China; Peng Cheng Laboratory , China"},{"name":"Jian Zhang","slug":"jian-zhang","given":"Jian","family":"Zhang","affiliation":"School of Electronic and Computer Engineering , Peking University , China; Peng Cheng Laboratory , China"}],"abstract":"RGB-Infrared (IR) cross-modality person re-identification (re-ID), which aims to search an IR image in RGB gallery or vice versa, is a challenging task due to the large discrepancy between IR and RGB modalities. Existing methods address this challenge typically by aligning feature distributions or image styles across modalities, whereas the very useful similarities among gallery samples of the same modality (i.e. intra-modality sample similarities) are largely neglected. This paper presents a novel similarity inference metric (SIM) that exploits the intra-modality sample similarities to circumvent the cross-modality discrepancy targeting optimal cross-modality image matching. SIM works by successive similarity graph reasoning and mutual nearest-neighbor reasoning that mine cross-modality sample similarities by leveraging intra-modality sample similarities from two different perspectives. Extensive experiments over two cross-modality re-ID datasets (SYSU-MM01 and RegDB) show that SIM achieves significant accuracy improvement but with little extra training as compared with the state-of-the-art.","doi":"10.24963/ijcai.2020/143","url":"https://ijcai.org/proceedings/2020/143","pdf":"https://ijcai.org/proceedings/2020/0143.pdf","markdown":"https://ijcai.org/proceedings/2020/143.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/143","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":143,"pages":{"first":1026,"last":1032},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25119/file_25119.m3u8","length_sec":293,"provider":"hls"}],"references":[{"position":1,"raw":"[Chen et al., 2017] Weihua Chen, Xiaotang Chen, Jianguo Zhang, and Kaiqi Huang. Beyond triplet loss: a deep quadruplet network for person re-identification. In CVPR, pages 403-412, 2017.","title":"Beyond triplet loss: a deep quadruplet network for person re-identification","authors":"Weihua Chen, Xiaotang Chen, Jianguo Zhang, Kaiqi Huang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dai et al., 2018] Pingyang Dai, Rongrong Ji, Haibin Wang, Qiong Wu, and Yuyu Huang. Cross-modality person re- identification with generative adversarial training. In IJ- CAI, pages 677-683, 2018.","title":"Cross-modality person reidentification with generative adversarial training","authors":"Pingyang Dai, Rongrong Ji, Haibin Wang, Qiong Wu, Yuyu Huang","year":"2018","venue":"IJ-CAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dalal and Triggs, 2005] Navneet Dalal and Bill Triggs. His- tograms of oriented gradients for human detection. In CVPR, 2005.","title":"Histograms of oriented gradients for human detection","authors":"Triggs Dalal, Navneet Dalal, Bill Triggs","year":"2005","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Garcia et al., 2015] Jorge Garcia, Niki Martinel, Christian Micheloni, and Alfredo Gardel. Person re-identification ranking optimisation by discriminant context information analysis. In ICCV, pages 1305-1313, 2015.","title":"Person re-identification ranking optimisation by discriminant context information analysis","authors":"Jorge Garcia, Niki Martinel, Christian Micheloni, Alfredo Gardel","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gou et al., 2014] Mengran Gou, Xiong Fei, Octavia Camps, and Mario Sznaier. Person re-identification using kernel- based metric learning methods. In ECCV, 2014.","title":"Person re-identification using kernelbased metric learning methods","authors":"Gou","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hao et al., 2019] Yi Hao, Nannan Wang, Jie Li, and Xinbo Gao. Hsme: hypersphere manifold embedding for visible thermal person re-identification. In AAAI, 2019.","title":"Hsme: hypersphere manifold embedding for visible thermal person re-identification","authors":"Yi Hao, Nannan Wang, Jie Li, Xinbo Gao","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hermans et al., 2017] Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017.","title":"In defense of the triplet loss for person re-identification","authors":"Alexander Hermans, Lucas Beyer, Bastian Leibe","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, pages 1097- 1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2018] Wei Li, Xiatian Zhu, and Shaogang Gong. Harmonious attention network for person re-identification. In CVPR, pages 2285-2294, 2018.","title":"Harmonious attention network for person re-identification","authors":"Wei Li, Xiatian Zhu, Shaogang Gong","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liao et al., 2015] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li. Person re-identification by local maximal occurrence representation and metric learning. In CVPR, pages 2197-2206, 2015.","title":"Person re-identification by local maximal occurrence representation and metric learning","authors":"Shengcai Liao, Yang Hu, Xiangyu Zhu, Stan Z Li","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2016] Liang Lin, Guangrun Wang, Wangmeng Zuo, Xiangchu Feng, and Lei Zhang. Cross-domain visual matching via generalized similarity measure and feature learning. TPAMI, 39(6):1089-1102, 2016.","title":"Cross-domain visual matching via generalized similarity measure and feature learning","authors":"Liang Lin, Guangrun Wang, Wangmeng Zuo, Xiangchu Feng, Lei Zhang","year":"2016","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nguyen et al., 2017] Dat Tien Nguyen, Hyung Gil Hong, Ki Wan Kim, and Kang Ryoung Park. Person recognition system based on a combination of body images from visi- ble light and thermal cameras. Sensors, 17(3):605, 2017.","title":"Person recognition system based on a combination of body images from visible light and thermal cameras","authors":"Hyung Dat Tien Nguyen, Gil Hong, Ki Wan Kim, Kang Ryoung Park","year":"2017","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wang et al., 2019a] Guan'an Wang, Tianzhu Zhang, Jian Cheng, Si Liu, Yang Yang, and Zengguang Hou. Rgb- infrared cross-modality person re-identification via joint pixel and feature alignment. In ICCV, October 2019.","title":"Rgbinfrared cross-modality person re-identification via joint pixel and feature alignment","authors":"Wang","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wang et al., 2019b] Zhixiang Wang, Zheng Wang, Yin- qiang Zheng, Yung-Yu Chuang, and Shin'ichi Satoh. Learning to reduce dual-level discrepancy for infrared- visible person re-identification. In CVPR, pages 618-626, 2019.","title":"Learning to reduce dual-level discrepancy for infraredvisible person re-identification","authors":"Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2020] Guan'an Wang, Tianzhu Zhang, Yang Yang, Jian Cheng, jianlong Chang, Xu Liang, and Zeng- guang Hou. Cross-modality paired-images generation for rgb-infrared person re-identification. In AAAI, February 2020.","title":"Cross-modality paired-images generation for rgb-infrared person re-identification","authors":"Tianzhu Guan'an Wang, Yang Zhang, Jian Yang, Cheng, Xu Chang, Zengguang Liang, Hou","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wu et al., 2017] Ancong Wu, Wei-Shi Zheng, Hong-Xing Yu, Shaogang Gong, and Jianhuang Lai. Rgb-infrared cross-modality person re-identification. In ICCV, pages 5380-5389, 2017.","title":"Rgb-infrared cross-modality person re-identification","authors":"Ancong Wu, Wei-Shi Zheng, Hong-Xing Yu, Shaogang Gong, Jianhuang Lai","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yang et al., 2019] Fan Yang, Ke Yan, Shijian Lu, Huizhu Jia, Xiaodong Xie, and Wen Gao. Attention driven person re-identification. Pattern Recognit., 86:143-155, 2019.","title":"Attention driven person re-identification","authors":"Fan Yang, Ke Yan, Shijian Lu, Huizhu Jia, Xiaodong Xie, Wen Gao","year":"2019","venue":"Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ye et al., 2016] Mang Ye, Chao Liang, Yi Yu, Zheng Wang, Qingming Leng, Chunxia Xiao, Jun Chen, and Ruimin Hu. Person reidentification via ranking aggregation of similar- ity pulling and dissimilarity pushing. IEEE Transactions on Multimedia, 18(12):2553-2566, 2016.","title":"Person reidentification via ranking aggregation of similarity pulling and dissimilarity pushing","authors":"Mang Ye, Chao Liang, Yi Yu, Zheng Wang, Qingming Leng, Chunxia Xiao, Jun Chen, Ruimin Hu","year":"2016","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ye et al., 2018a] Mang Ye, Xiangyuan Lan, Jiawei Li, and Pong C Yuen. Hierarchical discriminative learning for vis- ible thermal person re-identification. In AAAI, 2018.","title":"Hierarchical discriminative learning for visible thermal person re-identification","authors":"Ye","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ye et al., 2018b] Mang Ye, Zheng Wang, Xiangyuan Lan, and Pong C. Yuen. Visible thermal person re-identification via dual-constrained top-ranking. In IJCAI, 2018.","title":"Visible thermal person re-identification via dual-constrained top-ranking","authors":"Ye","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/152"},{"position":22,"raw":"[Ye et al., 2020] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. Deep learning for person re-identification: A survey and outlook. arXiv preprint arXiv:2001.04193, 2020.","title":"Deep learning for person re-identification: A survey and outlook","authors":"Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, Steven Ch Hoi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhai et al., 2020] Yunpeng Zhai, Shijian Lu, Qixiang Ye, Xuebo Shan, Jie Chen, Rongrong Ji, and Yonghong Tian. Ad-cluster: Augmented discriminative clustering for do- main adaptive person re-identification. In IEEE CVPR, 2020.","title":"Ad-cluster: Augmented discriminative clustering for domain adaptive person re-identification","authors":"Yunpeng Zhai, Shijian Lu, Qixiang Ye, Xuebo Shan, Jie Chen, Rongrong Ji, Yonghong Tian","year":"2020","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2017] Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, and Xiaoou Tang. Spindle net: Person re-identification with human body region guided feature decomposition and fu- sion. In CVPR, pages 1077-1085, 2017.","title":"Spindle net: Person re-identification with human body region guided feature decomposition and fusion","authors":"Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, Xiaoou Tang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhen et al., 2013] Li Zhen, Shiyu Chang, Liang Feng, Thomas S. Huang, and John R. Smith. Learning locally- adaptive decision functions for person verification. In CVPR, 2013.","title":"Learning locallyadaptive decision functions for person verification","authors":"Li Zhen, Shiyu Chang, Liang Feng, Thomas S Huang, John R Smith","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zheng et al., 2011] Wei Shi Zheng, Shaogang Gong, and Xiang Tao. Person re-identification by probabilistic rel- ative distance comparison. In CVPR, 2011.","title":"Person re-identification by probabilistic relative distance comparison","authors":"Wei Shi, Zheng, Shaogang Gong, Xiang Tao","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhong et al., 2017a] Zhun Zhong, Liang Zheng, Donglin Cao, and Shaozi Li. Re-ranking person re-identification with k-reciprocal encoding. In CVPR, July 2017.","title":"Re-ranking person re-identification with k-reciprocal encoding","authors":"Zhong","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhong et al., 2017b] Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data aug- mentation. arXiv preprint arXiv:1708.04896, 2017.","title":"Random erasing data augmentation","authors":"Zhong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}