{"type":"ScholarlyArticle","title":"Visible Thermal Person Re-Identification via Dual-Constrained Top-Ranking","authors":[{"name":"Mang Ye","slug":"mang-ye","given":"Mang","family":"Ye","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong"},{"name":"Zheng Wang","slug":"zheng-wang","given":"Zheng","family":"Wang","affiliation":"National Institute of Informatics , Japan"},{"name":"Xiangyuan Lan","slug":"xiangyuan-lan","given":"Xiangyuan","family":"Lan","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong"},{"name":"Pong C. Yuen","slug":"pong-c-yuen","given":"Pong C.","family":"Yuen","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong"}],"abstract":"Cross-modality person re-identification between the thermal and visible domains is extremely important for night-time surveillance applications. Existing works in this filed mainly focus on learning sharable feature representations to handle the cross-modality discrepancies. However, besides the cross-modality discrepancy caused by different camera spectrums, visible thermal person re-identification also suffers from large cross-modality and intra-modality variations caused by different camera views and human poses. In this paper, we propose a dual-path network with a novel bi-directional dual-constrained top-ranking loss to learn discriminative feature representations. It is advantageous in two aspects: 1) end-to-end feature learning directly from the data without extra metric learning steps, 2) it simultaneously handles the cross-modality and intra-modality variations to ensure the discriminability of the learnt representations. Meanwhile, identity loss is further incorporated to model the identity-specific information to handle large intra-class variations. Extensive experiments on two datasets demonstrate the superior performance compared to the state-of-the-arts.","doi":"10.24963/ijcai.2018/152","url":"https://ijcai.org/proceedings/2018/152","pdf":"https://ijcai.org/proceedings/2018/0152.pdf","markdown":"https://ijcai.org/proceedings/2018/152.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/152","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":152,"pages":{"first":1092,"last":1099},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Video: Events, Activities and Surveillance"],"videos":[],"references":[{"position":1,"raw":"[Barbosa et al., 2012] Igor Barros Barbosa, Marco Cristani, Alessio Del Bue, Loris Bazzani, and Vittorio Murino. Re- identification with rgb-d sensors. In ECCVW, pages 433- 442, 2012.","title":"Reidentification with rgb-d sensors","authors":"Igor Barros Barbosa, Marco Cristani, Alessio Del Bue, Loris Bazzani, Vittorio Murino","year":"2012","venue":"ECCVW","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2017] Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. Collective deep quantization for efficient cross-modal retrieval. In AAAI, pages 3974-3980, 2017.","title":"Collective deep quantization for efficient cross-modal retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Shichen Liu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Ying-Cong Chen, Xiatian Zhu, Wei-Shi Zheng, and Jian-Huang Lai. Person re-identification by camera correlation aware feature augmentation. IEEE T- PAMI, 40(2):392-408, 2018.","title":"Person re-identification by camera correlation aware feature augmentation","authors":"Ying-Cong Chen, Xiatian Zhu, Wei-Shi Zheng, Jian-Huang Lai","year":"2018","venue":"IEEE T-PAMI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2017] Ran He, Xiang Wu, Zhenan Sun, and Tieniu Tan. Learning invariant deep representation for nir-vis face recognition. In AAAI, pages 2000-2006, 2017.","title":"Learning invariant deep representation for nir-vis face recognition","authors":"Ran He, Xiang Wu, Zhenan Sun, Tieniu Tan","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hermans et al., 2017] Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017.","title":"In defense of the triplet loss for person re-identification","authors":"Alexander Hermans, Lucas Beyer, Bastian Leibe","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang and Frank Wang, 2013] De-An Huang and Yu- Chiang Frank Wang. Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In ICCV, pages 2496-2503, 2013.","title":"Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition","authors":"Frank Huang, An Wang ; De-, Yu-Chiang Huang, Wang Frank","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jiang et al., 2017] Junjun Jiang, Chen Chen, Jiayi Ma, Zheng Wang, Zhongyuan Wang, and Ruimin Hu. Srlsp: A face image super-resolution algorithm using smooth re- gression with local structure prior. IEEE Transactions on Multimedia, 19(1):27-40, 2017.","title":"Srlsp: A face image super-resolution algorithm using smooth regression with local structure prior","authors":"Junjun Jiang, Chen Chen, Jiayi Ma, Zheng Wang, Zhongyuan Wang, Ruimin Hu","year":"2017","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1097- 1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lan et al., 2015] Xiangyuan Lan, Andy J Ma, Pong C Yuen, and Rama Chellappa. Joint sparse representation and robust feature-level fusion for multi-cue visual tracking. IEEE TIP, 24(12):5826-5841, 2015.","title":"Joint sparse representation and robust feature-level fusion for multi-cue visual tracking","authors":"Xiangyuan Lan, Andy J Ma, Pong C Yuen, Rama Chellappa","year":"2015","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lan et al., 2018] Xiangyuan Lan, Shengping Zhang, Pong C Yuen, and Rama Chellappa. Learning common and feature-specific patterns: a novel multiple-sparse- representation-based tracker. IEEE Transactions on Image Processing, 27(4):2022-2037, 2018.","title":"Learning common and feature-specific patterns: a novel multiple-sparserepresentation-based tracker","authors":"Xiangyuan Lan, Shengping Zhang, Pong C Yuen, Rama Chellappa","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2017a] Shuang Li, Tong Xiao, and et al. Person search with natural language description. In CVPR, pages 1345-1353, 2017.","title":"Person search with natural language description","authors":"Li","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2017b] Shuang Li, Tong Xiao, Hongsheng Li, Wei Yang, and Xiaogang Wang. Identity-aware textual-visual matching with latent co-attention. In ICCV, pages 1890- 1899, 2017.","title":"Identity-aware textual-visual matching with latent co-attention","authors":"Li","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liao and Li, 2015] Shengcai Liao and Stan Z Li. Efficient psd constrained asymmetric metric learning for person re- identification. In ICCV, pages 3685-3693, 2015.","title":"Efficient psd constrained asymmetric metric learning for person reidentification","authors":"Li ; Liao, Shengcai Liao, Stan Z Li","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liao et al., 2015] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li. Person re-identification by local maximal occurrence representation and metric learning. In CVPR, pages 2197-2206, 2015.","title":"Person re-identification by local maximal occurrence representation and metric learning","authors":"Shengcai Liao, Yang Hu, Xiangyu Zhu, Stan Z Li","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2017a] Liang Lin, Guangrun Wang, Wangmeng Zuo, Xiangchu Feng, and Lei Zhang. Cross-domain visual matching via generalized similarity measure and feature learning. IEEE TPAMI, 39(6):1089-1102, 2017.","title":"Cross-domain visual matching via generalized similarity measure and feature learning","authors":"Lin","year":"2017","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2017b] Yutian Lin, Liang Zheng, Zhedong Zheng, Yu Wu, and Yi Yang. Improving person re- identification by attribute and identity learning. arXiv preprint arXiv:1703.07220, 2017.","title":"Improving person reidentification by attribute and identity learning","authors":"Lin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liong et al., 2017] Venice Erin Liong, Jiwen Lu, Yap-Peng Tan, and Jie Zhou. Cross-modal deep variational hashing. In ICCV, pages 4077-4085, 2017.","title":"Cross-modal deep variational hashing","authors":"Liong","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2017] Yu Liu, Yanming Guo, Erwin M Bakker, and Michael S Lew. Learning a recurrent residual fusion network for multimodal matching. In ICCV, pages 4107- 4116, 2017.","title":"Learning a recurrent residual fusion network for multimodal matching","authors":"Yu Liu, Yanming Guo, Erwin M Bakker, Michael S Lew","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mogelmose et al., 2013] Andreas Mogelmose, Chris Bahnsen, Thomas Moeslund, Albert Clapés, and Sergio Escalera. Tri-modal person re-identification with rgb, depth and thermal features. In CVPRW, pages 301-307, 2013.","title":"Tri-modal person re-identification with rgb, depth and thermal features","authors":"Andreas Mogelmose, Chris Bahnsen, Thomas Moeslund, Albert Clapés, Sergio Escalera","year":"2013","venue":"CVPRW","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nguyen et al., 2017] Dat Tien Nguyen, Hyung Gil Hong, Ki Wan Kim, and Kang Ryoung Park. Person recognition system based on a combination of body images from visi- ble light and thermal cameras. Sensors, 17(3):605, 2017.","title":"Person recognition system based on a combination of body images from visible light and thermal cameras","authors":"Hyung Dat Tien Nguyen, Gil Hong, Ki Wan Kim, Kang Ryoung Park","year":"2017","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sarfraz and Stiefelhagen, 2017] M Saquib Sarfraz and Rainer Stiefelhagen. Deep perceptual mapping for cross-modal face recognition. IJCV, 122(3):426-438, 2017.","title":"Deep perceptual mapping for cross-modal face recognition","authors":"M Saquib, Sarfraz, Rainer Stiefelhagen","year":"2017","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Song et al., 2018] Yibing Song, Chao Ma, Xiaohe Wu, Li- jun Gong, Linchao Bao, Wangmeng Zuo, Chunhua Shen, Rynson Lau, and Ming-Hsuan Yang. Vital: Visual track- ing via adversarial learning. In CVPR, 2018.","title":"Vital: Visual tracking via adversarial learning","authors":"Yibing Song, Chao Ma, Xiaohe Wu, Lijun Gong, Linchao Bao, Wangmeng Zuo, Chunhua Shen, Rynson Lau, Ming-Hsuan Yang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2012] S. Wang, L. Zhang, Liang Y., and Q. Pan. Semi-coupled dictionary learning with applications in im- age super-resolution and photo-sketch synthesis. In CVPR, pages 2216-2223, 2012.","title":"Semi-coupled dictionary learning with applications in image super-resolution and photo-sketch synthesis","authors":"S Wang, L Zhang, Y Liang, Q Pan","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2016a] Liwei Wang, Yin Li, and Svetlana Lazebnik. Learning deep structure-preserving image-text embeddings. In CVPR, pages 5005-5013, 2016.","title":"Learning deep structure-preserving image-text embeddings","authors":"Wang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2016b] Zheng Wang, Ruimin Hu, Chao Liang, Yi Yu, Junjun Jiang, Mang Ye, Jun Chen, and Qing- ming Leng. Zero-shot person re-identification via cross- view consistency. IEEE Transactions on Multimedia, 18(2):260-272, 2016.","title":"Zero-shot person re-identification via crossview consistency","authors":"Wang","year":"2016","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2017] Zheng Wang, Ruimin Hu, Chen Chen, Yi Yu, Junjun Jiang, Chao Liang, and Shin'ichi Satoh. Person reidentification via discrepancy matrix and matrix metric. IEEE Transactions on Cybernetics, 2017.","title":"Person reidentification via discrepancy matrix and matrix metric","authors":"Zheng Wang, Ruimin Hu, Chen Chen, Yi Yu, Junjun Jiang, Chao Liang, Shin'ichi Satoh","year":"2017","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wu et al., 2017a] Ancong Wu, Wei-Shi Zheng, and Jian- Huang Lai. Robust depth-based person re-identification. IEEE TIP, 26(6):2588-2603, 2017.","title":"Robust depth-based person re-identification","authors":"Wu","year":"2017","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wu et al., 2017b] Ancong Wu, Wei-shi Zheng, Hong-Xing Yu, Shaogang Gong, and Jianghuang Lai. Rgb-infrared cross-modality person re-identification. In ICCV, pages 5380-5389, 2017.","title":"Rgb-infrared cross-modality person re-identification","authors":"Wu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wu et al., 2018a] Lin Wu, Yang Wang, Junbin Gao, and X- ue Li. Deep adaptive feature embedding with local sample distributions for person re-identification. Pattern Recogni- tion (PR), 73:275-288, 2018.","title":"Deep adaptive feature embedding with local sample distributions for person re-identification","authors":"Wu","year":"2018","venue":"Pattern Recognition (PR)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wu et al., 2018b] Lin Wu, Yang Wang, Xue Li, and Junbin Gao. What-and-where to match: Deep spatially multi- plicative integration networks for person re-identification. Pattern Recognition (PR), 76:727-738, 2018.","title":"What-and-where to match: Deep spatially multiplicative integration networks for person re-identification","authors":"Wu","year":"2018","venue":"Pattern Recognition (PR)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2018c] Xiang Wu, Lingxiao Song, Ran He, and Tieniu Tan. Coupled deep learning for heterogeneous face recognition. In AAAI, 2018.","title":"Coupled deep learning for heterogeneous face recognition","authors":"Wu","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xiao et al., 2017] Qiqi Xiao, Hao Luo, and Chi Zhang. Margin sample mining loss: A deep learning based method for person re-identification. arXiv preprint arX- iv:1710.00478, 2017.","title":"Margin sample mining loss: A deep learning based method for person re-identification","authors":"Qiqi Xiao, Hao Luo, Chi Zhang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ye et al., 2015] Mang Ye, Chao Liang, Zheng Wang, Qing- ming Leng, Jun Chen, and Jun Liu. Specific person re- trieval via incomplete text description. In ICMR, pages 547-550, 2015.","title":"Specific person retrieval via incomplete text description","authors":"Mang Ye, Chao Liang, Zheng Wang, Qingming Leng, Jun Chen, Jun Liu","year":"2015","venue":"ICMR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Ye et al., 2016] Mang Ye, Chao Liang, Yi Yu, Zheng Wang, Qingming Leng, Chunxia Xiao, Jun Chen, and Ruimin Hu. Person reidentification via ranking aggregation of similar- ity pulling and dissimilarity pushing. IEEE Transactions on Multimedia, 18(12):2553-2566, 2016.","title":"Person reidentification via ranking aggregation of similarity pulling and dissimilarity pushing","authors":"Mang Ye, Chao Liang, Yi Yu, Zheng Wang, Qingming Leng, Chunxia Xiao, Jun Chen, Ruimin Hu","year":"2016","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Ye et al., 2017] Mang Ye, Andy J Ma, Liang Zheng, Jiawei Li, and Pong C. Yuen. Dynamic label graph matching for unsupervised video re-identification. In ICCV, pages 5142-5150, 2017.","title":"Dynamic label graph matching for unsupervised video re-identification","authors":"Mang Ye, Andy J Ma, Liang Zheng, Jiawei Li, Pong C Yuen","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ye et al., 2018] Mang Ye, Xiangyuan Lan, Jiawei Li, and Pong C. Yuen. Hierarchical discriminative learning for visible thermal person re-identification. In AAAI, 2018.","title":"Hierarchical discriminative learning for visible thermal person re-identification","authors":"Mang Ye, Xiangyuan Lan, Jiawei Li, Pong C Yuen","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zheng et al., 2016] Liang Zheng, Yi Yang, and Alexander G Hauptmann. Person re-identification: Past, present and future. arXiv preprint arXiv:1610.02984, 2016.","title":"Person re-identification: Past, present and future","authors":"Liang Zheng, Yi Yang, Alexander G Hauptmann","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zheng et al., 2017] Liang Zheng, Yi Yang, and Qi Tian. SIFT meets CNN: A decade survey of instance retrieval. IEEE TPAMI, 2017.","title":"SIFT meets CNN: A decade survey of instance retrieval","authors":"Liang Zheng, Yi Yang, Qi Tian","year":"2017","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhou et al., 2018] Joey Tianyi Zhou, Heng Zhao, Xi Peng, Meng Fang, Zheng Qin, Zheng Qin, and Rick Siow Mong Goh. Transfer hashing: From shallow to deep. IEEE Transaction on Neural Network and Learning Systems, 2018.","title":"Transfer hashing: From shallow to deep","authors":"Joey Tianyi Zhou, Heng Zhao, Xi Peng, Meng Fang, Zheng Qin, Zheng Qin, Rick Siow, Mong Goh","year":"2018","venue":"IEEE Transaction on Neural Network and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhu et al., 2018] Xiatian Zhu, Botong Wu, Dongcheng Huang, and Wei-Shi Zheng. Fast open-world person re- identification. IEEE Transactions on Image Processing (TIP), 27(5):2286 -2300, 2018.","title":"Fast open-world person reidentification","authors":"Xiatian Zhu, Botong Wu, Dongcheng Huang, Wei-Shi Zheng","year":"2018","venue":"IEEE Transactions on Image Processing (TIP)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}