{"type":"ScholarlyArticle","title":"Adversarial Attribute-Image Person Re-identification","authors":[{"name":"Zhou Yin","slug":"zhou-yin","given":"Zhou","family":"Yin","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China"},{"name":"Wei-Shi Zheng","slug":"wei-shi-zheng","given":"Wei-Shi","family":"Zheng","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China; Key Laboratory of Machine Intelligence and Advanced Computing , Ministry of Education , China"},{"name":"Ancong Wu","slug":"ancong-wu","given":"Ancong","family":"Wu","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China"},{"name":"Hong-Xing Yu","slug":"hong-xing-yu","given":"Hong-Xing","family":"Yu","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China"},{"name":"Hai Wan","slug":"hai-wan","given":"Hai","family":"Wan","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China"},{"name":"Xiaowei Guo","slug":"xiaowei-guo","given":"Xiaowei","family":"Guo","affiliation":"YouTu Lab , Tencent"},{"name":"Feiyue Huang","slug":"feiyue-huang","given":"Feiyue","family":"Huang","affiliation":"YouTu Lab , Tencent"},{"name":"Jianhuang Lai","slug":"jianhuang-lai","given":"Jianhuang","family":"Lai","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China"}],"abstract":"While attributes have been widely used for person re-identification (Re-ID) which aims at matching the same person images across disjoint camera views, they are used either as extra features or for performing multi-task learning to assist the image-image\r\nmatching task. However, how to find a set of person images according to a given attribute description, which is very practical in many surveillance applications, remains a rarely investigated cross-modality matching problem in person Re-ID. In this work, we present this challenge and leverage adversarial learning to formulate the attribute-image cross-modality person Re-ID model. By imposing a semantic consistency constraint across modalities as a regularization, the adversarial learning enables to generate image-analogous concepts of query attributes for matching the corresponding images at both global level and semantic ID level. We conducted extensive experiments on three attribute datasets and demonstrated that the regularized adversarial modelling is so far the most effective method for the attribute-image cross-modality person Re-ID problem.","doi":"10.24963/ijcai.2018/153","url":"https://ijcai.org/proceedings/2018/153","pdf":"https://ijcai.org/proceedings/2018/0153.pdf","markdown":"https://ijcai.org/proceedings/2018/153.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/153","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":153,"pages":{"first":1100,"last":1106},"keywords":["Computer Vision: Biometrics, Face and Gesture Recognition","Computer Vision: Video: Events, Activities and Surveillance"],"videos":[],"references":[{"position":1,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Karen Livescu, and Jeff Bilmes. Deep canonical correlation anal- ysis. In ICML, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Karen Livescu, Jeff Bilmes","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Deng et al., 2014] Y. Deng, P. Luo, C. Loy, and X. Tang. Pedestrian attribute recognition at far distance. In ACMMM, 2014.","title":"Pedestrian attribute recognition at far distance","authors":"Y Deng, P Luo, C Loy, X Tang","year":"2014","venue":"ACMMM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Eisenschtat and Wolf, 2017] A. Eisenschtat and L. Wolf. Linking image and text with 2-way nets. In CVPR, 2017.","title":"Linking image and text with 2-way nets","authors":"A Eisenschtat, L Wolf","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Eric et al., 2017] Tzeng Eric, Hoffman Judy, Saenko Kate, and Darrell Trevor. Adversarial discriminative domain adaptation. In CVPR, 2017.","title":"Adversarial discriminative domain adaptation","authors":"Tzeng Eric, Hoffman Judy, Kate Saenko, Darrell Trevor","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Feng et al., 2014] Fangxiang Feng, Xiaojie Wang, and Ruifan Li. Cross-modal retrieval with correspondence au- toencoder. In ACMMM, 2014.","title":"Cross-modal retrieval with correspondence autoencoder","authors":"Feng","year":"2014","venue":"ACMMM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ganin and Lempitsky, 2015] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backprop- agation. In ICML, 2015.","title":"Unsupervised domain adaptation by backpropagation","authors":"Yaroslav Ganin, Victor Lempitsky","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NIPS, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hotelling, 1936] Harold Hotelling. Relations between two sets of variates. Biometrika, 1936.","title":"Relations between two sets of variates","authors":"Harold Hotelling","year":"1936","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Lei Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kiros et al., 2014] R. Kiros, R. Salakhutdinov, and R.S. Zemel. Unifying visual-semantic embeddings with mul- timodal neural language models. In arXiv, 2014.","title":"Unifying visual-semantic embeddings with multimodal neural language models","authors":"R Kiros, R Salakhutdinov, R S Zemel","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Layne et al., 2012a] Ryan Layne, Tim Hospedales, and Shaogang Gong. Person re-identification by attributes. In BMVC, 2012.","title":"Person re-identification by attributes","authors":"Layne","year":"2012","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Layne et al., 2012b] Ryan Layne, Timothy M. Hospedales, and Shaogang Gong. Towards person identification and re-identification with attributes. In ECCV, 2012.","title":"Towards person identification and re-identification with attributes","authors":"Layne","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Layne et al., 2014a] Ryan Layne, Tim Hospedales, and Shaogang Gong. Re-id: Hunting attributes in the wild. In BMVC, 2014.","title":"Re-id: Hunting attributes in the wild","authors":"Layne","year":"2014","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Layne et al., 2014b] Ryan Layne, Timothy M. Hospedales, and Shaogang Gong. Attributes-Based Re-identification. 2014.","title":"Attributes-Based Re-identification","authors":"Layne","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2015] D. Li, X. Chen, and K. Huang. Multi- attribute learning for pedestrian attribute recognition in surveillance scenarios. In ACPR, 2015.","title":"Multiattribute learning for pedestrian attribute recognition in surveillance scenarios","authors":"D Li, X Chen, K Huang","year":"2015","venue":"ACPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2017] Shuang Li, Tong Xiao, Hongsheng Li, Wei Yang, and Xiaogang Wang. Identity-aware textual-visual matching with latent co-attention. In ICCV, 2017.","title":"Identity-aware textual-visual matching with latent co-attention","authors":"Shuang Li, Tong Xiao, Hongsheng Li, Wei Yang, Xiaogang Wang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2017] Yutian Lin, Liang Zheng, and Wu Yu and Yang Yi Zheng, Zhedong and. Improving person re- identification by attribute and identity learning. In arXiv, 2017.","title":"Improving person reidentification by attribute and identity learning","authors":"Yutian Lin, Liang Zheng, Wu Yu, Yang Yi Zheng, Zhedong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Long and Wang, 2015] Mingsheng Long and Jianmin Wang. Learning transferable features with deep adapta- tion networks. 02 2015.","title":"Learning transferable features with deep adaptation networks","authors":"Wang ; Long, Mingsheng Long, Jianmin Wang","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mineiro and Karampatziakis, 2014] Paul Mineiro and Nikos Karampatziakis. A randomized algorithm for cca. In CoRR, 2014.","title":"A randomized algorithm for cca","authors":"Karampatziakis Mineiro, Paul Mineiro, Nikos Karampatziakis","year":"2014","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ngiam et al., 2011] Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y. Ng. Mul- timodal deep learning. In ICML, 2011.","title":"Multimodal deep learning","authors":"Ngiam","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Reed et al., 2016] S. Reed, Z. Akata, X. Yan, L. Lo- geswaran, B. Schiele, and H. Lee. Generative adversarial text to image synthesis. In ICML, 2016.","title":"Generative adversarial text to image synthesis","authors":"S Reed, Z Akata, X Yan, L Logeswaran, B Schiele, H Lee","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Scheirer et al., 2012] W. Scheirer, N. Kumar, P. Belhumeur, and T. Boult. Multi-attribute spaces: Calibration for at- tribute fusion and similarity search. In CVPR, 2012.","title":"Multi-attribute spaces: Calibration for attribute fusion and similarity search","authors":"W Scheirer, N Kumar, P Belhumeur, T Boult","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Siddiquie et al., 2011] B. Siddiquie, R. S. Feris, and L. Davis. Image ranking and retrieval based on multi- attribute queries. In CVPR, 2011.","title":"Image ranking and retrieval based on multiattribute queries","authors":"B Siddiquie, R S Feris, L Davis","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[S.Li et al., 2017] S.Li, T.Xiao, H.Li, and X.Wang et al. Per- son search with natural language description. In CVPR, 2017.","title":"Person search with natural language description","authors":"S Li","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Su et al., 2015a] C. Su, F. Yang, S. Zhang, Q. Tian, L. S. Davis, and W. Gao. Multi-task learning with low rank at- tribute embedding for person re-identification. In ICCV, 2015.","title":"Multi-task learning with low rank attribute embedding for person re-identification","authors":"Su","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Su et al., 2015b] Chi Su, Shiliang Zhang, Fan Yang, Guangxiao Zhang, Qi Tian, Wen Gao, and Larry S. Davis. Tracklet-to-tracklet person re-identification by attributes with discriminative latent space mapping. In ICMS, 2015.","title":"Tracklet-to-tracklet person re-identification by attributes with discriminative latent space mapping","authors":"Su","year":"2015","venue":"ICMS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Su et al., 2016] Chi Su, Shiliang Zhang, Junliang Xing, Wen Gao, and Qi Tian. Deep attributes driven multi- camera person re-identification. In ECCV, 2016.","title":"Deep attributes driven multicamera person re-identification","authors":"Su","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Su et al., 2018] Chi Su, Shiliang Zhang, Junliang Xing, Wen Gao, and Qi Tian. Multi-type attributes driven multi- camera person re-identification. In PR, 2018.","title":"Multi-type attributes driven multicamera person re-identification","authors":"Su","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sun and Saenko, 2016] Baochen Sun and Kate Saenko. Deep coral: correlation alignment for deep domain adap- tation. In ICCV, 2016.","title":"Deep coral: correlation alignment for deep domain adaptation","authors":"Saenko ; Sun, Baochen Sun, Kate Saenko","year":"2016","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tzeng et al., 2014] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. CoRR, 2014.","title":"Deep domain confusion: Maximizing for domain invariance","authors":"Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, Trevor Darrell","year":"2014","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Vaquero et al., 2009] D. A. Vaquero, R. S. Feris, D. Tran, L. Brown, A. Hampapur, and M. Turk. Attribute-based people search in surveillance environments. In WACV, 2009.","title":"Attribute-based people search in surveillance environments","authors":"D A Vaquero, R S Feris, D Tran, L Brown, A Hampapur, M Turk","year":"2009","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2015] Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-view represen- tation learning. In ICML, 2015.","title":"On deep multi-view representation learning","authors":"Wang","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wei et al., 2017] Y. Wei, Y. Zhao, C. Lu, S. Wei, L. Liu, Z. Zhu, and S. Yan. Cross-modal retrieval with cnn visual features: A new baseline. IEEE T. CYBERN, 2017.","title":"Cross-modal retrieval with cnn visual features: A new baseline","authors":"Y Wei, Y Zhao, C Lu, S Wei, L Liu, Z Zhu, S Yan","year":"2017","venue":"IEEE T. CYBERN","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2016] Y. Yang, Z. Lei, S. Zhang, H. Shi, and S. Z. Li. Metric embedded discriminative vocabulary learning for high-level person representation. In AAAI, 2016.","title":"Metric embedded discriminative vocabulary learning for high-level person representation","authors":"Y Yang, Z Lei, S Zhang, H Shi, S Z Li","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhao et al., 2014] R. Zhao, W. Ouyang, and X. Wang. Learning mid-level filters for person re-identification. In CVPR, 2014.","title":"Learning mid-level filters for person re-identification","authors":"R Zhao, W Ouyang, X Wang","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}