{"type":"ScholarlyArticle","title":"Deep Attribute Guided Representation for Heterogeneous Face Recognition","authors":[{"name":"Decheng Liu","slug":"decheng-liu","given":"Decheng","family":"Liu","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Electronic Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Nannan Wang","slug":"nannan-wang","given":"Nannan","family":"Wang","affiliation":"‡ State Key Laboratory of Integrated Services Networks , School of Telecommunications Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Chunlei Peng","slug":"chunlei-peng","given":"Chunlei","family":"Peng","affiliation":"§ School of Cyber Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Jie Li","slug":"jie-li","given":"Jie","family":"Li","affiliation":""},{"name":"Xinbo Gao","slug":"xinbo-gao","given":"Xinbo","family":"Gao","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Electronic Engineering , Xidian University , Xi'an 710071 , China"}],"abstract":"Heterogeneous face recognition (HFR) is a challenging problem in face recognition, subject to large texture and spatial structure differences of face images. Different from conventional face recognition in homogeneous environments, there exist many face images taken from different sources (including different sensors or different mechanisms) in reality. Motivated by human cognitive mechanism, we naturally utilize the explicit invariant semantic information (face attributes) to help address the gap of different modalities. Existing related face recognition methods mostly regard attributes as the high level feature integrated with other engineering features enhancing recognition performance, ignoring the inherent relationship between face attributes and identities. In this paper, we propose a novel deep attribute guided representation based heterogeneous face recognition method (DAG-HFR) without labeling attributes manually. Deep convolutional networks are employed to directly map face images in heterogeneous scenarios to a compact common space where distances mean similarities of pairs. An attribute guided triplet loss (AGTL) is designed to train an end-to-end HFR network which could effectively eliminate defects of incorrectly detected attributes. Extensive experiments on multiple heterogeneous scenarios (composite sketches, resident ID cards) demonstrate that the proposed method achieves superior performances compared with state-of-the-art methods.","doi":"10.24963/ijcai.2018/116","url":"https://ijcai.org/proceedings/2018/116","pdf":"https://ijcai.org/proceedings/2018/0116.pdf","markdown":"https://ijcai.org/proceedings/2018/116.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/116","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":116,"pages":{"first":835,"last":841},"keywords":["Computer Vision: Computer Vision","Computer Vision: Biometrics, Face and Gesture Recognition"],"videos":[],"references":[{"position":1,"raw":"[Belhumeur et al., 1997] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. fisherfaces: recog- nition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell., 19(7):711-720, 1997.","title":"Eigenfaces vs. fisherfaces: recognition using class specific linear projection","authors":"P N Belhumeur, J P Hespanha, D J Kriegman","year":"1997","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bhatt et al., 2012] Himanshu S Bhatt, Samarth Bharadwaj, Richa Singh, and Mayank Vatsa. Memetically optimized mcwld for matching sketches with digital face images. IEEE Transactions on Information Forensics and Security, 7(5):1522-1535, 2012.","title":"Memetically optimized mcwld for matching sketches with digital face images","authors":"S Himanshu, Samarth Bhatt, Richa Bharadwaj, Mayank Singh, Vatsa","year":"2012","venue":"IEEE Transactions on Information Forensics and Security","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Duchi et al., 2011] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul):2121-2159, 2011.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"John Duchi, Elad Hazan, Yoram Singer","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Han et al., 2017] Hu Han, Anil K Jain, Shiguang Shan, and Xilin Chen. Heterogeneous face attribute estimation: A deep multi-task learning approach. IEEE transactions on pattern analysis and machine intelligence, 2017.","title":"Heterogeneous face attribute estimation: A deep multi-task learning approach","authors":"Hu Han, K Anil, Shiguang Jain, Xilin Shan, Chen","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2017] Guosheng Hu, Yang Hua, Yang Yuan, Zhi- hong Zhang, Zheng Lu, Sankha S Mukherjee, Timoth- y M Hospedales, Neil M Robertson, and Yongxin Yang. Attribute-enhanced face recognition with neural tensor fu- sion networks. In Proc. Int. Conf. Comput. Vis.(ICCV), 2017.","title":"Attribute-enhanced face recognition with neural tensor fusion networks","authors":"Guosheng Hu, Yang Hua, Yang Yuan, Zhihong Zhang, Zheng Lu, Timothy M Sankha S Mukherjee, Neil M Hospedales, Yongxin Robertson, Yang","year":"2017","venue":"Proc. Int. Conf. Comput. Vis.(ICCV)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huo et al., 2017] Jing Huo, Yang Gao, Yinghuan Shi, Wan- qi Yang, and Hujun Yin. Heterogeneous face recognition by margin-based cross-modality metric learning. IEEE transactions on cybernetics, 2017.","title":"Heterogeneous face recognition by margin-based cross-modality metric learning","authors":"Jing Huo, Yang Gao, Yinghuan Shi, Wanqi Yang, Hujun Yin","year":"2017","venue":"IEEE transactions on cybernetics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kan et al., 2016] Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, and Xilin Chen. Multi-view discrim- inant analysis. IEEE transactions on pattern analysis and machine intelligence, 38(1):188-194, 2016.","title":"Multi-view discriminant analysis","authors":"Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, Xilin Chen","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Klare et al., Mar 2011]","title":null,"authors":"Klare","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"B. Klare, Z. Li, and A. Jain. Match- ing forensic sketches to mug shot photos. IEEE Trans. Pattern Anal. Mach. Intell., 33(3):639-646, Mar. 2011.","title":"Matching forensic sketches to mug shot photos","authors":"B Klare, Z Li, A Jain","year":"2011","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kumar et al., 2011] Neeraj Kumar, Alexander Berg, Pe- ter N Belhumeur, and Shree Nayar. Describable visual attributes for face verification and image search. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 33(10):1962-1977, 2011.","title":"Describable visual attributes for face verification and image search","authors":"Neeraj Kumar, Alexander Berg, Peter N Belhumeur, Shree Nayar","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2015] Yan Li, Ruiping Wang, Haomiao Liu, Hua- jie Jiang, Shiguang Shan, and Xilin Chen. Two birds, one stone: Jointly learning binary code for large-scale face im- age retrieval and attributes prediction. In Proceedings of the IEEE International Conference on Computer Vision, pages 3819-3827, 2015.","title":"Two birds, one stone: Jointly learning binary code for large-scale face image retrieval and attributes prediction","authors":"Yan Li, Ruiping Wang, Haomiao Liu, Huajie Jiang, Shiguang Shan, Xilin Chen","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin and Tang, 2006] Dahua Lin and Xiaoou Tang. Inter- modality face recognition. In European Conference on Computer Vision, pages 13-26. Springer, 2006.","title":"Intermodality face recognition","authors":"Lin, Tang, Dahua Lin, Xiaoou Tang","year":"2006","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2005] Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, and Songde Ma. A nonlinear approach for face sketch synthesis and recognition. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, pages 1005-1010 vol. 1, 2005.","title":"A nonlinear approach for face sketch synthesis and recognition","authors":"Liu","year":"2005","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pages 3730-3738, 2015.","title":"Deep learning face attributes in the wild","authors":"Ziwei Liu, Ping Luo, Xiaogang Wang, Xiaoou Tang","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lu et al., 2017] Jiwen Lu, Venice Erin Liong, and Jie Zhou. Simultaneous local binary feature learning and encod- ing for homogeneous and heterogeneous face recognition. IEEE transactions on pattern analysis and machine intel- ligence, 2017.","title":"Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition","authors":"Jiwen Lu, Venice Erin Liong, Jie Zhou","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mittal et al., 2014] P. Mittal, A. Jain, G. Goswami, R. S- ingh, and M. Vatsa. Recognizing composite sketches with digital face images via ssd dictionary. In IEEE Interna- tional Joint Conference on Biometrics, pages 1-6, 2014.","title":"Recognizing composite sketches with digital face images via ssd dictionary","authors":"P Mittal, A Jain, G Goswami, R Singh, M Vatsa","year":"2014","venue":"IEEE International Joint Conference on Biometrics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mittal et al., 2015] P. Mittal, M. Vatsa, and R Singh. Com- posite sketch recognition via deep network. In Proc. Int. Conf. Biom, pages 1091-1097, 2015.","title":"Composite sketch recognition via deep network","authors":"P Mittal, M Vatsa, Singh","year":"2015","venue":"Proc. Int. Conf. Biom","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mittal et al., 2017] Paritosh Mittal, Aishwarya Jain, Gaurav Goswami, Mayank Vatsa, and Richa Singh. Composite sketch recognition using saliency and attribute feedback. Information Fusion, 33:86-99, 2017.","title":"Composite sketch recognition using saliency and attribute feedback","authors":"Paritosh Mittal, Aishwarya Jain, Gaurav Goswami, Mayank Vatsa, Richa Singh","year":"2017","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ouyang et al., 2014] Shuxin Ouyang, Timothy Hospedales, Yi-Zhe Song, and Xueming Li. Cross-modal face match- ing: beyond viewed sketches. In Asian Conference on Computer Vision, pages 210-225. Springer, 2014.","title":"Cross-modal face matching: beyond viewed sketches","authors":"Ouyang","year":"2014","venue":"Asian Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ouyang et al., 2016] Shuxin Ouyang, Timothy M Hospedales, Yi-Zhe Song, and Xueming Li. Forget- menot: memory-aware forensic facial sketch matching. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., pages 5571-5579, 2016.","title":"Forgetmenot: memory-aware forensic facial sketch matching","authors":"Ouyang","year":"2016","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recog","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Peng et al., 2017] Chunlei Peng, Xinbo Gao, Nannan Wang, and Jie Li. Graphical representation for heterogeneous face recognition. IEEE Trans. Pattern Anal. Mach. Intell., 39(2):301-312, 2017.","title":"Graphical representation for heterogeneous face recognition","authors":"Chunlei Peng, Xinbo Gao, Nannan Wang, Jie Li","year":"2017","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Saxena and Verbeek, 2016] Shreyas Saxena and Jakob Ver- beek. Heterogeneous face recognition with cnns. In Eu- ropean Conference on Computer Vision, pages 483-491. Springer, 2016.","title":"Shreyas Saxena and Jakob Verbeek. Heterogeneous face recognition with cnns","authors":"Verbeek Saxena","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tang and Wang, 2003] Xiaoou Tang and Xiaogang Wang. Face sketch synthesis and recognition. In Computer vision, 2003. proceedings. ninth ieee international conference on, pages 687-694. IEEE, 2003.","title":"Face sketch synthesis and recognition","authors":"Wang ; Tang, Xiaoou Tang, Xiaogang Wang","year":"2003","venue":"ninth ieee international conference on","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang and Tang, 2009] Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell., 31(11):1955-1967, 2009.","title":"Face photo-sketch synthesis and recognition","authors":"Wang, Tang, Xiaogang Wang, Xiaoou Tang","year":"2009","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}