{"type":"ScholarlyArticle","title":"Multi-Margin based Decorrelation Learning for Heterogeneous Face Recognition","authors":[{"name":"Bing Cao","slug":"bing-cao","given":"Bing","family":"Cao","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Electronic Engineering , Xidian University , Xi'an , China"},{"name":"Nannan Wang","slug":"nannan-wang","given":"Nannan","family":"Wang","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Telecommunications Engineering , Xidian University , Xi'an , China"},{"name":"Xinbo Gao","slug":"xinbo-gao","given":"Xinbo","family":"Gao","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Electronic Engineering , Xidian University , Xi'an , China"},{"name":"Jie Li","slug":"jie-li","given":"Jie","family":"Li","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Electronic Engineering , Xidian University , Xi'an , China"},{"name":"Zhifeng Li","slug":"zhifeng-li","given":"Zhifeng","family":"Li","affiliation":"Tencent AI Lab , Shenzhen , China"}],"abstract":"Heterogeneous face recognition (HFR) refers to matching face images acquired from different domains with wide applications in security scenarios. However, HFR is still a challenging problem due to the significant cross-domain discrepancy and the lacking of sufficient training data in different domains. This paper presents a deep neural network approach namely Multi-Margin based Decorrelation Learning (MMDL) to extract decorrelation representations in a hyperspherical space for cross-domain face images. The proposed framework can be divided into two components: heterogeneous representation network and decorrelation representation learning. First, we employ a large scale of accessible visual face images to train heterogeneous representation network. The decorrelation layer projects the output of the first component into decorrelation latent subspace and obtain decorrelation representation. In addition, we design a multi-margin loss (MML), which consists of tetradmargin loss (TML) and heterogeneous angular margin loss (HAML), to constrain the proposed framework. Experimental results on two challenging heterogeneous face databases show that our approach achieves superior performance on both verification and recognition tasks, comparing with state-of-the-art methods.","doi":"10.24963/ijcai.2019/96","url":"https://ijcai.org/proceedings/2019/96","pdf":"https://ijcai.org/proceedings/2019/0096.pdf","markdown":"https://ijcai.org/proceedings/2019/96.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/96","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":96,"pages":{"first":680,"last":686},"keywords":["Computer Vision: Biometrics, Face and Gesture Recognition","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Cao et al., 2019] Bing Cao, Nannan Wang, Jie Li, and Xinbo Gao. Data augmentation-based joint learning for heterogeneous face recognition. IEEE Transactions on Neural Networks and Learning Systems, 30(6):1731- 1743, June 2019.","title":"Data augmentation-based joint learning for heterogeneous face recognition","authors":"Bing Cao, Nannan Wang, Jie Li, Xinbo Gao","year":"2019","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2009] Jie Chen, Dong Yi, Jimei Yang, Guoy- ing Zhao, Stan Z Li, and Matti Pietikainen. Learning mappings for face synthesis from near infrared to visual light images. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 156-163. IEEE, 2009.","title":"Learning mappings for face synthesis from near infrared to visual light images","authors":"Jie Chen, Dong Yi, Jimei Yang, Guoying Zhao, Stan Z Li, Matti Pietikainen","year":"2009","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017] Dong Chen, Xudong Cao, David Wipf, Fang Wen, and Jian Sun. An efficient joint formulation for bayesian face verification. IEEE Transactions on pattern analysis and machine intelligence, 39(1):32-46, 2017.","title":"An efficient joint formulation for bayesian face verification","authors":"Dong Chen, Xudong Cao, David Wipf, Fang Wen, Jian Sun","year":"2017","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2018] Jiankang Deng, Jia Guo, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. arXiv:1801.07698, 2018.","title":"Arcface: Additive angular margin loss for deep face recognition","authors":"Deng","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gong et al., 2017] Dihong Gong, Zhifeng Li, Weilin Huang, Xuelong Li, and Dacheng Tao. Heteroge- neous face recognition: A common encoding feature discriminant approach. IEEE Trans. Image Processing, 26(5):2079-2089, 2017.","title":"Heterogeneous face recognition: A common encoding feature discriminant approach","authors":"Gong","year":"2017","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guo et al., 2016] Yandong Guo, Lei Zhang, Yuxiao Hu, Xi- aodong He, and Jianfeng Gao. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In Eu- ropean Conference on Computer Vision, pages 87-102. Springer, 2016.","title":"Ms-celeb-1m: A dataset and benchmark for large-scale face recognition","authors":"Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, Jianfeng Gao","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017] Ran He, Xiang Wu, Zhenan Sun, and Tieniu Tan. Learning invariant deep representation for nir-vis face recognition. In AAAI, volume 4, page 7, 2017.","title":"Learning invariant deep representation for nir-vis face recognition","authors":"Ran He, Xiang Wu, Zhenan Sun, Tieniu Tan","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2018] Ran He, Xiang Wu, Zhenan Sun, and Tieniu Tan. Wasserstein cnn: Learning invariant features for nir- vis face recognition. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 2018.","title":"Wasserstein cnn: Learning invariant features for nirvis face recognition","authors":"Ran He, Xiang Wu, Zhenan Sun, Tieniu Tan","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2013] Xiangsheng Huang, Zhen Lei, Mingyu Fan, Xiao Wang, and Stan Z Li. Regularized dis- criminative spectral regression method for heterogeneous face matching. IEEE Transactions on Image Processing, 22(1):353-362, 2013.","title":"Regularized discriminative spectral regression method for heterogeneous face matching","authors":"Xiangsheng Huang, Zhen Lei, Mingyu Fan, Xiao Wang, Stan Z Li","year":"2013","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jin et al., 2015] Yi Jin, Jiwen Lu, and Qiuqi Ruan. Cou- pled discriminative feature learning for heterogeneous face recognition. IEEE Transactions on Information Forensics and Security, 10(3):640-652, 2015.","title":"Coupled discriminative feature learning for heterogeneous face recognition","authors":"Yi Jin, Jiwen Lu, Qiuqi Ruan","year":"2015","venue":"IEEE Transactions on Information Forensics and Security","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kan et al., 2016] Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, and Xilin Chen. Multi-view discrim- inant analysis. IEEE transactions on pattern analysis and machine intelligence, 38(1):188-194, 2016.","title":"Multi-view discriminant analysis","authors":"Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, Xilin Chen","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Klare and Jain, 2013] Brendan F Klare and Anil K Jain. Heterogeneous face recognition using kernel prototype similarities. IEEE transactions on pattern analysis and machine intelligence, 35(6):1410-1422, 2013.","title":"Heterogeneous face recognition using kernel prototype similarities","authors":"F Brendan, Anil K Klare, Jain","year":"2013","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lei and Li, 2009] Zhen Lei and Stan Z Li. Coupled spectral regression for matching heterogeneous faces. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 1123-1128. IEEE, 2009.","title":"Coupled spectral regression for matching heterogeneous faces","authors":"Li ; ] Zhen Lei, Stan Z Lei, Li","year":"2009","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lei et al., 2014] Zhen Lei, Matti Pietikäinen, and Stan Z Li. Learning discriminant face descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2):289- 302, 2014.","title":"Learning discriminant face descriptor","authors":"Zhen Lei, Matti Pietikäinen, Stan Z Li","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2013] Stan Li, Dong Yi, Zhen Lei, and Shengcai Liao. The casia nir-vis 2.0 face database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 348-353, 2013.","title":"The casia nir-vis 2.0 face database","authors":"Stan Li, Dong Yi, Zhen Lei, Shengcai Liao","year":"2013","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2016] Xiaoxiang Liu, Lingxiao Song, Xiang Wu, and Tieniu Tan. Transferring deep representation for nir- vis heterogeneous face recognition. In Biometrics (ICB), 2016 International Conference on, pages 1-8. IEEE, 2016.","title":"Transferring deep representation for nirvis heterogeneous face recognition","authors":"Xiaoxiang Liu, Lingxiao Song, Xiang Wu, Tieniu Tan","year":"2016","venue":"Biometrics (ICB), 2016 International Conference on","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2017] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, page 1, 2017.","title":"Sphereface: Deep hypersphere embedding for face recognition","authors":"Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, Le Song","year":"2017","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Parkhi et al., 2015] Omkar M Parkhi, Andrea Vedaldi, An- drew Zisserman, et al. Deep face recognition. In BMVC, volume 1, page 6, 2015.","title":"Deep face recognition","authors":"Andrea Omkar M Parkhi, Andrew Vedaldi, Zisserman","year":"2015","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pereyra et al., 2017] Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, and Geoffrey Hinton. Reg- ularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv:1701.06548, 2017.","title":"Regularizing neural networks by penalizing confident output distributions","authors":"Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, Geoffrey Hinton","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Saxena and Verbeek, 2016] Shreyas Saxena and Jakob Ver- beek. Heterogeneous face recognition with cnns. In Eu- ropean Conference on Computer Vision, pages 483-491. Springer, 2016.","title":"Shreyas Saxena and Jakob Verbeek. Heterogeneous face recognition with cnns","authors":"Verbeek Saxena","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shao and Fu, 2017] Ming Shao and Yun Fu. Cross-modality feature learning through generic hierarchical hyperlingual- words. IEEE transactions on neural networks and learn- ing systems, 28(2):451-463, 2017.","title":"Cross-modality feature learning through generic hierarchical hyperlingualwords","authors":"Ming Shao, Yun Fu","year":"2017","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Song et al., 2018] Lingxiao Song, Man Zhang, Xiang Wu, and Ran He. Adversarial discriminative heterogeneous face recognition. In AAAI Conference on Artificial Intelli- gence, pages 7355-7362, 2018.","title":"Adversarial discriminative heterogeneous face recognition","authors":"Song","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2013] Kaiye Wang, Ran He, Wei Wang, Liang Wang, and Tieniu Tan. Learning coupled feature spaces for cross-modal matching. In Proceedings of the IEEE In- ternational Conference on Computer Vision, pages 2088- 2095, 2013.","title":"Learning coupled feature spaces for cross-modal matching","authors":"Wang","year":"2013","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2017] Xiang Wu, Lingxiao Song, Ran He, and Tieniu Tan. Coupled deep learning for heterogeneous face recognition. arXiv preprint arXiv:1704.02450, 2017.","title":"Coupled deep learning for heterogeneous face recognition","authors":"Xiang Wu, Lingxiao Song, Ran He, Tieniu Tan","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2018] Xiang Wu, Huaibo Huang, Vishal M Pa- tel, Ran He, and Zhenan Sun. Disentangled variational representation for heterogeneous face recognition. arXiv preprint arXiv:1809.01936, 2018.","title":"Disentangled variational representation for heterogeneous face recognition","authors":"Xiang Wu, Huaibo Huang, M Vishal, Ran Patel, Zhenan He, Sun","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yi et al., 2015] Dong Yi, Zhen Lei, and Stan Z Li. Shared representation learning for heterogenous face recognition. In Automatic Face and Gesture Recognition (FG), 2015 11th IEEE International Conference and Workshops on, volume 1, pages 1-7. IEEE, 2015.","title":"Shared representation learning for heterogenous face recognition","authors":"Dong Yi, Zhen Lei, Stan Z Li","year":"2015","venue":"11th IEEE International Conference and Workshops on","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}