{"type":"ScholarlyArticle","title":"Cross-Domain Facial Expression Recognition via Disentangling Identity Representation","authors":[{"name":"Tong Liu","slug":"tong-liu","given":"Tong","family":"Liu","affiliation":"Wuhan University , China"},{"name":"Jing Li","slug":"jing-li","given":"Jing","family":"Li","affiliation":"Wuhan University , China"},{"name":"Jia Wu","slug":"jia-wu","given":"Jia","family":"Wu","affiliation":"Macquarie University , Sydney"},{"name":"Lefei Zhang","slug":"lefei-zhang","given":"Lefei","family":"Zhang","affiliation":"Wuhan University , China"},{"name":"Shanshan Zhao","slug":"shanshan-zhao","given":"Shanshan","family":"Zhao","affiliation":"JD Explore Academy , China"},{"name":"Jun Chang","slug":"jun-chang","given":"Jun","family":"Chang","affiliation":"Wuhan University , China"},{"name":"Jun Wan","slug":"jun-wan","given":"Jun","family":"Wan","affiliation":"Zhongnan University of Economics and Law , China"}],"abstract":"Most existing cross-domain facial expression recognition (FER) works require target domain data to assist the model in analyzing distribution shifts to overcome negative effects. However, it is often hard to obtain expression images of the target domain in practical applications. Moreover, existing methods suffer from the interference of identity information, thus limiting the discriminative ability of the expression features. We exploit the idea of domain generalization (DG) and propose a representation disentanglement model to address the above problems. Specifically, we learn three independent potential subspaces corresponding to the domain, expression, and identity information from facial images. Meanwhile, the extracted expression and identity features are recovered as Fourier phase information reconstructed images, thereby ensuring that the high-level semantics of images remain unchanged after disentangling the domain information. Our proposed method can disentangle expression features from expression-irrelevant ones (i.e., identity and domain features). Therefore, the learned expression features exhibit sufficient domain invariance and discriminative ability. We conduct experiments with different settings on multiple benchmark datasets, and the results show that our method achieves superior performance compared with state-of-the-art methods.","doi":"10.24963/ijcai.2023/135","url":"https://ijcai.org/proceedings/2023/135","pdf":"https://ijcai.org/proceedings/2023/0135.pdf","markdown":"https://ijcai.org/proceedings/2023/135.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/135","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":135,"pages":{"first":1213,"last":1221},"keywords":["Computer Vision: CV: Adversarial learning, adversarial attack and defense methods","Computer Vision: CV: Representation learning","Computer Vision: CV: Recognition (object detection, categorization)"],"videos":[],"references":[{"position":1,"raw":"[Carlucci et al., 2019] Fabio M Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Ta- tiana Tommasi. Domain generalization by solving jigsaw puzzles. In CVPR, pages 2229-2238, 2019.","title":"Domain generalization by solving jigsaw puzzles","authors":"Fabio M Carlucci, D' Antonio, Silvia Innocente, Barbara Bucci, Tatiana Caputo, Tommasi","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chattopadhyay et al., 2020] Prithvijit Chattopadhyay, Yo- gesh Balaji, and Judy Hoffman. Learning to balance speci- ficity and invariance for in and out of domain generaliza- tion. In ECCV, pages 301-318, 2020.","title":"Learning to balance specificity and invariance for in and out of domain generalization","authors":"Chattopadhyay","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and P. Abbeel. Infogan: Interpretable representation learning by information max- imizing generative adversarial nets. In NIPS, 2016.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, P Abbeel","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Tianshui Chen, Tao Pu, Hefeng Wu, Yuan Xie, Lingbo Liu, and Liang Lin. Cross-domain facial expression recognition: A unified evaluation benchmark and adversarial graph learning. TPAMI, pages 9887-9903, 2021.","title":"Cross-domain facial expression recognition: A unified evaluation benchmark and adversarial graph learning","authors":"Chen","year":"2021","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dhall et al., 2011] Abhinav Dhall, Roland Goecke, Simon Lucey, and Tom Gedeon. Static facial expression analysis in tough conditions: Data, evaluation protocol and bench- mark. In ICCV workshops, pages 2106-2112, 2011.","title":"Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark","authors":"Abhinav Dhall, Roland Goecke, Simon Lucey, Tom Gedeon","year":"2011","venue":"ICCV workshops","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dhall et al., 2015] Abhinav Dhall, OV Ramana Murthy, Roland Goecke, Jyoti Joshi, and Tom Gedeon. Video and image based emotion recognition challenges in the wild: Emotiw 2015. In Proceedings of the 2015 ACM on in- ternational conference on multimodal interaction, pages 423-426, 2015.","title":"Video and image based emotion recognition challenges in the wild: Emotiw 2015","authors":"Abhinav Dhall, Roland Ov Ramana Murthy, Jyoti Goecke, Tom Joshi, Gedeon","year":"2015","venue":"Proceedings of the 2015 ACM on international conference on multimodal interaction","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gan et al., 2016] Chuang Gan, Tianbao Yang, and Boqing Gong. Learning attributes equals multi-source domain generalization. In CVPR, pages 87-97, 2016.","title":"Learning attributes equals multi-source domain generalization","authors":"Chuang Gan, Tianbao Yang, Boqing Gong","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hansen and Hess, 2007] Bruce C Hansen and Robert F Hess. Structural sparseness and spatial phase alignment in natural scenes. JOSA A, pages 1873-1885, 2007.","title":"Structural sparseness and spatial phase alignment in natural scenes","authors":"C Bruce, Robert F Hansen, Hess","year":"2007","venue":"JOSA A","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2021a] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Fsdr: Frequency space domain ran- domization for domain generalization. In CVPR, pages 6891-6902, 2021.","title":"Fsdr: Frequency space domain randomization for domain generalization","authors":"Huang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2021b] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Rda: Robust domain adaptation via fourier adversarial attacking. In ICCV, pages 8988-8999, 2021.","title":"Rda: Robust domain adaptation via fourier adversarial attacking","authors":"Huang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ji et al., 2019] Yanli Ji, Yuhan Hu, Yang Yang, Fumin Shen, and Heng Tao Shen. Cross-domain facial expression recognition via an intra-category common feature and inter-category distinction feature fusion network. Neuro- computing, pages 231-239, 2019.","title":"Cross-domain facial expression recognition via an intra-category common feature and inter-category distinction feature fusion network","authors":"Yanli Ji, Yuhan Hu, Yang Yang, Fumin Shen, Heng Tao Shen","year":"2019","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kang et al., 2019] Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In CVPR, pages 4893-4902, 2019.","title":"Contrastive adaptation network for unsupervised domain adaptation","authors":"Guoliang Kang, Lu Jiang, Yi Yang, Alexander G Hauptmann","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Deng, 2018] Shan Li and Weihong Deng. Deep emo- tion transfer network for cross-database facial expression recognition. In 2018 24th International Conference on Pattern Recognition, pages 3092-3099, 2018.","title":"Deep emotion transfer network for cross-database facial expression recognition","authors":"Shan Li, Weihong Deng","year":"2018","venue":"24th International Conference on Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li and Deng, 2020] Shan Li and Weihong Deng. A deeper look at facial expression dataset bias. IEEE Transactions on Affective Computing, pages 881-893, 2020.","title":"A deeper look at facial expression dataset bias","authors":"Shan Li, Weihong Deng","year":"2020","venue":"IEEE Transactions on Affective Computing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2017a] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier do- main generalization. In ICCV, pages 5542-5550, 2017.","title":"Deeper, broader and artier domain generalization","authors":"Li","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2017b] Shan Li, Weihong Deng, and JunPing Du. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In CVPR, pages 2852-2861, 2017.","title":"Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild","authors":"Li","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2018] Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In CVPR, pages 5400-5409, 2018.","title":"Domain generalization with adversarial feature learning","authors":"Haoliang Li, Sinno Jialin Pan, Shiqi Wang, Alex C Kot","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2021] Yingjian Li, Yingnan Gao, Bingzhi Chen, Zheng Zhang, Lei Zhu, and Guangming Lu. Jdman: Joint discriminative and mutual adaptation networks for cross- domain facial expression recognition. In ACM MM, pages 3312-3320, 2021.","title":"Jdman: Joint discriminative and mutual adaptation networks for crossdomain facial expression recognition","authors":"Li","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2022] Yingjian Li, Zheng Zhang, Bingzhi Chen, Guangming Lu, and David Zhang. Deep margin-sensitive representation learning for cross-domain facial expression recognition. IEEE Transactions on Multimedia, pages 1359-1373, 2022.","title":"Deep margin-sensitive representation learning for cross-domain facial expression recognition","authors":"Yingjian Li, Zheng Zhang, Bingzhi Chen, Guangming Lu, David Zhang","year":"2022","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lv et al., 2022] Fangrui Lv, Jian Liang, Shuang Li, Bin Zang, Chi Harold Liu, Ziteng Wang, and Di Liu. Causality inspired representation learning for domain generalization. In CVPR, pages 8046-8056, 2022.","title":"Causality inspired representation learning for domain generalization","authors":"Lv","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lyons et al., 1998] Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba. Coding facial expres- sions with gabor wavelets. In Proceedings Third IEEE international conference on automatic face and gesture recognition, pages 200-205, 1998.","title":"Coding facial expressions with gabor wavelets","authors":"Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, Jiro Gyoba","year":"1998","venue":"Proceedings Third IEEE international conference on automatic face and gesture recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Muandet et al., 2013] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via in- variant feature representation. In ICML, pages 10-18, 2013.","title":"Domain generalization via invariant feature representation","authors":"Muandet","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nussbaumer, 1981] Henri J Nussbaumer. The fast Fourier transform. Springer Berlin Heidelberg, 1981.","title":"The fast Fourier transform","authors":"J Henri, Nussbaumer","year":"1981","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Oppenheim and Lim, 1981] Alan V Oppenheim and Jae S Lim. The importance of phase in signals. Proceedings of the IEEE, pages 529-541, 1981.","title":"The importance of phase in signals","authors":"Alan V Oppenheim, Jae S Lim","year":"1981","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Piratla et al., 2020] Vihari Piratla, Praneeth Netrapalli, and Sunita Sarawagi. Efficient domain generalization via common-specific low-rank decomposition. In ICML, pages 7728-7738, 2020.","title":"Efficient domain generalization via common-specific low-rank decomposition","authors":"Praneeth Vihari Piratla, Sunita Netrapalli, Sarawagi","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Recht et al., 2019] Benjamin Recht, Rebecca Roelofs, Lud- wig Schmidt, and Vaishaal Shankar. Do imagenet classi- fiers generalize to imagenet? In ICML, pages 5389-5400, 2019.","title":"Do imagenet classifiers generalize to imagenet?","authors":"Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, Vaishaal Shankar","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, pages 618-626, 2017.","title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","authors":"Michael Ramprasaath R Selvaraju, Abhishek Cogswell, Ramakrishna Das, Devi Vedantam, Dhruv Parikh, Batra","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shao et al., 2019] Rui Shao, Xiangyuan Lan, Jiawei Li, and Pong C Yuen. Multi-adversarial discriminative deep do- main generalization for face presentation attack detection. In CVPR, pages 10023-10031, 2019.","title":"Multi-adversarial discriminative deep domain generalization for face presentation attack detection","authors":"Rui Shao, Xiangyuan Lan, Jiawei Li, Pong C Yuen","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tzeng et al., 2017] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, pages 7167-7176, 2017.","title":"Adversarial discriminative domain adaptation","authors":"Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, pages 2579-2605, 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2018] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In CVPR, pages 5265-5274, 2018.","title":"Cosface: Large margin cosine loss for deep face recognition","authors":"Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, Wei Liu","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2022] Zhuo Wang, Zezheng Wang, Zitong Yu, Weihong Deng, Jiahong Li, Tingting Gao, and Zhongyuan Wang. Domain generalization via shuffled style assembly for face anti-spoofing. In CVPR, pages 4123-4133, 2022.","title":"Domain generalization via shuffled style assembly for face anti-spoofing","authors":"Zhuo Wang, Zezheng Wang, Zitong Yu, Weihong Deng, Jiahong Li, Tingting Gao, Zhongyuan Wang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xu et al., 2021] Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. A fourier-based framework for domain generalization. In CVPR, pages 14383-14392, 2021.","title":"A fourier-based framework for domain generalization","authors":"Xu","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang et al., 2020] Yanchao Yang, Dong Lao, Ganesh Sun- daramoorthi, and Stefano Soatto. Phase consistent ecolog- ical domain adaptation. In CVPR, pages 9011-9020, 2020.","title":"Phase consistent ecological domain adaptation","authors":"Yanchao Yang, Dong Lao, Ganesh Sundaramoorthi, Stefano Soatto","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zavarez et al., 2017] Marcus Vinicius Zavarez, Rodrigo F Berriel, and Thiago Oliveira-Santos. Cross-database facial expression recognition based on fine-tuned deep convolu- tional network. In 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images, pages 405-412, 2017.","title":"Cross-database facial expression recognition based on fine-tuned deep convolutional network","authors":"Marcus Vinicius Zavarez, Rodrigo F Berriel, Thiago Oliveira-Santos","year":"2017","venue":"30th SIBGRAPI Conference on Graphics, Patterns and Images","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2021] Haifeng Zhang, Wen Su, Jun Yu, and Zengfu Wang. Weakly supervised local-global relation network for facial expression recognition. In IJCAI, pages 1040-1046, 2021.","title":"Weakly supervised local-global relation network for facial expression recognition","authors":"Haifeng Zhang, Wen Su, Jun Yu, Zengfu Wang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/145"},{"position":38,"raw":"[Zhang et al., 2022] Hanlin Zhang, Yi-Fan Zhang, Weiyang Liu, Adrian Weller, Bernhard Schölkopf, and Eric P Xing. Towards principled disentanglement for domain general- ization. In CVPR, pages 8024-8034, 2022.","title":"Towards principled disentanglement for domain generalization","authors":"Hanlin Zhang, Yi-Fan Zhang, Weiyang Liu, Adrian Weller, Bernhard Schölkopf, Eric P Xing","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhao et al., 2011] Guoying Zhao, Xiaohua Huang, Matti Taini, Stan Z Li, and Matti PietikäInen. Facial expression recognition from near-infrared videos. Image and vision computing, pages 607-619, 2011.","title":"Facial expression recognition from near-infrared videos","authors":"Guoying Zhao, Xiaohua Huang, Matti Taini, Stan Z Li, Matti Pietikäinen","year":"2011","venue":"Image and vision computing","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhou et al., 2020] Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Deep domain-adversarial im- age generation for domain generalisation. In AAAI, pages 13025-13032, 2020.","title":"Deep domain-adversarial image generation for domain generalisation","authors":"Zhou","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhou et al., 2021a] Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. TPAMI, pages 4396-4415, 2021.","title":"Domain generalization: A survey","authors":"Zhou","year":"2021","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhou et al., 2021b] Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008, 2021.","title":"Domain generalization with mixstyle","authors":"Zhou","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhu et al., 2016] Ronghang Zhu, Gaoli Sang, and Qijun Zhao. Discriminative feature adaptation for cross-domain facial expression recognition. In 2016 International Con- ference on Biometrics, pages 1-7, 2016.","title":"Discriminative feature adaptation for cross-domain facial expression recognition","authors":"Zhu","year":"2016","venue":"2016 International Conference on Biometrics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}