{"type":"ScholarlyArticle","title":"ExpTalk: Diverse Emotional Expression via Adaptive Disentanglement and Refined Alignment for Speech-Driven 3D Facial Animation","authors":[{"name":"Zhan Qu","slug":"zhan-qu","given":"Zhan","family":"Qu","affiliation":""},{"name":"Shengyu Zhang","slug":"shengyu-zhang","given":"Shengyu","family":"Zhang","affiliation":""},{"name":"Mengze Li","slug":"mengze-li","given":"Mengze","family":"Li","affiliation":""},{"name":"Zhuo Chen","slug":"zhuo-chen","given":"Zhuo","family":"Chen","affiliation":""},{"name":"Chengfei Lv","slug":"chengfei-lv","given":"Chengfei","family":"Lv","affiliation":""},{"name":"Zhou Zhao","slug":"zhou-zhao","given":"Zhou","family":"Zhao","affiliation":""},{"name":"Fei Wu","slug":"fei-wu","given":"Fei","family":"Wu","affiliation":""}],"abstract":"Speech-driven 3D facial animation aims to create lifelike facial expressions that synchronize accurately with speech. Despite significant progress, many existing methods may focus on generating facial animation with a fixed emotional state, neglecting the diverse transformations of facial emotions under a given speech input. To solve this issue, we focus on exploring the refined alignment between speech representations and multiple domains in facial expression information. We aim to disentangle the spoken language and emotion facial priors from speech expressions, to guide the refinement of the facial vertices based on speech. To accomplish this objective, we propose ExpTalk, which first applies an Adaptive Disentanglement Variational Autoencoder (AD-VAE) to decouple facial expression aligned with spoken language and emotions of speech through contrastive learning. Then a Refined Alignment Diffusion (RAD) is employed to iteratively refine the decoupled facial expression priors through diffusion-based perturbations, producing facial animations that align with the emotional variations of the given speech. Extensive experiments prove the effectiveness of our ExpTalk by surpassing state-of-the-arts by a large margin.","doi":"10.24963/ijcai.2025/202","url":"https://ijcai.org/proceedings/2025/202","pdf":"https://ijcai.org/proceedings/2025/0202.pdf","markdown":"https://ijcai.org/proceedings/2025/202.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/202","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Computer Vision","track":"Main Track","number":202,"pages":{"first":1811,"last":1819},"keywords":["Computer Vision: CV: 3D computer vision","Agent-based and Multi-agent Systems: MAS: Human-agent interaction","Machine Learning: ML: Generative models","Computer Vision: CV: Multimodal learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}