GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer

Lin, Yihong; Fan, Zhaoxin; Wu, Xianjia; Xiong, Lingyu; Li, Xiandong; Kang, Wenxiong; Peng, Liang; Lei, Songju; Xu, Huang

doi:10.24963/ijcai.2025/173

GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer

Yihong Lin, Zhaoxin Fan, Xianjia Wu, Lingyu Xiong, Xiandong Li, Wenxiong Kang, Liang Peng, Songju Lei, Huang Xu

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 1548-1556.

https://doi.org/10.24963/ijcai.2025/173

Keywords

Computer Vision: CV: Multimodal learning
Computer Vision: CV: 3D computer vision

Abstract

Speech-driven talking head generation is a critical yet challenging task with applications in augmented reality and virtual human modeling. While recent approaches using autoregressive and diffusion-based models have achieved notable progress, they often suffer from modality inconsistencies, particularly misalignment between audio and mesh, leading to reduced motion diversity and lip-sync accuracy. To address this, we propose GLDiTalker, a novel speech-driven 3D facial animation model based on a Graph Latent Diffusion Transformer. GLDiTalker resolves modality misalignment by diffusing signals within a quantized spatiotemporal latent space. It employs a two-stage training pipeline: the Graph-Enhanced Quantized Space Learning Stage ensures lip-sync accuracy, while the Space-Time Powered Latent Diffusion Stage enhances motion diversity. Together, these stages enable GLDiTalker to generate realistic, temporally stable 3D facial animations. Extensive evaluations on standard benchmarks demonstrate that GLDiTalker outperforms existing methods, achieving superior results in both lip-sync accuracy and motion diversity.

Yihong Lin, Zhaoxin Fan, Xianjia Wu, Lingyu Xiong, Xiandong Li, Wenxiong Kang, Liang Peng, Songju Lei, and Huang Xu. 2025. GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 1548–1556. https://doi.org/10.24963/ijcai.2025/173

@inproceedings{ijcai2025p173,
  title     = {GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer},
  author    = {Lin, Yihong and Fan, Zhaoxin and Wu, Xianjia and Xiong, Lingyu and Li, Xiandong and Kang, Wenxiong and Peng, Liang and Lei, Songju and Xu, Huang},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {1548--1556},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/173},
  url       = {https://doi.org/10.24963/ijcai.2025/173},
}

TY  - CPAPER
TI  - GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer
AU  - Lin, Yihong
AU  - Fan, Zhaoxin
AU  - Wu, Xianjia
AU  - Xiong, Lingyu
AU  - Li, Xiandong
AU  - Kang, Wenxiong
AU  - Peng, Liang
AU  - Lei, Songju
AU  - Xu, Huang
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 1548
EP  - 1556
DO  - 10.24963/ijcai.2025/173
UR  - https://doi.org/10.24963/ijcai.2025/173
ER  -