---
title: "SyncGaussian: Stable 3D Gaussian-Based Talking Head Generation with Enhanced Lip Sync via Discriminative Speech Features"
authors:
  - "Ke Liu"
  - "Jiwei Wei"
  - "Shiyuan He"
  - "Zeyu Ma"
  - "Chaoning Zhang"
  - "Ning Xie"
  - "Yang Yang"
doi: "10.24963/ijcai.2025/176"
url: "https://ijcai.org/proceedings/2025/176"
pdf: "https://ijcai.org/proceedings/2025/0176.pdf"
venue: "Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence"
year: 2025
pages: "1576-1584"
track: "Main Track"
keywords:
  - "Computer Vision: CV: Image and video synthesis and generation"
  - "Computer Vision: CV: Image and video retrieval"
  - "Humans and AI: HAI: Applications"
  - "Computer Vision: CV: Multimodal learning"
---

# SyncGaussian: Stable 3D Gaussian-Based Talking Head Generation with Enhanced Lip Sync via Discriminative Speech Features

Ke Liu, Jiwei Wei, Shiyuan He, Zeyu Ma, Chaoning Zhang, Ning Xie, Yang Yang

*Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence*, pages 1576-1584. DOI: [10.24963/ijcai.2025/176](https://doi.org/10.24963/ijcai.2025/176)

## Abstract

Generating high-fidelity talking heads that maintain stable head poses and achieve robust lip sync remains a significant challenge. Although methods based on 3D Gaussian Splatting (3DGS) offer a promising solution via point-based deformation, they suffer from inconsistent head dynamics and mismatched mouth movements due to unstable Gaussian initialization and incomplete speech features. To overcome these limitations, we introduce SyncGaussian, a 3DGS-based framework that ensures stable head poses, enhanced lip sync, and realistic appearances with real-time rendering. SyncGaussian employs a stable head Gaussian initialization strategy to mitigate head jitter by optimizing commonly used rough head pose parameters. To enhance lip sync, we propose a sync-enhanced encoder that leverages audio-to-text and audio-to-visual speech features. Guided by a tailored cosine similarity loss function, the encoder integrates discriminative speech features through a multi-level sync adaptation mechanism, enabling the learning of an adaptive speech feature space. Extensive experiments demonstrate that SyncGaussian outperforms state-of-the-art methods in image quality, dynamic motion, and lip sync, with the potential for real-time applications.