A Speech-driven Sign Language Avatar Animation System for Hearing Impaired Applications

Li Hu, Jiahui Li, Jiashuo Zhang, Qi Wang, Bang Zhang, Ping Tan

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

Demo Track. Pages 5912-5915.

https://doi.org/10.24963/ijcai.2022/852

PDF BibTeX Video

Abstract

Sign language is the communication language used in hearing impaired community. Recently, the research of sign language production has made great progress but still need to cope with some critical challenges. In this paper, we propose a system-level scheme and push forward the implementation of sign language production for practical usage. We build a system capable of translating speech into sign language avatar. Different from previous approach only focusing on single technology, we systematically combine algorithms of language translation, body gesture animation and facial avatar generation. We also develop two applications: Sign Language Interpretation APP and Virtual Sign Language Anchor, to facilitate easy and clear communication for hearing impaired people.

Keywords

  • Humans and AI: Cognitive Systems
  • Humans and AI: Applications
  • Humans and AI: Human-Computer Interaction
  • Machine Learning: Applications
Welcome Future Conferences AI Journal Division IJCAI Proceedings Travel Grant Program IJCAI Awards