{"type":"ScholarlyArticle","title":"Bridge to Non-Barrier Communication: Gloss-Prompted Fine-Grained Cued Speech Gesture Generation with Diffusion Model","authors":[{"name":"Wentao Lei","slug":"wentao-lei","given":"Wentao","family":"Lei","affiliation":"The Hong Kong University of Science and Technology ( Guangzhou ); Tencent AI Lab"},{"name":"Li Liu","slug":"li-liu","given":"Li","family":"Liu","affiliation":"The Hong Kong University of Science and Technology ( Guangzhou ); The Hong Kong University of Science and Technology"},{"name":"Jun Wang","slug":"jun-wang","given":"Jun","family":"Wang","affiliation":"Tencent AI Lab"}],"abstract":"Cued Speech (CS) is an advanced visual phonetic encoding system that integrates lip reading with hand codings, enabling people with hearing impairments to communicate efficiently. CS video generation aims to produce specific lip and gesture movements of CS from audio or text inputs. The main challenge is that given limited CS data, we strive to simultaneously generate fine-grained hand and finger movements, as well as lip movements, meanwhile the two kinds of movements need to be asynchronously aligned. Existing CS generation methods are fragile and prone to poor performance due to template-based statistical models and careful hand-crafted pre-processing to fit the models. Therefore, we propose a novel Gloss-prompted Diffusion-based CS Gesture generation framework (called GlossDiff). Specifically, to integrate additional linguistic rules knowledge into the model. we first introduce a bridging instruction called Gloss, which is an automatically generated descriptive text to establish a direct and more delicate semantic connection between spoken language and CS gestures. Moreover, we first suggest rhythm is an important paralinguistic feature for CS to improve the communication efficacy. Therefore, we propose a novel Audio-driven Rhythmic Module (ARM) to learn rhythm that matches audio speech. Moreover, in this work, we design, record, and publish the first Chinese CS dataset with four CS cuers. Extensive experiments demonstrate that our method quantitatively and qualitatively outperforms current state-of-the-art (SOTA) methods. We will release the code and data at glossdiff.github.io/.","doi":"10.24963/ijcai.2024/700","url":"https://ijcai.org/proceedings/2024/700","pdf":"https://ijcai.org/proceedings/2024/0700.pdf","markdown":"https://ijcai.org/proceedings/2024/700.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/700","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Natural Language Processing","track":"Main Track","number":700,"pages":{"first":6333,"last":6341},"keywords":["Computer Vision: CV: Applications","Natural Language Processing: NLP: Speech"],"videos":[],"references":[{"position":1,"raw":"[Ao et al., 2022] Tenglong Ao, Qingzhe Gao, Yuke Lou, Baoquan Chen, and Libin Liu. Rhythmic gesticulator. ACM Transactions on Graphics, 41(6):1-19, 2022.","title":"Rhythmic gesticulator","authors":"Ao","year":"2022","venue":"ACM Transactions on Graphics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ao et al., 2023] Tenglong Ao, Zeyi Zhang, and Libin Liu. Gesturediffuclip: Gesture diffusion model with clip la- tents. arXiv:2303.14613, 2023.","title":"Gesturediffuclip: Gesture diffusion model with clip latents","authors":"Ao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bailly et al., 2008] G. Bailly, Yu Fang, F. Elisei, and D. Beautemps. Retargeting cued speech hand gestures for different talking heads and speakers. In AVSP, 2008.","title":"Retargeting cued speech hand gestures for different talking heads and speakers","authors":"G Bailly, Yu Fang, F Elisei, D Beautemps","year":"2008","venue":"AVSP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ben et al., 2020] S. Ben, N. Camgoz, and B. Richard. Pro- gressive transformers for end-to-end sign language pro- duction. In ECCV, 2020.","title":"Progressive transformers for end-to-end sign language production","authors":"S Ben, N Camgoz, B Richard","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bhattacharya et al., 2021] U. Bhattacharya, E. Childs, N. Rewkowski, and D. Manocha. Speech2affectivegestures: Synthesizing co-speech gestures with generative adversarial affective expression learning. In ACM MM, 2021.","title":"Speech2affectivegestures: Synthesizing co-speech gestures with generative adversarial affective expression learning","authors":"U Bhattacharya, E Childs, N Rewkowski, D Manocha","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2022] Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505-1518, 2022.","title":"Wavlm: Large-scale self-supervised pre-training for full stack speech processing","authors":"Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao","year":"2022","venue":"IEEE Journal of Selected Topics in Signal Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Choutas et al., 2020] Vasileios Choutas, Georgios Pavlakos, Timo Bolkart, Dimitrios Tzionas, and Michael J. Black. Monocular expressive body regression through body- driven attention. In ECCV, 2020.","title":"Monocular expressive body regression through bodydriven attention","authors":"Vasileios Choutas, Georgios Pavlakos, Timo Bolkart, Dimitrios Tzionas, Michael J Black","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cornett, 1967] R. Orin Cornett. Cued speech. American Annals of the Deaf, 112(1):3-13, 1967.","title":"Cued speech","authors":"R, Orin Cornett","year":"1967","venue":"American Annals of the Deaf","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cox et al., 2002] Stephen Cox, Michael Lincoln, Judy Tryggvason, Melanie Nakisa, Mark Wells, Marcus Tutt, and Sanja Abbott. Tessa, a system to aid communica- tion with deaf people. In Proceedings of the fifth inter- national ACM conference on Assistive technologies, pages 205-212, 2002.","title":"Tessa, a system to aid communication with deaf people","authors":"Stephen Cox, Michael Lincoln, Judy Tryggvason, Melanie Nakisa, Mark Wells, Marcus Tutt, Sanja Abbott","year":"2002","venue":"Proceedings of the fifth international ACM conference on Assistive technologies","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Duchnowski et al., 1998] P. Duchnowski, Louis D. Braida, D. Lum, M. Sexton, Jean C. Krause, and S. Banthia. Au- tomatic generation of cued speech for the deaf: Status and outlook. In AVSP, 1998.","title":"Automatic generation of cued speech for the deaf: Status and outlook","authors":"P Duchnowski, Louis D Braida, D Lum, M Sexton, Jean C Krause, S Banthia","year":"1998","venue":"AVSP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fernandez-Lopez et al., 2017] A. Fernandez-Lopez, O. Martínez, and M. Sukno. Towards estimating the upper bound of visual-speech recognition: The visual lip-reading feasibility database. In FG, 2017.","title":"Towards estimating the upper bound of visual-speech recognition: The visual lip-reading feasibility database","authors":"Fernandez-Lopez","year":"2017","venue":"FG","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ginosar et al., 2019] S. Ginosar, A. Bar, G. Kohavi, C. Chan, A. Owens, and J. Malik. Learning individual styles of conversational gesture. In CVPR, 2019.","title":"Learning individual styles of conversational gesture","authors":"S Ginosar, A Bar, G Kohavi, C Chan, A Owens, J Malik","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guo et al., 2021] Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, and Juyong Zhang. Ad-nerf: Audio driven neural radiance fields for talking head syn- thesis. In ICCV, 2021.","title":"Ad-nerf: Audio driven neural radiance fields for talking head synthesis","authors":"Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, Juyong Zhang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ho and Salimans, 2022] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv:2207.12598, 2022.","title":"Classifier-free diffusion guidance","authors":"Salimans Ho, Jonathan Ho, Tim Salimans","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang and Belongie, 2017] Xun Huang and Serge Be- longie. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, 2017.","title":"Arbitrary style transfer in real-time with adaptive instance normalization","authors":"Belongie Huang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ji et al., 2023] Longbin Ji, Pengfei Wei, Yi Ren, Jinglin Liu, Chen Zhang, and Xiang Yin. C2g2: Controllable co-speech gesture generation with latent diffusion model. arXiv:2308.15016, 2023.","title":"C2g2: Controllable co-speech gesture generation with latent diffusion model","authors":"Longbin Ji, Pengfei Wei, Yi Ren, Jinglin Liu, Chen Zhang, Xiang Yin","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lebourdais et al., 2022] Martin Lebourdais, Marie Tahon, Antoine Laurent, and Sylvain Meignier. Overlapped speech and gender detection with wavlm pre-trained fea- tures. arXiv preprint arXiv:2209.04167, 2022.","title":"Overlapped speech and gender detection with wavlm pre-trained features","authors":"Martin Lebourdais, Marie Tahon, Antoine Laurent, Sylvain Meignier","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liddell and Johnson, 1989] K. Scott Liddell and E. Robert Johnson. American sign language: The phonological base. Sign Language Studies, pages 195-278, 1989.","title":"American sign language: The phonological base","authors":"K, Scott Liddell, E Robert Johnson","year":"1989","venue":"Sign Language Studies","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu and Feng, 2019] Li Liu and Gang Feng. A pilot study on mandarin chinese cued speech. American Annals of the Deaf, 164:496-518, 2019.","title":"A pilot study on mandarin chinese cued speech","authors":"Li Liu, Gang Feng","year":"2019","venue":"American Annals of the Deaf","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu and Liu, 2023] Lei Liu and Li Liu. Cross-modal mutual learning for cued speech recognition. In ICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.","title":"Cross-modal mutual learning for cued speech recognition","authors":"Lei Liu, Li Liu","year":"2023","venue":"ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2018] Li Liu, H. Thomas, Gang Feng, and B. De- nis. Visual recognition of continuous cued speech using a tandem cnn-hmm approach. In INTERSPEECH, 2018.","title":"Visual recognition of continuous cued speech using a tandem cnn-hmm approach","authors":"Li Liu, H Thomas, Gang Feng, B Denis","year":"2018","venue":"INTERSPEECH","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2019] Li Liu, Jianze Li, Gang Feng, and Xiao- Ping Steven Zhang. Automatic detection of the temporal segmentation of hand movements in british english cued speech. In INTERSPEECH, 2019.","title":"Automatic detection of the temporal segmentation of hand movements in british english cued speech","authors":"Li Liu, Jianze Li, Gang Feng, Xiao-Ping Steven Zhang","year":"2019","venue":"INTERSPEECH","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu et al., 2020] Li Liu, Gang Feng, Denis Beautemps, and Xiao-Ping Zhang. Re-synchronization using the hand pre- ceding model for multi-modal fusion in automatic contin- uous cued speech recognition. IEEE Transactions on Mul- timedia, 23:292-305, 2020.","title":"Re-synchronization using the hand preceding model for multi-modal fusion in automatic continuous cued speech recognition","authors":"Li Liu, Gang Feng, Denis Beautemps, Xiao-Ping Zhang","year":"2020","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liu et al., 2021] Li Liu, Gang Feng, B. Denis, and Xiao- Ping Zhang. Re-synchronization using the hand preced- ing model for multi-modal fusion in automatic continuous cued speech recognition. IEEE Transactions on Multime- dia, 23:292-305, 2021.","title":"Re-synchronization using the hand preceding model for multi-modal fusion in automatic continuous cued speech recognition","authors":"Li Liu, Gang Feng, B Denis, Xiao-Ping Zhang","year":"2021","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Liu et al., 2022] Xian Liu, Qianyi Wu, Hang Zhou, Ying- hao Xu, Rui Qian, Xinyi Lin, Xiaowei Zhou, Wayne Wu, Bo Dai, and Bolei Zhou. Learning hierarchical cross- modal association for co-speech gesture generation. In CVPR, 2022.","title":"Learning hierarchical crossmodal association for co-speech gesture generation","authors":"Xian Liu, Qianyi Wu, Hang Zhou, Yinghao Xu, Rui Qian, Xinyi Lin, Xiaowei Zhou, Wayne Wu, Bo Dai, Bolei Zhou","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liu et al., 2023] Haohe Liu, Zehua Chen, Yi Yuan, Xin- hao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio gen- eration with latent diffusion models. arXiv preprint arXiv:2301.12503, 2023.","title":"Audioldm: Text-to-audio generation with latent diffusion models","authors":"Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[OpenAI and et.al, 2023] OpenAI and Josh et.al. Gpt-4 tech- nical report, 2023.","title":"OpenAI and Josh et.al","authors":"Et Openai","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Petrovich et al., 2021] Mathis Petrovich, Michael J Black, and Gül Varol. Action-conditioned 3d human motion syn- thesis with transformer vae. In ICCV, 2021.","title":"Action-conditioned 3d human motion synthesis with transformer vae","authors":"Mathis Petrovich, Michael J Black, Gül Varol","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Power et al., 2007] Des Power, Mary R Power, and Bernd Rehling. German deaf people using text communication: Short message service, tty, relay services, fax, and e-mail. American Annals of the Deaf, 152(3):291-301, 2007.","title":"Des Power, Mary R Power, and Bernd Rehling. German deaf people using text communication: Short message service, tty, relay services, fax, and e-mail","authors":"Power","year":"2007","venue":"American Annals of the Deaf","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Puviarasan and Palanivel, 2011] N. Puviarasan and S. Palanivel. Lip reading of hearing impaired per- sons using hmm. Expert Systems with Applications, 38(4):4477-4481, 2011.","title":"Lip reading of hearing impaired persons using hmm","authors":"N Puviarasan, S Palanivel","year":"2011","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark","year":"2021","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Reynolds, 2007] S. Reynolds. An examination of cued speech as a tool for language, literacy, and bilingualism for children who are deaf or hard of hearing. Independent Studies and Capstones. Paper 315., 2007.","title":"An examination of cued speech as a tool for language, literacy, and bilingualism for children who are deaf or hard of hearing","authors":"S Reynolds","year":"2007","venue":"Independent Studies and Capstones","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.","title":"High-resolution image synthesis with latent diffusion models","authors":"Rombach","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Rothauser, 1969] E. H. Rothauser. Ieee recommended prac- tice for speech quality measurements. In Technical Report No. 297, 1969.","title":"Ieee recommended practice for speech quality measurements","authors":"E H Rothauser","year":"1969","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Stokoe, 2005] Jr. Stokoe, C. William. Sign Language Struc- ture: An Outline of the Visual Communication Systems of the American Deaf. The Journal of Deaf Studies and Deaf Education, 10(1):3-37, 2005.","title":"Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf","authors":"C Stokoe, William","year":"2005","venue":"The Journal of Deaf Studies and Deaf Education","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Stoll et al., 2020] S. Stoll, Necati C. Camgoz, S. Hadfield, and R. Bowden. Text2sign: Towards sign language pro- duction using neural machine translation and generative adversarial networks. IJCV, 2020.","title":"Text2sign: Towards sign language production using neural machine translation and generative adversarial networks","authors":"S Stoll, Necati C Camgoz, S Hadfield, R Bowden","year":"2020","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Tevet et al., 2022a] Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Ex- posing human motion generation to clip space. In ECCV. Springer, 2022.","title":"Motionclip: Exposing human motion generation to clip space","authors":"Tevet","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Tevet et al., 2022b] Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In ICLR, 2022.","title":"Human motion diffusion model","authors":"Tevet","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Timothy, 2003] R. Timothy. Linguistics of american sign language: An introduction. Studies in Second Language Acquisition, 25(1):157-158, 2003.","title":"Linguistics of american sign language: An introduction","authors":"R Timothy","year":"2003","venue":"Studies in Second Language Acquisition","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[van der Maaten and Hinton, 2008] L. van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of Ma- chine Learning Research, 9(86):2579-2605, 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, ; L Hinton, G Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yang et al., 2023] Sicheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, Ming Cheng, and Long Xiao. Diffusestylegesture: Stylized audio- driven co-speech gesture generation with diffusion mod- els. arXiv:2305.04919, 2023.","title":"Diffusestylegesture: Stylized audiodriven co-speech gesture generation with diffusion models","authors":"Sicheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, Ming Cheng, Long Xiao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yi and Deva, 2013] Y. Yi and R. Deva. Articulated hu- man detection with flexible mixtures of parts. TPAMI, 35(12):2878-2890, 2013.","title":"Articulated human detection with flexible mixtures of parts","authors":"Y Yi, R Deva","year":"2013","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Youngwoo et al., 2019] Y. Youngwoo, K. Woo-Ri, J. Minsu, L. Jaeyeon, K. Jaehong, and L. Geehyuk. Robots learn social skills: End-to-end learning of co- speech gesture generation for humanoid robots. In ICRA, 2019.","title":"Robots learn social skills: End-to-end learning of cospeech gesture generation for humanoid robots","authors":"Y Youngwoo, K Woo-Ri, J Minsu, L Jaeyeon, K Jaehong, L Geehyuk","year":"2019","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Youngwoo et al., 2020] Y. Youngwoo, C. Bok, L. Joo- Haeng, J. Minsu, L. Jaeyeon, K. Jaehong, and L. Gee- hyuk. Speech gesture generation from the trimodal context of text, audio, and speaker identity. ACM Transactions on Graphics, 39(6):1-16, 2020.","title":"Speech gesture generation from the trimodal context of text, audio, and speaker identity","authors":"Y Youngwoo, C Bok, L Joo-Haeng, J Minsu, L Jaeyeon, K Jaehong, L Geehyuk","year":"2020","venue":"ACM Transactions on Graphics","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhang et al., 2022] Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. arXiv:2208.15001, 2022.","title":"Motiondiffuse: Text-driven human motion generation with diffusion model","authors":"Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, Ziwei Liu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Zhao et al., 2023] Weiyu Zhao, Liangxiao Hu, and Sheng- ping Zhang. Diffugesture: Generating human gesture from two-person dialogue with diffusion models. In ICMI. 2023.","title":"Diffugesture: Generating human gesture from two-person dialogue with diffusion models","authors":"Weiyu Zhao, Liangxiao Hu, Shengping Zhang","year":"2023","venue":"ICMI","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zhi et al., 2023] Yihao Zhi, Xiaodong Cun, Xuelin Chen, Xi Shen, Wen Guo, Shaoli Huang, and Shenghua Gao. Livelyspeaker: Towards semantic-aware co-speech ges- ture generation. In ICCV, 2023.","title":"Livelyspeaker: Towards semantic-aware co-speech gesture generation","authors":"Yihao Zhi, Xiaodong Cun, Xuelin Chen, Xi Shen, Wen Guo, Shaoli Huang, Shenghua Gao","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhu et al., 2023] Lingting Zhu, Xian Liu, Xuanyu Liu, Rui Qian, Ziwei Liu, and Lequan Yu. Taming diffusion models for audio-driven co-speech gesture generation. In CVPR, 2023.","title":"Taming diffusion models for audio-driven co-speech gesture generation","authors":"Lingting Zhu, Xian Liu, Xuanyu Liu, Rui Qian, Ziwei Liu, Lequan Yu","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}