{"type":"ScholarlyArticle","title":"DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models","authors":[{"name":"Sicheng Yang","slug":"sicheng-yang","given":"Sicheng","family":"Yang","affiliation":"Shenzhen International Graduate School , Tsinghua University , Shenzhen , China"},{"name":"Zhiyong Wu","slug":"zhiyong-wu","given":"Zhiyong","family":"Wu","affiliation":"Shenzhen International Graduate School , Tsinghua University , Shenzhen , China; The Chinese University of Hong Kong , Hong Kong SAR , China"},{"name":"Minglei Li","slug":"minglei-li","given":"Minglei","family":"Li","affiliation":"Huawei Cloud Computing Technologies Co., Ltd , Shenzhen , China"},{"name":"Zhensong Zhang","slug":"zhensong-zhang","given":"Zhensong","family":"Zhang","affiliation":"Huawei Noah's Ark Lab , Shenzhen , China"},{"name":"Lei Hao","slug":"lei-hao","given":"Lei","family":"Hao","affiliation":"Huawei Noah's Ark Lab , Shenzhen , China"},{"name":"Weihong Bao","slug":"weihong-bao","given":"Weihong","family":"Bao","affiliation":"Shenzhen International Graduate School , Tsinghua University , Shenzhen , China"},{"name":"Ming Cheng","slug":"ming-cheng","given":"Ming","family":"Cheng","affiliation":"Shenzhen International Graduate School , Tsinghua University , Shenzhen , China"},{"name":"Long Xiao","slug":"long-xiao","given":"Long","family":"Xiao","affiliation":"Shenzhen International Graduate School , Tsinghua University , Shenzhen , China"}],"abstract":"The art of communication beyond speech there are gestures. The automatic co-speech gesture generation draws much attention in computer animation. It is a challenging task due to the diversity of gestures and the difficulty of matching the rhythm and semantics of the gesture to the corresponding speech. To address these problems, we present DiffuseStyleGesture, a diffusion model based speech-driven gesture generation approach. It generates high-quality, speech-matched, stylized, and diverse co-speech gestures based on given speeches of arbitrary length. Specifically, we introduce cross-local attention and self-attention to the gesture diffusion pipeline to generate better speech matched and realistic gestures. We then train our model with classifier-free guidance to control the gesture style by interpolation or extrapolation. Additionally, we improve the diversity of generated gestures with different initial gestures and noise. Extensive experiments show that our method outperforms recent approaches on speech-driven gesture generation. Our code, pre-trained models, and demos are available at https://github.com/YoungSeng/DiffuseStyleGesture.","doi":"10.24963/ijcai.2023/650","url":"https://ijcai.org/proceedings/2023/650","pdf":"https://ijcai.org/proceedings/2023/0650.pdf","markdown":"https://ijcai.org/proceedings/2023/650.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/650","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI and Arts","track":"AI and Arts","number":650,"pages":{"first":5860,"last":5868},"keywords":["Application domains: Music and sound","Theory and philosophy of arts and creativity in AI systems: Social (multi-agent) creativity and human-computer co-creation","Application domains: Performances, dance","Methods and resources: Machine learning, deep learning, neural models, reinforcement learning"],"videos":[],"references":[{"position":1,"raw":"[Ahuja et al., 2020] Chaitanya Ahuja, Dong Won Lee, Yukiko I. Nakano, and Louis-Philippe Morency. Style transfer for co-speech gesture animation: A multi-speaker conditional-mixture approach. In Computer Vision - ECCV, volume 12363 of Lecture Notes in Computer Sci- ence, pages 248-265, 2020.","title":"Style transfer for co-speech gesture animation: A multi-speaker conditional-mixture approach","authors":"Chaitanya Ahuja, Dong Won Lee, Yukiko I Nakano, Louis-Philippe Morency","year":"2020","venue":"Computer Vision -ECCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alexanderson et al., 2020a] Simon Alexanderson, Gus- tav Eje Henter, Taras Kucherenko, and Jonas Beskow. Style-controllable speech-driven gesture synthesis using normalising flows. Comput. Graph. Forum, 39(2):487- 496, 2020.","title":"Style-controllable speech-driven gesture synthesis using normalising flows","authors":"Alexanderson","year":"2020","venue":"Comput. Graph. Forum","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Alexanderson et al., 2020b] Simon Alexanderson, Gus- tav Eje Henter, Taras Kucherenko, and Jonas Beskow. Style-controllable speech-driven gesture synthesis us- ing normalising flows. Computer Graphics Forum, 39(2):487-496, 2020.","title":"Style-controllable speech-driven gesture synthesis using normalising flows","authors":"Alexanderson","year":"2020","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Alexanderson et al., 2021] Simon Alexanderson, Éva Székely, Gustav Eje Henter, Taras Kucherenko, and Jonas Beskow. Generating coherent spontaneous speech and gesture from text. CoRR, abs/2101.05684, 2021.","title":"Generating coherent spontaneous speech and gesture from text","authors":"Éva Simon Alexanderson, Gustav Eje Székely, Taras Henter, Jonas Kucherenko, Beskow","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Alexanderson et al., 2022] Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter. Listen, de- noise, action! audio-driven motion synthesis with diffu- sion models. CoRR, abs/2211.09707, 2022.","title":"Listen, denoise, action! audio-driven motion synthesis with diffusion models","authors":"Rajmund Simon Alexanderson, Jonas Nagy, Gustav Eje Beskow, Henter","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ao et al., 2022] Tenglong Ao, Qingzhe Gao, Yuke Lou, Baoquan Chen, and Libin Liu. Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchi- cal neural embeddings. ACM Trans. Graph., 41(6):209:1- 209:19, 2022.","title":"Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchical neural embeddings","authors":"Ao","year":"2022","venue":"ACM Trans. Graph","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Beltagy et al., 2020] Iz Beltagy, Matthew E. Peters, and Ar- man Cohan. Longformer: The long-document trans- former. CoRR, abs/2004.05150, 2020.","title":"Longformer: The long-document transformer","authors":"Iz Beltagy, Matthew E Peters, Arman Cohan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chang et al., 2022a] Che-Jui Chang, Sen Zhang, and Mub- basir Kapadia. The ivi lab entry to the genea challenge 2022 -a tacotron2 based method for co-speech gesture generation with locality-constraint attention mechanism. In Proceedings of the 2022 International Conference on Multimodal Interaction, ICMI '22, page 784-789, 2022.","title":"The ivi lab entry to the genea challenge 2022 -a tacotron2 based method for co-speech gesture generation with locality-constraint attention mechanism","authors":"Chang","year":"2022","venue":"Proceedings of the 2022 International Conference on Multimodal Interaction, ICMI '22","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chang et al., 2022b] Ziyi Chang, Edmund JC Findlay, Haozheng Zhang, and Hubert PH Shum. Unifying human motion synthesis and style transfer with de- noising diffusion probabilistic models. arXiv preprint arXiv:2212.08526, 2022.","title":"Unifying human motion synthesis and style transfer with denoising diffusion probabilistic models","authors":"Chang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chen et al., 2022] Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, and Furu Wei. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE J. Sel. Top. Signal Process., 16(6):1505-1518, 2022.","title":"Wavlm: Large-scale self-supervised pre-training for full stack speech processing","authors":"Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Jian Yao Qian, Michael Wu, Xiangzhan Zeng, Furu Yu, Wei","year":"2022","venue":"IEEE J. Sel. Top. Signal Process","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dabral et al., 2022] Rishabh Dabral, Muhammad Hamza Mughal, Vladislav Golyanik, and Christian Theobalt. Mo- fusion: A framework for denoising-diffusion-based mo- tion synthesis. CoRR, abs/2212.04495, 2022.","title":"Mofusion: A framework for denoising-diffusion-based motion synthesis","authors":"Rishabh Dabral, Muhammad Hamza Mughal, Vladislav Golyanik, Christian Theobalt","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ghorbani et al., 2022] Saeed Ghorbani, Ylva Ferstl, Daniel Holden, Nikolaus F Troje, and Marc-André Carbonneau. Zeroeggs: Zero-shot example-based gesture generation from speech. arXiv preprint arXiv:2209.07556, 2022.","title":"Zeroeggs: Zero-shot example-based gesture generation from speech","authors":"Saeed Ghorbani, Ylva Ferstl, Daniel Holden, Marc-André Nikolaus F Troje, Carbonneau","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ginosar et al., 2019] Shiry Ginosar, Amir Bar, Gefen Ko- havi, Caroline Chan, Andrew Owens, and Jitendra Malik. Learning individual styles of conversational gesture. In IEEE Conference on Computer Vision and Pattern Recog- nition, CVPR, pages 3497-3506, 2019.","title":"Learning individual styles of conversational gesture","authors":"Shiry Ginosar, Amir Bar, Gefen Kohavi, Caroline Chan, Andrew Owens, Jitendra Malik","year":"2019","venue":"IEEE Conference on Computer Vision and Pattern Recognition, CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Habibie et al., 2021] Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Lingjie Liu, Hans-Peter Seidel, Gerard Pons-Moll, Mohamed Elgharib, and Christian Theobalt. Learning speech-driven 3d conversational gestures from video. In IVA '21: ACM International Conference on Intelligent Virtual Agents, pages 101-108, 2021.","title":"Learning speech-driven 3d conversational gestures from video","authors":"Habibie","year":"2021","venue":"IVA '21: ACM International Conference on Intelligent Virtual Agents","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Habibie et al., 2022] Ikhsanul Habibie, Mohamed Elgharib, Kripasindhu Sarkar, Ahsan Abdullah, Simbarashe Nyat- sanga, Michael Neff, and Christian Theobalt. A motion matching-based framework for controllable gesture syn- thesis from speech. In SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference, pages 46:1-46:9, 2022.","title":"A motion matching-based framework for controllable gesture synthesis from speech","authors":"Habibie","year":"2022","venue":"SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ho and Salimans, 2022] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, abs/2207.12598, 2022.","title":"Classifier-free diffusion guidance","authors":"Jonathan Ho, Tim Salimans","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Huber, 1992] Peter J Huber. Robust estimation of a location parameter. In Breakthroughs in statistics, pages 492-518. 1992.","title":"Robust estimation of a location parameter","authors":"J Peter, Huber","year":"1992","venue":"Breakthroughs in statistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kim et al., 2022] Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free-form language-based motion synthe- sis & editing. arXiv preprint arXiv:2209.00349, 2022.","title":"Flame: Free-form language-based motion synthesis & editing","authors":"Jihoon Kim, Jiseob Kim, Sungjoon Choi","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kitaev et al., 2020] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In 8th International Conference on Learning Representa- tions, ICLR April 26-30, 2020.","title":"Reformer: The efficient transformer","authors":"Nikita Kitaev, Lukasz Kaiser, Anselm Levskaya","year":"2020","venue":"8th International Conference on Learning Representations, ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kucherenko et al., 2021] Taras Kucherenko, Patrik Jonell, Youngwoo Yoon, Pieter Wolfert, and Gustav Eje Henter. A large, crowdsourced evaluation of gesture generation sys- tems on common data: The GENEA challenge 2020. In IUI '21: 26th International Conference on Intelligent User Interfaces, April 13-17, pages 11-21, 2021.","title":"A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA challenge 2020","authors":"Taras Kucherenko, Patrik Jonell, Youngwoo Yoon, Pieter Wolfert, Gustav Eje Henter","year":"2021","venue":"IUI '21: 26th International Conference on Intelligent User Interfaces","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2021a] Jing Li, Di Kang, Wenjie Pei, Xuefei Zhe, Ying Zhang, Zhenyu He, and Linchao Bao. Au- dio2gestures: Generating diverse gestures from speech au- dio with conditional variational autoencoders. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV, pages 11273-11282, 2021.","title":"Audio2gestures: Generating diverse gestures from speech audio with conditional variational autoencoders","authors":"Li","year":"2021","venue":"IEEE/CVF International Conference on Computer Vision, ICCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2021b] Jing Li, Di Kang, Wenjie Pei, Xuefei Zhe, Ying Zhang, Zhenyu He, and Linchao Bao. Au- dio2gestures: Generating diverse gestures from speech au- dio with conditional variational autoencoders. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11273-11282, 2021.","title":"Audio2gestures: Generating diverse gestures from speech audio with conditional variational autoencoders","authors":"Li","year":"2021","venue":"IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2022] Siyao Li, Weijiang Yu, Tianpei Gu, Chunze Lin, Quan Wang, Chen Qian, Chen Change Loy, and Ziwei Liu. Bailando: 3d dance generation by actor-critic GPT with choreographic memory. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR June 18- 24, pages 11040-11049, 2022.","title":"Bailando: 3d dance generation by actor-critic GPT with choreographic memory","authors":"Li","year":"2022","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liang et al., 2022] Yuanzhi Liang, Qianyu Feng, Linchao Zhu, Li Hu, Pan Pan, and Yi Yang. SEEG: semantic ener- gized co-speech gesture generation. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR, pages 10463-10472, 2022.","title":"SEEG: semantic energized co-speech gesture generation","authors":"Yuanzhi Liang, Qianyu Feng, Linchao Zhu, Li Hu, Pan Pan, Yi Yang","year":"2022","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lin et al., 2022] Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Weizhu Chen, and Nan Duan. GENIE: large scale pre-training for text generation with diffusion model. CoRR, abs/2212.11685, 2022.","title":"GENIE: large scale pre-training for text generation with diffusion model","authors":"Lin","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liu et al., 2022] Xian Liu, Qianyi Wu, Hang Zhou, Ying- hao Xu, Rui Qian, Xinyi Lin, Xiaowei Zhou, Wayne Wu, Bo Dai, and Bolei Zhou. Learning hierarchical cross- modal association for co-speech gesture generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pages 10452-10462, 2022.","title":"Learning hierarchical crossmodal association for co-speech gesture generation","authors":"Xian Liu, Qianyi Wu, Hang Zhou, Yinghao Xu, Rui Qian, Xinyi Lin, Xiaowei Zhou, Wayne Wu, Bo Dai, Bolei Zhou","year":"2022","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Luvizon et al., 2023] Diogo Luvizon, Marc Habermann, Vladislav Golyanik, Adam Kortylewski, and Christian Theobalt. Scene-aware 3d multi-human motion capture from a single camera. CoRR, abs/2301.05175, 2023.","title":"Scene-aware 3d multi-human motion capture from a single camera","authors":"Marc Diogo Luvizon, Vladislav Habermann, Adam Golyanik, Christian Kortylewski, Theobalt","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[McGill et al., 1978] Robert McGill, John W Tukey, and Wayne A Larsen. Variations of box plots. The american statistician, 32(1):12-16, 1978.","title":"Variations of box plots","authors":"Robert Mcgill, John W Tukey, Wayne A Larsen","year":"1978","venue":"The american statistician","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Mei and Patel, 2022] Kangfu Mei and Vishal M. Pa- tel. VIDM: video implicit diffusion models. CoRR, abs/2212.00235, 2022.","title":"VIDM: video implicit diffusion models","authors":"Patel ; Mei, Kangfu Mei, Vishal M Patel","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Nyatsanga et al., 2023] Simbarashe Nyatsanga, Taras Kucherenko, Chaitanya Ahuja, Gustav Eje Henter, and Michael Neff. A comprehensive review of data- driven co-speech gesture generation. arXiv preprint arXiv:2301.05339, 2023.","title":"A comprehensive review of datadriven co-speech gesture generation","authors":"Simbarashe Nyatsanga, Taras Kucherenko, Chaitanya Ahuja, Gustav Eje Henter, Michael Neff","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Rae and Razavi, 2020] Jack W. Rae and Ali Razavi. Do transformers need deep long-range memory? In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7524-7529, 2020.","title":"Do transformers need deep long-range memory?","authors":"Jack W Rae, Ali Razavi","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ramesh et al., 2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.","title":"Hierarchical text-conditional image generation with clip latents","authors":"Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Ren et al., 2022] Zhiyuan Ren, Zhihong Pan, Xin Zhou, and Le Kang. Diffusion motion: Generate text-guided 3d human motion by diffusion model. arXiv preprint arXiv:2210.12315, 2022.","title":"Diffusion motion: Generate text-guided 3d human motion by diffusion model","authors":"Zhiyuan Ren, Zhihong Pan, Xin Zhou, Le Kang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Roy et al., 2021] Aurko Roy, Mohammad Saffar, Ashish Vaswani, and David Grangier. Efficient content-based sparse attention with routing transformers. Trans. Assoc. Comput. Linguistics, 9:53-68, 2021.","title":"Efficient content-based sparse attention with routing transformers","authors":"Roy","year":"2021","venue":"Trans. Assoc. Comput. Linguistics","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Tevet et al., 2022] Guy Tevet, Sigal Raab, Brian Gor- don, Yonatan Shafir, Daniel Cohen-Or, and Amit H Bermano. Human motion diffusion model. arXiv preprint arXiv:2209.14916, 2022.","title":"Human motion diffusion model","authors":"Tevet","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xie et al., 2022] Pan Xie, Qipeng Zhang, Zexian Li, Hao Tang, Yao Du, and Xiaohui Hu. Vector quantized diffu- sion model with codeunet for text-to-sign pose sequences generation. arXiv preprint arXiv:2208.09141, 2022.","title":"Vector quantized diffusion model with codeunet for text-to-sign pose sequences generation","authors":"Xie","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yang et al., 2022] Sicheng Yang, Zhiyong Wu, Minglei Li, Mengchen Zhao, Jiuxin Lin, Liyang Chen, and Weihong Bao. The reprgesture entry to the GENEA challenge 2022. In International Conference on Multimodal Interaction, ICMI, November 7-11, pages 758-763, 2022.","title":"The reprgesture entry to the GENEA challenge 2022","authors":"Sicheng Yang, Zhiyong Wu, Minglei Li, Mengchen Zhao, Jiuxin Lin, Liyang Chen, Weihong Bao","year":"2022","venue":"International Conference on Multimodal Interaction","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yi et al., 2022] Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, and Michael J Black. Generating holistic 3d human motion from speech. arXiv preprint arXiv:2212.04420, 2022.","title":"Generating holistic 3d human motion from speech","authors":"Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, Michael J Black","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yoon et al., 2019] Youngwoo Yoon, Woo-Ri Ko, Minsu Jang, Jaeyeon Lee, Jaehong Kim, and Geehyuk Lee. Robots learn social skills: End-to-end learning of co- speech gesture generation for humanoid robots. In Inter- national Conference on Robotics and Automation, ICRA, pages 4303-4309, 2019.","title":"Robots learn social skills: End-to-end learning of cospeech gesture generation for humanoid robots","authors":"Yoon","year":"2019","venue":"International Conference on Robotics and Automation, ICRA","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yoon et al., 2020] Youngwoo Yoon, Bok Cha, Joo-Haeng Lee, Minsu Jang, Jaeyeon Lee, Jaehong Kim, and Gee- hyuk Lee. Speech gesture generation from the trimodal context of text, audio, and speaker identity. ACM Transac- tions on Graphics (TOG), 39(6):1-16, 2020.","title":"Speech gesture generation from the trimodal context of text, audio, and speaker identity","authors":"Youngwoo Yoon, Bok Cha, Joo-Haeng Lee, Minsu Jang, Jaeyeon Lee, Jaehong Kim, Geehyuk Lee","year":"2020","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yoon et al., 2022] Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. The GENEA challenge 2022: A large evaluation of data-driven co-speech gesture generation. In International Conference on Multimodal Interaction, ICMI, November 7-11, pages 736-747, 2022.","title":"The GENEA challenge 2022: A large evaluation of data-driven co-speech gesture generation","authors":"Yoon","year":"2022","venue":"International Conference on Multimodal Interaction","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Yuan et al., 2022] Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics- guided human motion diffusion model. arXiv preprint arXiv:2212.02500, 2022.","title":"Physdiff: Physicsguided human motion diffusion model","authors":"Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhang et al., 2022] Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022.","title":"Motiondiffuse: Text-driven human motion generation with diffusion model","authors":"Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, Ziwei Liu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhou and Wang, 2022] Zixiang Zhou and Baoyuan Wang. Ude: A unified driving engine for human motion genera- tion. arXiv preprint arXiv:2211.16016, 2022.","title":"Ude: A unified driving engine for human motion generation","authors":"Wang ; Zhou, Zixiang Zhou, Baoyuan Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Zhou et al., 2022] Chi Zhou, Tengyue Bian, and Kang Chen. Gesturemaster: Graph-based speech-driven gesture generation. In International Conference on Multimodal Interaction, ICMI, pages 764-770, 2022.","title":"Gesturemaster: Graph-based speech-driven gesture generation","authors":"Zhou","year":"2022","venue":"International Conference on Multimodal Interaction","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}