{"type":"ScholarlyArticle","title":"Zero-shot High-fidelity and Pose-controllable Character Animation","authors":[{"name":"Bingwen Zhu","slug":"bingwen-zhu","given":"Bingwen","family":"Zhu","affiliation":"Shanghai Key Lab of Intell. Info. Processing , School of CS , Fudan University; Shanghai Collaborative Innovation Center of Intelligent Visual Computing"},{"name":"Fanyi Wang","slug":"fanyi-wang","given":"Fanyi","family":"Wang","affiliation":"OPPO AI Center"},{"name":"Tianyi Lu","slug":"tianyi-lu","given":"Tianyi","family":"Lu","affiliation":"Shanghai Key Lab of Intell. Info. Processing , School of CS , Fudan University; Shanghai Collaborative Innovation Center of Intelligent Visual Computing"},{"name":"Peng Liu","slug":"peng-liu","given":"Peng","family":"Liu","affiliation":"OPPO AI Center"},{"name":"Jingwen Su","slug":"jingwen-su","given":"Jingwen","family":"Su","affiliation":"OPPO AI Center"},{"name":"Jinxiu Liu","slug":"jinxiu-liu","given":"Jinxiu","family":"Liu","affiliation":"South China University Of Technology"},{"name":"Yanhao Zhang","slug":"yanhao-zhang","given":"Yanhao","family":"Zhang","affiliation":"OPPO AI Center"},{"name":"Zuxuan Wu","slug":"zuxuan-wu","given":"Zuxuan","family":"Wu","affiliation":"Shanghai Key Lab of Intell. Info. Processing , School of CS , Fudan University; Shanghai Collaborative Innovation Center of Intelligent Visual Computing"},{"name":"Guo-Jun Qi","slug":"guo-jun-qi","given":"Guo-Jun","family":"Qi","affiliation":"Westlake University"},{"name":"Yu-Gang Jiang","slug":"yu-gang-jiang","given":"Yu-Gang","family":"Jiang","affiliation":"Shanghai Key Lab of Intell. Info. Processing , School of CS , Fudan University; Shanghai Collaborative Innovation Center of Intelligent Visual Computing"}],"abstract":"Image-to-video (I2V) generation aims to create a video sequence from a single image, which requires high temporal coherence and visual fidelity. However, existing approaches suffer from inconsistency of character appearances and poor preservation of fine details. Moreover, they require a large amount of video data for training, which can be computationally demanding. To address these limitations, we propose PoseAnimate, a novel zero-shot I2V framework for character animation. PoseAnimate contains three key components: 1) a Pose-Aware Control Module (PACM) that incorporates diverse pose signals into text embeddings, to preserve character-independent content and maintain precise alignment of actions. 2) a Dual Consistency Attention Module (DCAM) that enhances temporal consistency and retains character identity and intricate background details. 3) a Mask-Guided Decoupling Module (MGDM) that refines distinct feature perception abilities, improving animation fidelity by decoupling the character and background. We also propose a Pose Alignment Transition Algorithm (PATA) to ensure smooth action transition. Extensive experiment results demonstrate that our approach outperforms the state-of-the-art training-based methods in terms of character consistency and detail fidelity. Moreover, it maintains a high level of temporal coherence throughout the generated animations.","doi":"10.24963/ijcai.2024/198","url":"https://ijcai.org/proceedings/2024/198","pdf":"https://ijcai.org/proceedings/2024/0198.pdf","markdown":"https://ijcai.org/proceedings/2024/198.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/198","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":198,"pages":{"first":1788,"last":1797},"keywords":["Machine Learning: ML: Multi-modal learning","Computer Vision: CV: Image and video synthesis and generation"],"videos":[],"references":[{"position":1,"raw":"[Balaji et al., 2019] Yogesh Balaji, Martin Renqiang Min, Bing Bai, Rama Chellappa, and Hans Peter Graf. Con- ditional gan with discriminative filter generation for text- to-video synthesis. In IJCAI, volume 1, page 2, 2019.","title":"Conditional gan with discriminative filter generation for textto-video synthesis","authors":"Yogesh Balaji, Martin Renqiang Min, Bing Bai, Rama Chellappa, Hans Peter Graf","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blattmann et al., 2023a] Andreas Blattmann, Tim Dock- horn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023.","title":"Stable video diffusion: Scaling latent video diffusion models to large datasets","authors":"Blattmann","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blattmann et al., 2023b] Andreas Blattmann, Robin Rom- bach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563- 22575, 2023.","title":"Align your latents: High-resolution video synthesis with latent diffusion models","authors":"Blattmann","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chan et al., 2019] Caroline Chan, Shiry Ginosar, Tinghui Zhou, and Alexei A Efros. Everybody dance now. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5933-5942, 2019.","title":"Everybody dance now","authors":"Caroline Chan, Shiry Ginosar, Tinghui Zhou, Alexei A Efros","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2023] Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023.","title":"Videocrafter1: Open diffusion models for high-quality video generation","authors":"Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dhariwal and Nichol, 2021] Prafulla Dhariwal and Alexan- der Nichol. Diffusion models beat gans on image synthe- sis. Advances in neural information processing systems, 34:8780-8794, 2021.","title":"Diffusion models beat gans on image synthesis","authors":"Nichol Dhariwal","year":"2021","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Esser et al., 2023] Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Ger- manidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7346-7356, 2023.","title":"Structure and content-guided video synthesis with diffusion models","authors":"Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, Anastasis Germanidis","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ge et al., 2023] Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji. Pre- serve your own correlation: A noise prior for video dif- fusion models. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 22930- 22941, 2023.","title":"Preserve your own correlation: A noise prior for video diffusion models","authors":"Ge","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. Advances in neural information processing systems, 27, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gu et al., 2023] Jiaxi Gu, Shicong Wang, Haoyu Zhao, Tianyi Lu, Xing Zhang, Zuxuan Wu, Songcen Xu, Wei Zhang, Yu-Gang Jiang, and Hang Xu. Reuse and diffuse: Iterative denoising for text-to-video generation. arXiv preprint arXiv:2309.03549, 2023.","title":"Reuse and diffuse: Iterative denoising for text-to-video generation","authors":"Gu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Güler et al., 2018] Rıza Alp Güler, Natalia Neverova, and Iasonas Kokkinos. Densepose: Dense human pose estima- tion in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7297- 7306, 2018.","title":"Densepose: Dense human pose estimation in the wild","authors":"Güler","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2022] Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity video generation with arbitrary lengths. arXiv preprint arXiv:2211.13221, 2022.","title":"Latent video diffusion models for high-fidelity video generation with arbitrary lengths","authors":"He","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hertz et al., 2022] Amir Hertz, Ron Mokady, Jay Tenen- baum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-prompt image editing with cross-attention con- trol. In The Eleventh International Conference on Learn- ing Representations, 2022.","title":"Prompt-to-prompt image editing with cross-attention control","authors":"Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, Daniel Cohen-Or","year":"2022","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Heusel et al., 2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.","title":"Gans trained by a two time-scale update rule converge to a local nash equilibrium","authors":"Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ho and Salimans, 2022] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.","title":"Classifier-free diffusion guidance","authors":"Jonathan Ho, Tim Salimans","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ho et al., 2022] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022.","title":"Imagen video: High definition video generation with diffusion models","authors":"Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, P Diederik, Ben Kingma, Mohammad Poole, David J Norouzi, Fleet","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hu et al., 2023] Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character anima- tion. arXiv preprint arXiv:2311.17117, 2023.","title":"Animate anyone: Consistent and controllable image-to-video synthesis for character animation","authors":"Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, Liefeng Bo","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jafarian and Park, 2021] Yasamin Jafarian and Hyun Soo Park. Learning high fidelity depths of dressed humans by watching social media dance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 12753-12762, 2021.","title":"Learning high fidelity depths of dressed humans by watching social media dance videos","authors":"Jafarian, Park ;, Yasamin Jafarian, Hyun Soo, Park","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Karras et al., 2019] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401-4410, 2019.","title":"A style-based generator architecture for generative adversarial networks","authors":"Karras","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Karras et al., 2023] Johanna Karras, Aleksander Holyn- ski, Ting-Chun Wang, and Ira Kemelmacher-Shlizerman. Dreampose: Fashion image-to-video synthesis via stable diffusion. arXiv preprint arXiv:2304.06025, 2023.","title":"Dreampose: Fashion image-to-video synthesis via stable diffusion","authors":"Karras","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2023a] Peng Liu, Fanyi Wang, Jingwen Su, Yan- hao Zhang, and Guojun Qi. Lightweight high-resolution subject matting in the real world. arXiv preprint arXiv:2312.07100, 2023.","title":"Lightweight high-resolution subject matting in the real world","authors":"Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2023b] Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tiey- ong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation mod- els. arXiv preprint arXiv:2310.11440, 2023.","title":"Evalcrafter: Benchmarking and evaluating large video generation models","authors":"Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ma et al., 2023] Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Ying Shan, Xiu Li, and Qifeng Chen. Fol- low your pose: Pose-guided text-to-video generation using pose-free videos. arXiv preprint arXiv:2304.01186, 2023.","title":"Follow your pose: Pose-guided text-to-video generation using pose-free videos","authors":"Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Ying Shan, Xiu Li, Qifeng Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mokady et al., 2023] Ron Mokady, Amir Hertz, Kfir Aber- man, Yael Pritch, and Daniel Cohen-Or. Null-text inver- sion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038-6047, 2023.","title":"Null-text inversion for editing real images using guided diffusion models","authors":"Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, Daniel Cohen-Or","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ni et al., 2022] Bolin Ni, Houwen Peng, Minghao Chen, Songyang Zhang, Gaofeng Meng, Jianlong Fu, Shiming Xiang, and Haibin Ling. Expanding language-image pre- trained models for general video recognition. In European Conference on Computer Vision, pages 1-18. Springer, 2022.","title":"Expanding language-image pretrained models for general video recognition","authors":"Bolin Ni, Houwen Peng, Minghao Chen, Songyang Zhang, Gaofeng Meng, Jianlong Fu, Shiming Xiang, Haibin Ling","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nikankin et al., 2022] Yaniv Nikankin, Niv Haim, and Michal Irani. Sinfusion: Training diffusion models on a single image or video. arXiv preprint arXiv:2211.11743, 2022.","title":"Sinfusion: Training diffusion models on a single image or video","authors":"Yaniv Nikankin, Niv Haim, Michal Irani","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International Conference on Machine Learning, pages 8748-8763. PMLR, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ren et al., 2020] Yurui Ren, Ge Li, Shan Liu, and Thomas H Li. Deep spatial transformation for pose-guided person image generation and animation. IEEE Transac- tions on Image Processing, 29:8622-8635, 2020.","title":"Deep spatial transformation for pose-guided person image generation and animation","authors":"Ren","year":"2020","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.","title":"High-resolution image synthesis with latent diffusion models","authors":"Rombach","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234- 241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Siarohin et al., 2019a] Aliaksandr Siarohin, Stéphane Lath- uilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. An- imating arbitrary objects via deep motion transfer. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2377-2386, 2019.","title":"Animating arbitrary objects via deep motion transfer","authors":"Siarohin","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Siarohin et al., 2019b] Aliaksandr Siarohin, Stéphane Lath- uilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. Advances in neural information processing systems, 32, 2019.","title":"First order motion model for image animation","authors":"Siarohin","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Siarohin et al., 2021] Aliaksandr Siarohin, Oliver J Wood- ford, Jian Ren, Menglei Chai, and Sergey Tulyakov. Mo- tion representations for articulated animation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13653-13662, 2021.","title":"Motion representations for articulated animation","authors":"Aliaksandr Siarohin, J Oliver, Jian Woodford, Menglei Ren, Sergey Chai, Tulyakov","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Song et al., 2020] Jiaming Song, Chenlin Meng, and Ste- fano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.","title":"Denoising diffusion implicit models","authors":"Jiaming Song, Chenlin Meng, Stefano Ermon","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Teed and Deng, 2020] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision-ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pages 402-419. Springer, 2020.","title":"Raft: Recurrent all-pairs field transforms for optical flow","authors":"Zachary Teed, Jia Deng","year":"2020","venue":"Computer Vision-ECCV 2020: 16th European Conference","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Unterthiner et al., 2018] Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate genera- tive models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018.","title":"Towards accurate generative models of video: A new metric & challenges","authors":"Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, Sylvain Gelly","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2022] Yaohui Wang, Di Yang, Francois Bre- mond, and Antitza Dantcheva. Latent image animator: Learning to animate images via latent space navigation. arXiv preprint arXiv:2203.09043, 2022.","title":"Latent image animator: Learning to animate images via latent space navigation","authors":"Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wang et al., 2023a] Tan Wang, Linjie Li, Kevin Lin, Yuan- hao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentan- gled control for realistic human dance generation. arXiv preprint arXiv:2307.00040, 2023.","title":"Disco: Disentangled control for realistic human dance generation","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wang et al., 2023b] Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yu- jun Shen, Deli Zhao, and Jingren Zhou. Videocomposer: Compositional video synthesis with motion controllability. arXiv preprint arXiv:2306.02018, 2023.","title":"Videocomposer: Compositional video synthesis with motion controllability","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wu et al., 2023] Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a- video: One-shot tuning of image diffusion models for text- to-video generation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 7623- 7633, 2023.","title":"Tune-avideo: One-shot tuning of image diffusion models for textto-video generation","authors":"Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, Mike Zheng Shou","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Xing et al., 2023a] Zhen Xing, Qi Dai, Han Hu, Zuxuan Wu, and Yu-Gang Jiang. Simda: Simple diffusion adapter for efficient video generation. arXiv preprint arXiv:2308.09710, 2023.","title":"Simda: Simple diffusion adapter for efficient video generation","authors":"Xing","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Xing et al., 2023b] Zhen Xing, Qi Dai, Zihao Zhang, Hui Zhang, Han Hu, Zuxuan Wu, and Yu-Gang Jiang. Vid- iff: Translating videos via multi-modal instructions with diffusion models. arXiv preprint arXiv:2311.18837, 2023.","title":"Vidiff: Translating videos via multi-modal instructions with diffusion models","authors":"Xing","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Xing et al., 2023c] Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang. A survey on video diffusion models. arXiv preprint arXiv:2310.10647, 2023.","title":"A survey on video diffusion models","authors":"Xing","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Xu et al., 2023] Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. arXiv preprint arXiv:2311.16498, 2023.","title":"Magicanimate: Temporally consistent human image animation using diffusion model","authors":"Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, Mike Zheng Shou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Yang et al., 2023] Ruihan Yang, Prakhar Srivastava, and Stephan Mandt. Diffusion probabilistic modeling for video generation. Entropy, 25(10):1469, 2023.","title":"Diffusion probabilistic modeling for video generation","authors":"Yang","year":"2023","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Ye et al., 2023] Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023.","title":"Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models","authors":"Jun Hu Ye, Sibo Zhang, Xiao Liu, Wei Han, Yang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zhang et al., 2018] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586-595, 2018.","title":"The unreasonable effectiveness of deep features as a perceptual metric","authors":"Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, Oliver Wang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhang et al., 2022] Pengze Zhang, Lingxiao Yang, Jian- Huang Lai, and Xiaohua Xie. Exploring dual-task correla- tion for pose guided person image generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7713-7722, 2022.","title":"Exploring dual-task correlation for pose guided person image generation","authors":"Zhang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zhang et al., 2023] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image dif- fusion models. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 3836-3847, 2023.","title":"Adding conditional control to text-to-image diffusion models","authors":"Zhang","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zhao and Zhang, 2022] Jian Zhao and Hui Zhang. Thin- plate spline motion model for image animation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3657-3666, 2022.","title":"Thinplate spline motion model for image animation","authors":"Jian Zhao, Hui Zhang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceed- ings of the IEEE International Conference on Computer Vision, pages 2223-2232, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Zhu","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}