{"type":"ScholarlyArticle","title":"Video Diffusion Models with Local-Global Context Guidance","authors":[{"name":"Siyuan Yang","slug":"siyuan-yang","given":"Siyuan","family":"Yang","affiliation":"Tsinghua University"},{"name":"Lu Zhang","slug":"lu-zhang","given":"Lu","family":"Zhang","affiliation":"Dalian University of Technology"},{"name":"Yu Liu","slug":"yu-liu","given":"Yu","family":"Liu","affiliation":"Tsinghua University"},{"name":"Zhizhuo Jiang","slug":"zhizhuo-jiang","given":"Zhizhuo","family":"Jiang","affiliation":"Tsinghua University"},{"name":"You He","slug":"you-he","given":"You","family":"He","affiliation":"Tsinghua University"}],"abstract":"Diffusion models have emerged as a powerful paradigm in video synthesis tasks including prediction, generation, and interpolation. Due to the limitation of the computational budget, existing methods usually implement conditional diffusion models with an autoregressive inference pipeline, in which the future fragment is predicted based on the distribution of adjacent past frames. However, only the conditions from a few previous frames can't capture the global temporal coherence, leading to inconsistent or even outrageous results in long-term video prediction. In this paper, we propose a Local-Global Context guided Video Diffusion model (LGC-VD) to capture multi-perception conditions for producing high-quality videos in both conditional/unconditional settings. In LGC-VD, the UNet is implemented with stacked residual blocks with self-attention units, avoiding the undesirable computational cost in 3D Conv. We construct a local-global context guidance strategy to capture the multi-perceptual embedding of the past fragment to boost the consistency of future prediction. Furthermore, we propose a two-stage training strategy to alleviate the effect of noisy frames for more stable predictions. Our experiments demonstrate that the proposed method achieves favorable performance on video prediction, interpolation, and unconditional video generation. We release code at https://github.com/exisas/LGC-VD.","doi":"10.24963/ijcai.2023/182","url":"https://ijcai.org/proceedings/2023/182","pdf":"https://ijcai.org/proceedings/2023/0182.pdf","markdown":"https://ijcai.org/proceedings/2023/182.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/182","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":182,"pages":{"first":1640,"last":1648},"keywords":["Computer Vision: CV: Video analysis and understanding","Computer Vision: CV: Neural generative models, auto encoders, GANs"],"videos":[],"references":[{"position":1,"raw":"[Akan et al., 2021] Adil Kaan Akan, Erkut Erdem, Aykut Er- dem, and Fatma Güney. SLAMP: stochastic latent appear- ance and motion prediction. In International Conference on Computer Vision, pages 14708-14717, 2021.","title":"SLAMP: stochastic latent appearance and motion prediction","authors":"Erkut Adil Kaan Akan, Aykut Erdem, Fatma Erdem, Güney","year":"2021","venue":"International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Babaeizadeh et al., 2018] Mohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy H. Campbell, and Sergey Levine. Stochastic variational video prediction. In International Conference on Learning Representations, 2018.","title":"Stochastic variational video prediction","authors":"Mohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy H Campbell, Sergey Levine","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Babaeizadeh et al., 2021] Mohammad Babaeizadeh, Mo- hammad Taghi Saffar, Suraj Nair, Sergey Levine, Chelsea Finn, and Dumitru Erhan. Fitvid: Overfitting in pixel-level video prediction. arXiv preprint arXiv:2106.13195, 2021.","title":"Fitvid: Overfitting in pixel-level video prediction","authors":"Mohammad Babaeizadeh, Mohammad Taghi Saffar, Suraj Nair, Sergey Levine, Chelsea Finn, Dumitru Erhan","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Castrejon et al., 2019a] Lluis Castrejon, Nicolas Ballas, and Aaron Courville. Improved conditional vrnns for video prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7608-7617, 2019.","title":"Improved conditional vrnns for video prediction","authors":"Castrejon","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Castrejón et al., 2019b] Lluís Castrejón, Nicolas Ballas, and Aaron C. Courville. Improved conditional vrnns for video prediction. In International Conference on Com- puter Vision, pages 7607-7616, 2019.","title":"Improved conditional vrnns for video prediction","authors":"Castrejón","year":"2019","venue":"International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Özgün C ¸ic ¸ek, Ahmed Abdulkadir, So- eren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u-net: learning dense volumetric segmentation from sparse annotation. In International conference on medi- cal image computing and computer-assisted intervention, pages 424-432. Springer, 2016.","title":"3d u-net: learning dense volumetric segmentation from sparse annotation","authors":"Ahmed Özgün C ¸ic ¸ek, Abdulkadir, S Soeren, Thomas Lienkamp, Olaf Brox, Ronneberger","year":"2016","venue":"International conference on medical image computing and computer-assisted intervention","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Clark et al., 2019] Aidan Clark, Jeff Donahue, and Karen Simonyan. Adversarial video generation on complex datasets. arXiv preprint arXiv:1907.06571, 2019.","title":"Adversarial video generation on complex datasets","authors":"Aidan Clark, Jeff Donahue, Karen Simonyan","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene under- standing. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 3213-3223, 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Denton and Fergus, 2018a] Emily Denton and Rob Fergus. Stochastic video generation with a learned prior. In Jen- nifer G. Dy and Andreas Krause, editors, International Conference on Machine Learning, volume 80 of Proceed- ings of Machine Learning Research, pages 1182-1191, 2018.","title":"Stochastic video generation with a learned prior","authors":"Fergus ; Denton, Emily Denton, Rob Fergus","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Denton and Fergus, 2018b] Emily Denton and Rob Fergus. Stochastic video generation with a learned prior. In In- ternational conference on machine learning, pages 1174- 1183. PMLR, 2018.","title":"Stochastic video generation with a learned prior","authors":"Fergus ; Denton, Emily Denton, Rob Fergus","year":"2018","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dhariwal and Nichol, 2021] Prafulla Dhariwal and Alexan- der Nichol. Diffusion models beat gans on image synthe- sis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.","title":"Diffusion models beat gans on image synthesis","authors":"Nichol Dhariwal","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ebert et al., 2017] Frederik Ebert, Chelsea Finn, Alex X Lee, and Sergey Levine. Self-supervised visual planning with temporal skip connections. In CoRL, pages 344-356, 2017.","title":"Self-supervised visual planning with temporal skip connections","authors":"Frederik Ebert, Chelsea Finn, Alex X Lee, Sergey Levine","year":"2017","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Franceschi et al., 2020] Jean-Yves Franceschi, Edouard Delasalles, Mickaël Chen, Sylvain Lamprier, and Patrick Gallinari. Stochastic latent residual video prediction. In International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 3233-3246, 2020.","title":"Stochastic latent residual video prediction","authors":"Franceschi","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Harvey et al., 2022] William Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos. arXiv preprint arXiv:2205.11495, 2022.","title":"Flexible diffusion modeling of long videos","authors":"William Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach, Frank Wood","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ho et al., 2022a] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022.","title":"Imagen video: High definition video generation with diffusion models","authors":"Ho","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ho et al., 2022b] Jonathan Ho, Tim Salimans, Alexey Grit- senko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. arXiv preprint arXiv:2204.03458, 2022.","title":"Video diffusion models","authors":"Ho","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Höppe et al., 2022] Tobias Höppe, Arash Mehrjou, Stefan Bauer, Didrik Nielsen, and Andrea Dittadi. Diffusion models for video prediction and infilling. arXiv preprint arXiv:2206.07696, 2022.","title":"Diffusion models for video prediction and infilling","authors":"Höppe","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hu et al., 2020] Anthony Hu, Fergal Cotter, Nikhil Mohan, Corina Gurau, and Alex Kendall. Probabilistic future pre- diction for video scene understanding. In European Con- ference on Computer Vision, pages 767-785. Springer, 2020.","title":"Probabilistic future prediction for video scene understanding","authors":"Anthony Hu, Fergal Cotter, Nikhil Mohan, Corina Gurau, Alex Kendall","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lee et al., 2018] Alex X Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, and Sergey Levine. Stochastic adversarial video prediction. arXiv preprint arXiv:1804.01523, 2018.","title":"Stochastic adversarial video prediction","authors":"Alex X Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, Sergey Levine","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Luc et al., 2020a] Pauline Luc, Aidan Clark, Sander Diele- man, Diego de Las Casas, Yotam Doron, Albin Cassirer, and Karen Simonyan. Transformation-based adversar- ial video prediction on large-scale data. arXiv preprint arXiv:2003.04035, 2020.","title":"Transformation-based adversarial video prediction on large-scale data","authors":"Luc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Luc et al., 2020b] Pauline Luc, Aidan Clark, Sander Diele- man, Diego de Las Casas, Yotam Doron, Albin Cas- sirer, and Karen Simonyan. Transformation-based ad- versarial video prediction on large-scale data. CoRR, abs/2003.04035, 2020.","title":"Transformation-based adversarial video prediction on large-scale data","authors":"Luc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lugmayr et al., 2022] Andreas Lugmayr, Martin Danell- jan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461-11471, 2022.","title":"Repaint: Inpainting using denoising diffusion probabilistic models","authors":"Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, Luc Van Gool","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Moing et al., 2021] Guillaume Le Moing, Jean Ponce, and Cordelia Schmid. CCVS: context-aware controllable video synthesis. In Marc'Aurelio Ranzato, Alina Beygelz- imer, Yann N. Dauphin, Percy Liang, and Jennifer Wort- man Vaughan, editors, Advances in Neural Information Processing Systems, pages 14042-14055, 2021.","title":"CCVS: context-aware controllable video synthesis","authors":"Guillaume Le, Moing, Jean Ponce, Cordelia Schmid","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Muñoz et al., 2021] Andrés Muñoz, Mohammadreza Zolfaghari, Max Argus, and Thomas Brox. Temporal shift GAN for large scale video generation. In IEEE Winter Conference on Applications of Computer Vision, pages 3178-3187, 2021.","title":"Temporal shift GAN for large scale video generation","authors":"Andrés Muñoz, Mohammadreza Zolfaghari, Max Argus, Thomas Brox","year":"2021","venue":"IEEE Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Nichol and Dhariwal, 2021] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion proba- bilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.","title":"Improved denoising diffusion probabilistic models","authors":"Nichol, Dhariwal, Alexander Quinn, Nichol, Prafulla Dhariwal","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nichol et al., 2021] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photore- alistic image generation and editing with text-guided dif- fusion models. arXiv preprint arXiv:2112.10741, 2021.","title":"Glide: Towards photorealistic image generation and editing with text-guided diffusion models","authors":"Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, Mark Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Niklaus et al., 2017] Simon Niklaus, Long Mai, and Feng Liu. Video frame interpolation via adaptive convolution. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, pages 670-679, 2017.","title":"Video frame interpolation via adaptive convolution","authors":"Simon Niklaus, Long Mai, Feng Liu","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rakhimov et al., 2021] Ruslan Rakhimov, Denis Volkhon- skiy, Alexey Artemov, Denis Zorin, and Evgeny Burnaev. Latent video transformer. In International Joint Confer- ence on Computer Vision, Imaging and Computer Graph- ics Theory and Applications, pages 101-112, 2021.","title":"Latent video transformer","authors":"Ruslan Rakhimov, Denis Volkhonskiy, Alexey Artemov, Denis Zorin, Evgeny Burnaev","year":"2021","venue":"International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ramesh et al., 2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.","title":"Hierarchical text-conditional image generation with clip latents","authors":"Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.","title":"High-resolution image synthesis with latent diffusion models","authors":"Rombach","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Saharia et al., 2022] Chitwan","title":"Chitwan","authors":"Saharia","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photoreal- istic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022.","title":"Photorealistic text-to-image diffusion models with deep language understanding","authors":"William Saharia, Saurabh Chan, Lala Saxena, Jay Li, Emily Whang, Seyed Denton, Seyed Kamyar, Burcu Ghasemipour, Sara Karagol Ayan, Rapha Mahdavi, Gontijo Lopes","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Salimans and Ho, 2022] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion mod- els. arXiv preprint arXiv:2202.00512, 2022.","title":"Progressive distillation for fast sampling of diffusion models","authors":"Ho Salimans, Tim Salimans, Jonathan Ho","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Singer et al., 2022] Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a- video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792, 2022.","title":"Make-avideo: Text-to-video generation without text-video data","authors":"Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Song et al., 2020a] Jiaming Song, Chenlin Meng, and Ste- fano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.","title":"Jiaming Song, Chenlin Meng, and Stefano Ermon","authors":"Song","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Song et al., 2020b] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Confer- ence on Learning Representations, 2020.","title":"Score-based generative modeling through stochastic differential equations","authors":"Song","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Tran et al., 2015] Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spa- tiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on com- puter vision, pages 4489-4497, 2015.","title":"Learning spatiotemporal features with 3d convolutional networks","authors":"Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, Manohar Paluri","year":"2015","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Tulyakov et al., 2018] Sergey Tulyakov, Ming-Yu Liu, Xi- aodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. In IEEE Confer- ence on Computer Vision and Pattern Recognition, pages 1526-1535, 2018.","title":"Mocogan: Decomposing motion and content for video generation","authors":"Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, Jan Kautz","year":"2018","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Unterthiner et al., 2018] Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate genera- tive models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018.","title":"Towards accurate generative models of video: A new metric & challenges","authors":"Thomas Unterthiner, Karol Sjoerd Van Steenkiste, Raphael Kurach, Marcin Marinier, Sylvain Michalski, Gelly","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Voleti et al., 2022] Vikram Voleti, Alexia Jolicoeur- Martineau, and Christopher Pal. Mcvd: Masked conditional video diffusion for prediction, generation, and interpolation. arXiv preprint arXiv:2205.09853, 2022.","title":"Mcvd: Masked conditional video diffusion for prediction, generation, and interpolation","authors":"Vikram Voleti, Alexia Jolicoeur-Martineau, Christopher Pal","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Vondrick et al., 2016] Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dy- namics. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, editors, Ad- vances in Neural Information Processing Systems, pages 613-621, 2016.","title":"Generating videos with scene dynamics","authors":"Carl Vondrick, Hamed Pirsiavash, Antonio Torralba","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Weissenborn et al., 2019] Dirk Weissenborn, Oscar Täckström, and Jakob Uszkoreit. Scaling autoregressive video models. arXiv preprint arXiv:1906.02634, 2019.","title":"Scaling autoregressive video models","authors":"Weissenborn","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wu et al., 2021] Bohan Wu, Suraj Nair, Roberto Martin- Martin, Li Fei-Fei, and Chelsea Finn. Greedy hierarchical variational autoencoders for large-scale video prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2318-2328, 2021.","title":"Greedy hierarchical variational autoencoders for large-scale video prediction","authors":"Bohan Wu, Suraj Nair, Roberto Martin-Martin, Li Fei-Fei, Chelsea Finn","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Xu et al., 2020] Qiangeng Xu, Hanwang Zhang, Weiyue Wang, Peter Belhumeur, and Ulrich Neumann. Stochas- tic dynamics for video infilling. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 2714-2723, 2020.","title":"Stochastic dynamics for video infilling","authors":"Xu","year":"2020","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Yan et al., 2021] Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using VQ-VAE and transformers. CoRR, abs/2104.10157, 2021.","title":"Videogpt: Video generation using VQ-VAE and transformers","authors":"Wilson Yan, Yunzhi Zhang, Pieter Abbeel, Aravind Srinivas","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Yang et al., 2022] Ruihan Yang, Prakhar Srivastava, and Stephan Mandt. Diffusion probabilistic modeling for video generation. arXiv preprint arXiv:2203.09481, 2022.","title":"Diffusion probabilistic modeling for video generation","authors":"Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yu et al., 2022] Sihyun Yu, Jihoon Tack, Sangwoo Mo, Hyunsu Kim, Junho Kim, Jung-Woo Ha, and Jinwoo Shin. Generating videos with dynamics-aware implicit genera- tive adversarial networks. In International Conference on Learning Representations, 2022.","title":"Generating videos with dynamics-aware implicit generative adversarial networks","authors":"Yu","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}