{"type":"ScholarlyArticle","title":"Diffusion Models for Non-autoregressive Text Generation: A Survey","authors":[{"name":"Yifan Li","slug":"yifan-li","given":"Yifan","family":"Li","affiliation":"Gaoling School of Artificial Intelligence , Renmin University of China"},{"name":"Kun Zhou","slug":"kun-zhou","given":"Kun","family":"Zhou","affiliation":"School of Information , Renmin University of China; Beijing Key Laboratory of Big Data Management and Analysis Methods"},{"name":"Wayne Xin Zhao","slug":"wayne-xin-zhao","given":"Wayne Xin","family":"Zhao","affiliation":"Gaoling School of Artificial Intelligence , Renmin University of China; Beijing Key Laboratory of Big Data Management and Analysis Methods"},{"name":"Ji-Rong Wen","slug":"ji-rong-wen","given":"Ji-Rong","family":"Wen","affiliation":"Gaoling School of Artificial Intelligence , Renmin University of China; School of Information , Renmin University of China; Beijing Key Laboratory of Big Data Management and Analysis Methods"}],"abstract":"Non-autoregressive (NAR) text generation has attracted much attention in the field of natural language processing, which greatly reduces the inference latency but has to sacrifice the generation accuracy. Recently, diffusion models, a class of latent variable generative models, have been introduced into NAR text generation, showing an improved text generation quality. In this survey, we review the recent progress in diffusion models for NAR text generation. As the background,  we first present the general definition of diffusion models and the text diffusion models, and then discuss their merits for NAR generation. As the core content, we further introduce two mainstream diffusion models in existing work of text diffusion, and review the key designs of the diffusion process. Moreover, we discuss the utilization of pre-trained language models (PLMs) for text diffusion models and introduce optimization techniques for text data. Finally, we discuss several promising directions and conclude this paper. Our survey aims to provide researchers with a systematic reference of related research on text diffusion models for NAR generation. We also demonstrate our collection of text diffusion models at https://github.com/RUCAIBox/Awesome-Text-Diffusion-Models.","doi":"10.24963/ijcai.2023/750","url":"https://ijcai.org/proceedings/2023/750","pdf":"https://ijcai.org/proceedings/2023/0750.pdf","markdown":"https://ijcai.org/proceedings/2023/750.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/750","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Survey Track","track":"Survey Track","number":750,"pages":{"first":6692,"last":6701},"keywords":["Survey: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Austin et al., 2021] Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state- spaces. In NeurIPS, pages 17981-17993, 2021.","title":"Structured denoising diffusion models in discrete statespaces","authors":"Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, Rianne Van Den, Berg","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2022] Ting Chen, Ruixiang Zhang, and Geof- frey E. Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning. CoRR, abs/2208.04202, 2022.","title":"Analog bits: Generating discrete data using diffusion models with self-conditioning","authors":"Ting Chen, Ruixiang Zhang, Geoffrey E Hinton","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, C ¸aglar Gülc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, pages 1724-1734. ACL, 2014.","title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cho et al., 2022] Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers. CoRR, abs/2202.04053, 2022.","title":"Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers","authors":"Jaemin Cho, Abhay Zala, Mohit Bansal","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dathathri et al., 2020] Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. In ICLR. OpenReview.net, 2020.","title":"Plug and play language models: A simple approach to controlled text generation","authors":"Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, Rosanne Liu","year":"2020","venue":"ICLR. OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT (1), pages 4171-4186. Association for Computational Linguistics, 2019.","title":"BERT: pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL-HLT (1)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dhariwal and Nichol, 2021] Prafulla Dhariwal and Alexan- der Quinn Nichol. Diffusion models beat gans on image synthesis. In NeurIPS, pages 8780-8794, 2021.","title":"Diffusion models beat gans on image synthesis","authors":"Nichol ; Dhariwal, Alexander Quinn, Nichol","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dieleman et al., 2022] Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl, and Jonas Adler. Continuous diffusion for categorical data. CoRR, abs/2211.15089, 2022.","title":"Continuous diffusion for categorical data","authors":"Laurent Sander Dieleman, Arman Sartran, Nikolay Roshannai, Yaroslav Savinov, Pierre H Ganin, Arnaud Richemond, Robin Doucet, Chris Strudel, Conor Dyer, Curtis Durkan, Rémi Hawthorne, Will Leblond, Jonas Grathwohl, Adler","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gao et al., 2022] Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, and Linli Xu. Dif- former: Empowering diffusion model on embedding space for text generation. CoRR, abs/2212.09412, 2022.","title":"Difformer: Empowering diffusion model on embedding space for text generation","authors":"Gao","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gatt and Krahmer, 2018] Albert Gatt and Emiel Krahmer. Survey of the state of the art in natural language gener- ation: Core tasks, applications and evaluation. J. Artif. Intell. Res., 61:65-170, 2018.","title":"Survey of the state of the art in natural language generation: Core tasks, applications and evaluation","authors":"Krahmer Gatt, Albert Gatt, Emiel Krahmer","year":"2018","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gehman et al., 2020] Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. Realtoxi- cityprompts: Evaluating neural toxic degeneration in lan- guage models. In EMNLP (Findings), volume EMNLP 2020 of Findings of ACL, pages 3356-3369. Association for Computational Linguistics, 2020.","title":"Realtoxicityprompts: Evaluating neural toxic degeneration in language models","authors":"Gehman","year":"2020","venue":"EMNLP 2020 of Findings of ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ghazvininejad et al., 2019] Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. Mask-predict: Parallel decoding of conditional masked language models. In EMNLP/IJCNLP (1), pages 6111-6120. Association for Computational Linguistics, 2019.","title":"Mask-predict: Parallel decoding of conditional masked language models","authors":"Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer","year":"2019","venue":"EMNLP/IJCNLP (1)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gong et al., 2022] Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. CoRR, abs/2210.08933, 2022.","title":"Diffuseq: Sequence to sequence text generation with diffusion models","authors":"Gong","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gu et al., 2018] Jiatao Gu, James Bradbury, Caiming Xiong, Victor O. K. Li, and Richard Socher. Non- autoregressive neural machine translation. In ICLR (Poster). OpenReview.net, 2018.","title":"Nonautoregressive neural machine translation","authors":"Gu","year":"2018","venue":"ICLR (Poster)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gu et al., 2019] Jiatao Gu, Changhan Wang, and Junbo Zhao. Levenshtein transformer. In NeurIPS, pages 11179- 11189, 2019.","title":"Levenshtein transformer","authors":"Gu","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Han et al., 2022] Xiaochuang Han, Sachin Kumar, and Yu- lia Tsvetkov. SSD-LM: semi-autoregressive simplex- based diffusion language model for text generation and modular control. CoRR, abs/2210.17432, 2022.","title":"SSD-LM: semi-autoregressive simplexbased diffusion language model for text generation and modular control","authors":"Han","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[He et al., 2022] Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. Diffusionbert: Improving generative masked language models with diffu- sion models. CoRR, abs/2211.15029, 2022.","title":"Diffusionbert: Improving generative masked language models with diffusion models","authors":"He","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ho and Salimans, 2021] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Work- shop on Deep Generative Models and Downstream Appli- cations, 2021.","title":"Classifier-free diffusion guidance","authors":"Jonathan Ho, Tim Salimans","year":"2021","venue":"NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Hoogeboom et al., 2021] Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning cat- egorical distributions. In NeurIPS, pages 12454-12465, 2021.","title":"Argmax flows and multinomial diffusion: Learning categorical distributions","authors":"Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, Max Welling","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Hu et al., 2022] Minghui Hu, Chuanxia Zheng, Heliang Zheng, Tat-Jen Cham, Chaoyue Wang, Zuopeng Yang, Dacheng Tao, and Ponnuthurai N. Suganthan. Unified dis- crete diffusion for simultaneous vision-language genera- tion. CoRR, abs/2211.14842, 2022.","title":"Suganthan. Unified discrete diffusion for simultaneous vision-language generation","authors":"Hu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kasai et al., 2020] Jungo Kasai, James Cross, Marjan Ghazvininejad, and Jiatao Gu. Non-autoregressive ma- chine translation with disentangled context transformer. In International conference on machine learning, pages 5144-5155. PMLR, 2020.","title":"Non-autoregressive machine translation with disentangled context transformer","authors":"Jungo Kasai, James Cross, Marjan Ghazvininejad, Jiatao Gu","year":"2020","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kong et al., 2021] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In ICLR. OpenRe- view.net, 2021.","title":"Diffwave: A versatile diffusion model for audio synthesis","authors":"Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, Bryan Catanzaro","year":"2021","venue":"ICLR. OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lewis et al., 2020] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In ACL, pages 7871-7880. Association for Computational Linguistics, 2020.","title":"BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Li et al., 2021] Junyi Li, Tianyi Tang, Wayne Xin Zhao, and Ji-Rong Wen. Pretrained language models for text genera- tion: A survey. CoRR, abs/2105.10311, 2021.","title":"Pretrained language models for text generation: A survey","authors":"Junyi Li, Tianyi Tang, Wayne Xin Zhao, Ji-Rong Wen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Li et al., 2022] Xiang Lisa Li, John Thickstun, Ishaan Gul- rajani, Percy Liang, and Tatsunori Hashimoto. Diffusion- lm improves controllable text generation. In Advances in Neural Information Processing Systems, 2022.","title":"Diffusionlm improves controllable text generation","authors":"Lisa Xiang, John Li, Ishaan Thickstun, Percy Gulrajani, Tatsunori Liang, Hashimoto","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Li et al., 2023] Junyi Li, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Glyphdiffusion: Text generation as im- age generation. arXiv preprint arXiv:2304.12519, 2023.","title":"Glyphdiffusion: Text generation as image generation","authors":"Junyi Li, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lin et al., 2022] Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Weizhu Chen, and Nan Duan. GENIE: large scale pre-training for text generation with diffusion model. CoRR, abs/2212.11685, 2022.","title":"GENIE: large scale pre-training for text generation with diffusion model","authors":"Lin","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692, 2019.","title":"Roberta: A robustly optimized BERT pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Liu et al., 2021] Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. Dexperts: Decoding-time con- trolled text generation with experts and anti-experts. In ACL/IJCNLP (1), pages 6691-6706. Association for Com- putational Linguistics, 2021.","title":"Dexperts: Decoding-time controlled text generation with experts and anti-experts","authors":"Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A Smith, Yejin Choi","year":"2021","venue":"ACL/IJCNLP (1)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Liu et al., 2022] Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiaodan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, and Zhiting Hu. Composable text control operations in latent space with ordinary differential equations. CoRR, abs/2208.00638, 2022.","title":"Composable text control operations in latent space with ordinary differential equations","authors":"Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiaodan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, Zhiting Hu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Lovelace et al., 2022] Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman, and Kilian Q. Wein- berger. Latent diffusion for language generation. CoRR, abs/2212.09462, 2022.","title":"Latent diffusion for language generation","authors":"Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman, Kilian Q Weinberger","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Luo et al., 2022] Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Jianlin Feng, Hongyang Chao, and Tao Mei. Semantic-conditional diffusion networks for image cap- tioning. CoRR, abs/2212.03099, 2022.","title":"Semantic-conditional diffusion networks for image captioning","authors":"Luo","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Nallapati et al., 2017] Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. Summarunner: A recurrent neural net- work based sequence model for extractive summarization of documents. In AAAI, pages 3075-3081. AAAI Press, 2017.","title":"Summarunner: A recurrent neural network based sequence model for extractive summarization of documents","authors":"Ramesh Nallapati, Feifei Zhai, Bowen Zhou","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Nichol and Dhariwal, 2021] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion prob- abilistic models. In ICML, volume 139 of Proceedings of Machine Learning Research, pages 8162-8171. PMLR, 2021.","title":"Improved denoising diffusion probabilistic models","authors":"Nichol, Dhariwal, Alexander Quinn, Nichol, Prafulla Dhariwal","year":"2021","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Qi et al., 2021] Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Weizhu Chen, Dayiheng Liu, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, and Nan Duan. BANG: bridging autoregressive and non- autoregressive generation with large scale pretraining. In ICML, volume 139 of Proceedings of Machine Learning Research, pages 8630-8639. PMLR, 2021.","title":"BANG: bridging autoregressive and nonautoregressive generation with large scale pretraining","authors":"Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Weizhu Chen, Dayiheng Liu, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan","year":"2021","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Qian et al., 2022] Lihua Qian, Mingxuan Wang, Yang Liu, and Hao Zhou. Diff-glat: Diffusion glancing trans- former for parallel sequence to sequence learning. CoRR, abs/2212.10240, 2022.","title":"Diff-glat: Diffusion glancing transformer for parallel sequence to sequence learning","authors":"Lihua Qian, Mingxuan Wang, Yang Liu, Hao Zhou","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Ramesh et al., 2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchi- cal text-conditional image generation with CLIP latents. CoRR, abs/2204.06125, 2022.","title":"Hierarchical text-conditional image generation with CLIP latents","authors":"Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, pages 10674-10685. IEEE, 2022.","title":"High-resolution image synthesis with latent diffusion models","authors":"Rombach","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI (3), vol- ume 9351 of Lecture Notes in Computer Science, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"MICCAI (3)","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Sohl-Dickstein et al., 2015] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermody- namics. In ICML, volume 37 of JMLR Workshop and Con- ference Proceedings, pages 2256-2265. JMLR.org, 2015.","title":"Deep unsupervised learning using nonequilibrium thermodynamics","authors":"Jascha Sohl-Dickstein, Eric A Weiss, Niru Maheswaranathan, Surya Ganguli","year":"2015","venue":"JMLR Workshop and Conference Proceedings","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Song et al., 2021] Jiaming Song, Chenlin Meng, and Ste- fano Ermon. Denoising diffusion implicit models. In ICLR. OpenReview.net, 2021.","title":"Denoising diffusion implicit models","authors":"Jiaming Song, Chenlin Meng, Stefano Ermon","year":"2021","venue":"ICLR. OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Strudel et al., 2022] Robin Strudel, Corentin Tallec, Florent Altché, Yilun Du, Yaroslav Ganin, Arthur Mensch, Will Grathwohl, Nikolay Savinov, Sander Dieleman, Laurent Sifre, and Rémi Leblond. Self-conditioned embedding dif- fusion for text generation. CoRR, abs/2211.04236, 2022.","title":"Self-conditioned embedding diffusion for text generation","authors":"Strudel","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Xu et al., 2022] Xingqian Xu, Zhangyang Wang, Eric Zhang, Kai Wang, and Humphrey Shi. Versatile diffusion: Text, images and variations all in one diffusion model. CoRR, abs/2211.08332, 2022.","title":"Versatile diffusion: Text, images and variations all in one diffusion model","authors":"Xu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Ye et al., 2023] Jiasheng Ye, Zaixiang Zheng, Yu Bao, Li- hua Qian, and Mingxuan Wang. DINOISER: diffused con- ditional sequence learning by manipulating noises. CoRR, abs/2302.10025, 2023.","title":"DINOISER: diffused conditional sequence learning by manipulating noises","authors":"Ye","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Yuan et al., 2022] Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Fei Huang, and Songfang Huang. Seqdiffuseq: Text diffusion with encoder-decoder transformers. CoRR, abs/2212.10325, 2022.","title":"Seqdiffuseq: Text diffusion with encoder-decoder transformers","authors":"Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Fei Huang, Songfang Huang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zhao et al., 2023] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.","title":"A survey of large language models","authors":"Kun Wayne Xin Zhao, Junyi Zhou, Tianyi Li, Xiaolei Tang, Yupeng Wang, Yingqian Hou, Beichen Min, Junjie Zhang, Zican Zhang, Yifan Dong, Chen Du, Yushuo Yang, Zhipeng Chen, Jinhao Chen, Ruiyang Jiang, Yifan Ren, Xinyu Li, Zikang Tang, Peiyu Liu, Jian-Yun Liu, Ji-Rong Nie, Wen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zheng et al., 2023] Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. CoRR, abs/2302.05737, 2023.","title":"A reparameterized discrete diffusion model for text generation","authors":"Lin Zheng, Jianbo Yuan, Lei Yu, Lingpeng Kong","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhou et al., 2020] Chunting Zhou, Jiatao Gu, and Graham Neubig. Understanding knowledge distillation in non- autoregressive machine translation. In ICLR. OpenRe- view.net, 2020.","title":"Understanding knowledge distillation in nonautoregressive machine translation","authors":"Zhou","year":"2020","venue":"ICLR. OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhou et al., 2023] Kun Zhou, Yifan Li, Wayne Xin Zhao, and Ji-Rong Wen. Diffusion-nat: Self-prompting discrete diffusion for non-autoregressive text generation. arXiv preprint arXiv:2305.04044, 2023.","title":"Diffusion-nat: Self-prompting discrete diffusion for non-autoregressive text generation","authors":"Kun Zhou, Yifan Li, Wayne Xin Zhao, Ji-Rong Wen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}