{"type":"ScholarlyArticle","title":"FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generation","authors":[{"name":"Jianyi Chen","slug":"jianyi-chen","given":"Jianyi","family":"Chen","affiliation":"The Hong Kong University of Science and Technology"},{"name":"Wei Xue","slug":"wei-xue","given":"Wei","family":"Xue","affiliation":"The Hong Kong University of Science and Technology"},{"name":"Xu Tan","slug":"xu-tan","given":"Xu","family":"Tan","affiliation":"Microsoft Research Asia"},{"name":"Zhen Ye","slug":"zhen-ye","given":"Zhen","family":"Ye","affiliation":"The Hong Kong University of Science and Technology"},{"name":"Qifeng Liu","slug":"qifeng-liu","given":"Qifeng","family":"Liu","affiliation":"The Hong Kong University of Science and Technology"},{"name":"Yike Guo","slug":"yike-guo","given":"Yike","family":"Guo","affiliation":"The Hong Kong University of Science and Technology"}],"abstract":"Singing Accompaniment Generation (SAG), which generates instrumental music to accompany input vocals, is crucial to developing human-AI symbiotic art creation systems. The state-of-the-art method, SingSong, utilizes a multi-stage autoregressive (AR) model for SAG, however, this method is extremely slow as it generates semantic and acoustic tokens recursively, and this makes it impossible for real-time applications. In this paper, we aim to develop a Fast SAG method that can create high-quality and coherent accompaniments. A non-AR diffusion-based framework is developed, which by carefully designing the conditions inferred from the vocal signals, generates the Mel spectrogram of the target accompaniment directly. With diffusion and Mel spectrogram modeling, the proposed method significantly simplifies the AR token-based SingSong framework, and largely accelerates the generation. We also design semantic projection, prior projection blocks as well as a set of loss functions, to ensure the generated accompaniment has semantic and rhythm coherence with the vocal signal. By intensive experimental studies, we demonstrate that the proposed method can generate better samples than SingSong, and accelerate the generation by at least 30 times. Audio samples and code are available at this link.","doi":"10.24963/ijcai.2024/843","url":"https://ijcai.org/proceedings/2024/843","pdf":"https://ijcai.org/proceedings/2024/0843.pdf","markdown":"https://ijcai.org/proceedings/2024/843.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/843","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI, Arts & Creativity","track":"AI, Arts & Creativity","number":843,"pages":{"first":7618,"last":7626},"keywords":["Application domains: Music and sound","Methods and resources: AI systems for collaboration and co-creation"],"videos":[],"references":[{"position":1,"raw":"[Agostinelli et al., 2023] Andrea Agostinelli, Timo I Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Cail- lon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325, 2023.","title":"Musiclm: Generating music from text","authors":"Andrea Agostinelli, I Timo, Zalán Denk, Jesse Borsos, Mauro Engel, Antoine Verzetti, Qingqing Caillon, Aren Huang, Adam Jansen, Marco Roberts, Tagliasacchi","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Borsos et al., 2023] Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al. Audiolm: a language modeling approach to audio generation. IEEE/ACM Trans. Audio, Speech, Lang. Process., 2023.","title":"Audiolm: a language modeling approach to audio generation","authors":"Borsos","year":"2023","venue":"IEEE/ACM Trans. Audio, Speech","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chung et al., 2021] Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, and Yonghui Wu. W2v-bert: Combining contrastive learn- ing and masked language modeling for self-supervised speech pre-training. In Proc. IEEE Workshop on Au- tomatic Speech Recognition and Understanding (ASRU), 2021.","title":"W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training","authors":"Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, Yonghui Wu","year":"2021","venue":"Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Copet et al., 2023] Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. Simple and controllable music gen- eration. arXiv preprint arXiv:2306.05284, 2023.","title":"Simple and controllable music generation","authors":"Copet","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dai et al., 2019] Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdi- nov. Transformer-xl: Attentive language models beyond a fixed-length context. In Proc. Assoc. for Computational Linguistics (ACL), 2019.","title":"Transformer-xl: Attentive language models beyond a fixed-length context","authors":"Dai","year":"2019","venue":"Proc. Assoc. for Computational Linguistics (ACL)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Défossez et al., 2019] Alexandre Défossez, Nicolas Usunier, Léon Bottou, and Francis Bach. Demucs: Deep extractor for music sources with extra unlabeled data remixed. arXiv preprint arXiv:1909.01174, 2019.","title":"Demucs: Deep extractor for music sources with extra unlabeled data remixed","authors":"Alexandre Défossez, Nicolas Usunier, Léon Bottou, Francis Bach","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Défossez et al., 2022] Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural au- dio compression. arXiv preprint arXiv:2210.13438, 2022.","title":"High fidelity neural audio compression","authors":"Alexandre Défossez, Jade Copet, Gabriel Synnaeve, Yossi Adi","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ding and Cui, 2023] Fanyu Ding and Yidong Cui. Muse- flow: music accompaniment generation based on flow. Ap- plied Intelligence, 2023.","title":"Museflow: music accompaniment generation based on flow","authors":"Cui Ding, Fanyu Ding, Yidong Cui","year":"2023","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Donahue et al., 2023] Chris Donahue, Antoine Caillon, Adam Roberts, Ethan Manilow, Philippe Esling, An- drea Agostinelli, Mauro Verzetti, Ian Simon, Olivier Pietquin, Neil Zeghidour, et al. Singsong: Generating musical accompaniments from singing. arXiv preprint arXiv:2301.12662, 2023.","title":"Singsong: Generating musical accompaniments from singing","authors":"Chris Donahue, Antoine Caillon, Adam Roberts, Ethan Manilow, Philippe Esling, Andrea Agostinelli, Mauro Verzetti, Ian Simon, Olivier Pietquin, Neil Zeghidour","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gui et al., 2023] Azalea Gui, Hannes Gamper, Sebastian Braun, and Dimitra Emmanouilidou. Adapting frechet audio distance for generative music evaluation. arXiv preprint arXiv:2311.01616, 2023.","title":"Adapting frechet audio distance for generative music evaluation","authors":"Gui","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hershey et al., 2017] Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Chan- ning Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. Cnn architectures for large-scale audio classification. In Proc. IEEE Intl. Conf. Acoustics, Speech, Signal Process. (ICASSP), 2017.","title":"Cnn architectures for large-scale audio classification","authors":"Shawn Hershey, Sourish Chaudhuri, P W Daniel, Ellis, Aren Jort F Gemmeke, R Channing Jansen, Manoj Moore, Devin Plakal, Rif A Platt, Bryan Saurous, Seybold","year":"2017","venue":"Proc. IEEE Intl. Conf. Acoustics, Speech, Signal Process","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2023a] Qingqing Huang, Daniel S Park, Tao Wang, Timo I Denk, Andy Ly, Nanxin Chen, Zhengdong Zhang, Zhishuai Zhang, Jiahui Yu, Christian Frank, et al. Noise2music: Text-conditioned music generation with dif- fusion models. arXiv preprint arXiv:2302.03917, 2023.","title":"Noise2music: Text-conditioned music generation with diffusion models","authors":"Huang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2023b] Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, and Zhou Zhao. Make-an- audio: Text-to-audio generation with prompt-enhanced diffusion models. In Proc. Intl. Conf. Machine Learning (ICML), 2023.","title":"Make-anaudio: Text-to-audio generation with prompt-enhanced diffusion models","authors":"Huang","year":"2023","venue":"Proc. Intl. Conf. Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jaegle et al., 2022] Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. In Proc. Intl. Conf. Machine Learning (ICML), 2022.","title":"Perceiver io: A general architecture for structured inputs & outputs","authors":"Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer","year":"2022","venue":"Proc. Intl. Conf. Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Karras et al., 2022] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Proc. Conf. Neural Information Processing Systems (NeurIPS), 2022.","title":"Elucidating the design space of diffusion-based generative models","authors":"Tero Karras, Miika Aittala, Timo Aila, Samuli Laine","year":"2022","venue":"Proc. Conf. Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kong et al., 2020] Jungil Kong, Jaehyeon Kim, and Jaeky- oung Bae. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. In Proc. Conf. Neural Information Processing Systems (NeurIPS), 2020.","title":"Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis","authors":"Kong","year":"2020","venue":"Proc. Conf. Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kreuk et al., 2022] Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. Audiogen: Textually guided audio generation. In Proc. Intl. Conf. Machine Learning (ICML), 2022.","title":"Audiogen: Textually guided audio generation","authors":"Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, Yossi Adi","year":"2022","venue":"Proc. Intl. Conf. Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kumar et al., 2023] Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. High-fidelity audio compression with improved rvqgan. In Proc. Conf. Neural Information Processing Systems (NeurIPS), 2023.","title":"High-fidelity audio compression with improved rvqgan","authors":"Kumar","year":"2023","venue":"Proc. Conf. Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lee et al., 2023] Sang-gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, and Sungroh Yoon. Bigvgan: A univer- sal neural vocoder with large-scale training. In Proc. Intl. Conf. Machine Learning (ICML), 2023.","title":"Bigvgan: A universal neural vocoder with large-scale training","authors":"Lee","year":"2023","venue":"Proc. Intl. Conf. Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2023] Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, et al. Mert: Acoustic music understanding model with large-scale self- supervised training. arXiv preprint arXiv:2306.00107, 2023.","title":"Mert: Acoustic music understanding model with large-scale selfsupervised training","authors":"Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2022] Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, and Zhou Zhao. Diffsinger: Singing voice synthesis via shallow diffusion mechanism. In Proc. AAAI Conf. Artif. Intell. (AAAI), 2022.","title":"Diffsinger: Singing voice synthesis via shallow diffusion mechanism","authors":"Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, Zhou Zhao","year":"2022","venue":"Proc. AAAI Conf. Artif. Intell. (AAAI)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2023] Haohe Liu, Zehua Chen, Yi Yuan, Xin- hao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models. In Proc. Intl. Conf. Machine Learning (ICML), 2023.","title":"Audioldm: Text-to-audio generation with latent diffusion models","authors":"Liu","year":"2023","venue":"Proc. Intl. Conf. Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mariani et al., 2023] Giorgio Mariani, Irene Tallini, Emil- ian Postolache, Michele Mancusi, Luca Cosmo, and Emanuele Rodolà. Multi-source diffusion models for simultaneous music generation and separation. arXiv preprint arXiv:2302.02257, 2023.","title":"Multi-source diffusion models for simultaneous music generation and separation","authors":"Giorgio Mariani, Irene Tallini, Emilian Postolache, Michele Mancusi, Luca Cosmo, Emanuele Rodolà","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Oord et al., 2016] Aaron van den Oord, Sander Diele- man, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw au- dio. In ISCA Speech Synthesis Workshop, 2016.","title":"Wavenet: A generative model for raw audio","authors":"Oord","year":"2016","venue":"ISCA Speech Synthesis Workshop","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Popov et al., 2021] Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad-tts: A diffusion probabilistic model for text-to- speech. In Proc. Intl. Conf. Machine Learning (ICML), pages 8599-8608. PMLR, 2021.","title":"Grad-tts: A diffusion probabilistic model for text-tospeech","authors":"Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, Mikhail Kudinov","year":"2021","venue":"Proc. Intl. Conf. Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Prenger et al., 2019] Ryan Prenger, Rafael Valle, and Bryan Catanzaro. Waveglow: A flow-based generative network for speech synthesis. In Proc. IEEE Intl. Conf. Acoustics, Speech, Signal Process. (ICASSP), 2019.","title":"Waveglow: A flow-based generative network for speech synthesis","authors":"Ryan Prenger, Rafael Valle, Bryan Catanzaro","year":"2019","venue":"Proc. IEEE Intl. Conf. Acoustics, Speech, Signal Process","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rafii et al., 2017] Zafar Rafii, Antoine Liutkus, Fabian- Robert Stöter, Stylianos Ioannis Mimilakis, and Rachel Bittner. Musdb18-a corpus for music separation. 2017.","title":"Musdb18-a corpus for music separation","authors":"Zafar Rafii, Antoine Liutkus, Fabian-Robert Stöter, Stylianos Ioannis Mimilakis, Rachel Bittner","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ren et al., 2019] Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. Fastspeech: Fast, robust and controllable text to speech. In Proc. Conf. Neural Information Processing Systems (NeurIPS), vol- ume 32, 2019.","title":"Fastspeech: Fast, robust and controllable text to speech","authors":"Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, Tie-Yan Liu","year":"2019","venue":"Proc. Conf. Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ren et al., 2020] Yi Ren, Jinzheng He, Xu Tan, Tao Qin, Zhou Zhao, and Tie-Yan Liu. Popmag: Pop music ac- companiment generation. In Proc. ACM Multimedia (ACM MM), 2020.","title":"Popmag: Pop music accompaniment generation","authors":"Yi Ren, Jinzheng He, Xu Tan, Tao Qin, Zhou Zhao, Tie-Yan Liu","year":"2020","venue":"Proc. ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Proc. Conf. on Medical Image Computing and Computer Assisted Inter- vention (MICCAI), 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"Proc. Conf. on Medical Image Computing and Computer Assisted Intervention (MICCAI)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Schneider et al., 2023] Flavio Schneider, Zhijing Jin, and Bernhard Schölkopf. Mo\\ˆusai: Text-to-music gener- ation with long-context latent diffusion. arXiv preprint arXiv:2301.11757, 2023.","title":"Mo\\ˆusai: Text-to-music generation with long-context latent diffusion","authors":"Flavio Schneider, Zhijing Jin, Bernhard Schölkopf","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Shen et al., 2023] Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. arXiv preprint arXiv:2304.09116, 2023.","title":"Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers","authors":"Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, Jiang Bian","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Simon et al., 2008] Ian Simon, Dan Morris, and Sumit Basu. Mysong: automatic accompaniment generation for vocal melodies. In Proc. Conf. on Human Factors in Com- puting Systems (CHI), pages 725-734, 2008.","title":"Mysong: automatic accompaniment generation for vocal melodies","authors":"Ian Simon, Dan Morris, Sumit Basu","year":"2008","venue":"Proc. Conf. on Human Factors in Computing Systems (CHI)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Siuzdak, 2023] Hubert Siuzdak. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. arXiv preprint arXiv:2306.00814, 2023.","title":"Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis","authors":"Hubert Siuzdak, Siuzdak","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Song et al., 2020] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proc. Intl. Conf. Learning Representations (ICLR), 2020.","title":"Score-based generative modeling through stochastic differential equations","authors":"Yang Song, Jascha Sohl-Dickstein, P Diederik, Abhishek Kingma, Stefano Kumar, Ben Ermon, Poole","year":"2020","venue":"Proc. Intl. Conf. Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wang et al., 2022] Zihao Wang, Kejun Zhang, Yuxing Wang, Chen Zhang, Qihao Liang, Pengfei Yu, Yongsheng Feng, Wenbo Liu, Yikai Wang, Yuntao Bao, et al. Song- driver: Real-time music accompaniment generation with- out logical latency nor exposure bias. In Proc. ACM Mul- timedia (ACM MM), 2022.","title":"Songdriver: Real-time music accompaniment generation without logical latency nor exposure bias","authors":"Wang","year":"2022","venue":"Proc. ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wang et al., 2023] Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yan- qing Liu, Huaming Wang, Jinyu Li, et al. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111, 2023.","title":"Neural codec language models are zero-shot text to speech synthesizers","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wu et al., 2022] Yueh-Kao Wu, Ching-Yu Chiu, and Yi- Hsuan Yang. Jukedrummer: conditional beat-aware audio- domain drum accompaniment generation via transformer vq-va. In Proc. Intl. Soc. for Music Information Retrieval Conf. (ISMIR), 2022.","title":"Jukedrummer: conditional beat-aware audiodomain drum accompaniment generation via transformer vq-va","authors":"Yueh-Kao Wu, Ching-Yu Chiu, Yi-Hsuan Yang","year":"2022","venue":"Proc. Intl. Soc. for Music Information Retrieval Conf. (ISMIR)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wu et al., 2023] Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dub- nov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In Proc. IEEE Intl. Conf. Acoustics, Speech, Signal Pro- cess. (ICASSP), 2023.","title":"Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation","authors":"Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, Shlomo Dubnov","year":"2023","venue":"Proc. IEEE Intl. Conf. Acoustics, Speech, Signal Process","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Ye et al., 2023] Zhen Ye, Wei Xue, Xu Tan, Jie Chen, Qifeng Liu, and Yike Guo. Comospeech: One-step speech and singing voice synthesis via consistency model. In Proc. ACM Multimedia (ACM MM), 2023.","title":"Comospeech: One-step speech and singing voice synthesis via consistency model","authors":"Zhen Ye, Wei Xue, Xu Tan, Jie Chen, Qifeng Liu, Yike Guo","year":"2023","venue":"Proc. ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zeghidour et al., 2021] Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasac- chi. Soundstream: An end-to-end neural audio codec. IEEE/ACM Trans. Audio, Speech, Lang. Process., 30:495- 507, 2021.","title":"Soundstream: An end-to-end neural audio codec","authors":"Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, Marco Tagliasacchi","year":"2021","venue":"IEEE/ACM Trans. Audio, Speech, Lang. Process","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}