{"type":"ScholarlyArticle","title":"Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretraining","authors":[{"name":"Takaaki Saeki","slug":"takaaki-saeki","given":"Takaaki","family":"Saeki","affiliation":"The University of Tokyo , Japan"},{"name":"Soumi Maiti","slug":"soumi-maiti","given":"Soumi","family":"Maiti","affiliation":"Carnegie Mellon University , USA"},{"name":"Xinjian Li","slug":"xinjian-li","given":"Xinjian","family":"Li","affiliation":"Carnegie Mellon University , USA"},{"name":"Shinji Watanabe","slug":"shinji-watanabe","given":"Shinji","family":"Watanabe","affiliation":"Carnegie Mellon University , USA"},{"name":"Shinnosuke Takamichi","slug":"shinnosuke-takamichi","given":"Shinnosuke","family":"Takamichi","affiliation":"The University of Tokyo , Japan"},{"name":"Hiroshi Saruwatari","slug":"hiroshi-saruwatari","given":"Hiroshi","family":"Saruwatari","affiliation":"The University of Tokyo , Japan"}],"abstract":"While neural text-to-speech (TTS) has achieved human-like natural synthetic speech, multilingual TTS systems are limited to resource-rich languages due to the need for paired text and studio-quality audio data. This paper proposes a method for zero-shot multilingual TTS using text-only data for the target language. The use of text-only data allows the development of TTS systems for low-resource languages for which only textual resources are available, making TTS accessible to thousands of languages. Inspired by the strong cross-lingual transferability of multilingual language models, our framework first performs masked language model pretraining with multilingual text-only data. Then we train this model with a paired data in a supervised manner, while freezing a language-aware embedding layer. This allows inference even for languages not included in the paired data but present in the text-only data. Evaluation results demonstrate highly intelligible zero-shot TTS with a character error rate of less than 12% for an unseen language.","doi":"10.24963/ijcai.2023/575","url":"https://ijcai.org/proceedings/2023/575","pdf":"https://ijcai.org/proceedings/2023/0575.pdf","markdown":"https://ijcai.org/proceedings/2023/575.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/575","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Natural Language Processing","track":"Main Track","number":575,"pages":{"first":5179,"last":5187},"keywords":["Natural Language Processing: NLP: Speech","Natural Language Processing: NLP: Language models"],"videos":[],"references":[{"position":1,"raw":"[Ba et al., 2016] Jimmy Lei Ba, Jamie Ryan Kiros, and Ge- offrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.","title":"Layer normalization","authors":"Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E Hinton","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bapna et al., 2019] Ankur Bapna, N. Arivazhagan, and Orhan Firat. Simple, scalable adaptation for neural ma- chine translation. In Proc. EMNLP-IJCNLP, pages 1538- 1548, 2019.","title":"Simple, scalable adaptation for neural machine translation","authors":"Ankur Bapna, N Arivazhagan, Orhan Firat","year":"2019","venue":"Proc. EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Conneau and Lample, 2019] Alexis Conneau and Guil- laume Lample. Cross-lingual language model pretraining. In Proc. NeurIPS, pages 7059-7069, 2019.","title":"Alexis Conneau and Guillaume Lample. Cross-lingual language model pretraining","authors":"Lample Conneau","year":"2019","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, et al. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. NAACL, pages 4171-4186, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee","year":"2019","venue":"Proc. NAACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ebrahimi and Kann, 2021] Abteen Ebrahimi and Katharina Kann. How to adapt your pretrained multilingual model to 1600 languages. In Proc. ACL-IJCNLP, pages 4555-4567, 2021.","title":"How to adapt your pretrained multilingual model to 1600 languages","authors":"Kann Ebrahimi, Abteen Ebrahimi, Katharina Kann","year":"2021","venue":"Proc. ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fukada et al., 1992] Toshiaki Fukada, Keiichi Tokuda, Takao Kobayashi, et al. An adaptive algorithm for mel- cepstral analysis of speech. In Proc. ICASSP, pages 137- 140, 1992.","title":"An adaptive algorithm for melcepstral analysis of speech","authors":"Toshiaki Fukada, Keiichi Tokuda, Takao Kobayashi","year":"1992","venue":"Proc. ICASSP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gordon Jr, 2005] Raymond G. Gordon Jr. Ethnologue, lan- guages of the world. https://www.ethnologue.com/, 2005. Accessed: 2023-05-27.","title":"Ethnologue, languages of the world","authors":"G Raymond, Gordon","year":"2023","venue":null,"doi":null,"url":"https://www.ethnologue.com/,2005","internal_url":null},{"position":8,"raw":"[Gouws et al., 2015] Stephan Gouws, Yoshua Bengio, and Gregory S. Corrado. Bilbowa: Fast bilingual distributed representations without word alignments. In Proc. ICML, pages 748-756, 2015.","title":"Bilbowa: Fast bilingual distributed representations without word alignments","authors":"Stephan Gouws, Yoshua Bengio, Gregory S Corrado","year":"2015","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gutkin, 2017] Alexander Gutkin. Uniform multilingual multi-speaker acoustic model for statistical parametric speech synthesis of low-resourced languages. In Proc. In- terspeech, pages 2183-2187, 2017.","title":"Uniform multilingual multi-speaker acoustic model for statistical parametric speech synthesis of low-resourced languages","authors":"Alexander Gutkin","year":"2017","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hammarström et al., 2021] Harald Hammarström, Robert Forkel, Martin Haspelmath, and Sebastian Bank. Glot- tolog 4.","title":"Glottolog 4","authors":"Hammarström","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hayashi et al., 2019] Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, et al. Pre-trained text embeddings for en- hanced text-to-speech synthesis. In Proc. Interspeech, pages 4430-4434, 2019.","title":"Pre-trained text embeddings for enhanced text-to-speech synthesis","authors":"Hayashi","year":"2019","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hayashi et al., 2021] Tomoki Hayashi, Ryuichi Yamamoto, Takenori Yoshimura, et al. Espnet2-tts: Extending the edge of tts research. arXiv preprint arXiv:2110.07840, 2021.","title":"Espnet2-tts: Extending the edge of tts research","authors":"Hayashi","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2021] Mutian He, Jingzhou Yang, Lei He, et al. Multilingual byte2speech models for scal- able low-resource speech synthesis. arXiv preprint arXiv:2103.03541, 2021.","title":"Multilingual byte2speech models for scalable low-resource speech synthesis","authors":"He","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hendrycks and Gimpel, 2016] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016.","title":"Gaussian error linear units","authors":"Gimpel Hendrycks, Dan Hendrycks, Kevin Gimpel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2021] Ye Jia, Heiga Zen, Jonathan Shen, et al. PnG BERT: Augmented BERT on phonemes and graphemes for neural TTS. arXiv preprint arXiv:2103.15060, 2021.","title":"PnG BERT: Augmented BERT on phonemes and graphemes for neural TTS","authors":"Jia","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kim et al., 2021] Jaehyeon Kim, Jungil Kong, and Juhee Son. Conditional variational autoencoder with adversar- ial learning for end-to-end text-to-speech. In Proc. ICML, pages 5530-5540, 2021.","title":"Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech","authors":"Kim","year":"2021","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kong et al., 2020] Jungil Kong, Jaehyeon Kim, and Jaeky- oung Bae. HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. Proc. NeurIPS, 33:17022-17033, 2020.","title":"HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis","authors":"Kong","year":"2020","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li and Zen, 2016] Bo Li and Heiga Zen. Multi-language multi-speaker acoustic modeling for LSTM-RNN based statistical parametric speech synthesis. In Proc. Inter- speech, pages 2468-2472, 2016.","title":"Multi-language multi-speaker acoustic modeling for LSTM-RNN based statistical parametric speech synthesis","authors":"Bo Li, Heiga Zen","year":"2016","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2019a] Bo Li, Yu Zhang, Tara Sainath, et al. Bytes are all you need: End-to-end multilingual speech recog- nition and synthesis with bytes. In Proc. ICASSP, pages 5621-5625, 2019.","title":"Bytes are all you need: End-to-end multilingual speech recognition and synthesis with bytes","authors":"Li","year":"2019","venue":"Proc. ICASSP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2019b] Naihan Li, Shujie Liu, Yanqing Liu, et al. Neural speech synthesis with Transformer network. In Proc. AAAI, pages 6706-6713, 2019.","title":"Neural speech synthesis with Transformer network","authors":"Li","year":"2019","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2022] Xinjian Li, Florian Metze, David R. Mortensen, et al. ASR2K: Speech recognition for around 2000 languages without audio. arXiv preprint arXiv:2209.02842, 2022.","title":"ASR2K: Speech recognition for around 2000 languages without audio","authors":"Xinjian Li, Florian Metze, David R Mortensen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lux and Vu, 2022] Florian Lux and Ngoc Thang Vu. Language-agnostic meta-learning for low-resource text-to- speech with articulatory features. In Proc. ACL, pages 6858-6868, 2022.","title":"Florian Lux and Ngoc Thang Vu. Language-agnostic meta-learning for low-resource text-tospeech with articulatory features","authors":"Vu Lux","year":"2022","venue":"Proc. ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"The M-AILABS speech dataset. https://www.caito.de/ 2019/01/the-m-ailabs-speech-dataset/, 2017. Accessed: 2023-05-27.","title":"The M-AILABS speech dataset","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://www.caito.de/2019/01/the-m-ailabs-speech-dataset/,2017","internal_url":null},{"position":24,"raw":"[Nagrani et al., 2017] Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. VoxCeleb: A large-scale speaker iden- tification dataset. In Proc. Interspeech, pages 2616-2620, 2017.","title":"VoxCeleb: A large-scale speaker identification dataset","authors":"Arsha Nagrani, Joon, Son Chung, Andrew Zisserman","year":"2017","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Nair and Hinton, 2010] Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann ma- chines. In Proc. ICML, 2010.","title":"Rectified linear units improve restricted boltzmann machines","authors":"Vinod Nair, Geoffrey E Hinton","year":"2010","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ni et al., 2022] Junrui Ni, Liming Wang, Heting Gao, et al. Unsupervised text-to-speech synthesis by unsupervised automatic speech recognition. In Proc. Interspeech, pages 461-465, 2022.","title":"Unsupervised text-to-speech synthesis by unsupervised automatic speech recognition","authors":"Ni","year":"2022","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nystrom et al., 2015] Nicholas A. Nystrom, Michael J. Levine, Ralph Roskies, et al. Bridges: a uniquely flexi- ble hpc resource for new communities and data analytics. In Proc. XSEDE, pages 1-8, 2015.","title":"Bridges: a uniquely flexible hpc resource for new communities and data analytics","authors":"Nicholas A Nystrom, Michael J Levine, Ralph Roskies","year":"2015","venue":"Proc. XSEDE","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Paolacci et al., 2010] Gabriele Paolacci, Jesse Chandler, and Panagiotis G. Ipeirotis. Running experiments on ama- zon mechanical turk. Judgment and Decision making, 5(5):411-419, 2010.","title":"Running experiments on amazon mechanical turk","authors":"Gabriele Paolacci, Jesse Chandler, Panagiotis G Ipeirotis","year":"2010","venue":"Judgment and Decision making","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Park and Mulc, 2019] Kyubyong Park and Thomas Mulc. CSS10: A collection of single speaker speech datasets for 10 languages. Proc. Interspeech, pages 1566-1570, 2019.","title":"Kyubyong Park and Thomas Mulc. CSS10: A collection of single speaker speech datasets for 10 languages","authors":"Mulc Park","year":"2019","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Pires et al., 2019] Telmo Pires, Eva Schlinger, and Dan Gar- rette. How multilingual is multilingual BERT? In Proc. ACL, pages 4996-5001, 2019.","title":"How multilingual is multilingual BERT?","authors":"Pires","year":"2019","venue":"Proc. ACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Prakash et al., 2019] Anusha Prakash, A Leela Thomas, S. Umesh, et al. Building multilingual end-to-end speech synthesisers for Indian languages. In Proc. SSW, pages 194-199, 2019.","title":"Building multilingual end-to-end speech synthesisers for Indian languages","authors":"Anusha Prakash, Leela Thomas, S Umesh","year":"2019","venue":"Proc. SSW","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Radford et al., 2022] Alec Radford, Jong Wook Kim, Tao Xu, et al. Robust speech recognition via large-scale weak supervision. arXiv preprint arXiv:2212.04356, 2022.","title":"Robust speech recognition via large-scale weak supervision","authors":"Alec Radford, Jong Wook Kim, Tao Xu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ravanelli et al., 2021] Mirco Ravanelli, Titouan Parcollet, Peter William VanHarn Plantinga, et al. Speech- Brain: A general-purpose speech toolkit. arXiv preprint arXiv:2106.04624, 2021.","title":"Speech-Brain: A general-purpose speech toolkit","authors":"Ravanelli","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Ren et al., 2019] Yi Ren, Xu Tan, Tao Qin, et al. Almost unsupervised text to speech and automatic speech recog- nition. In Proc. ICML, pages 5410-5419, 2019.","title":"Almost unsupervised text to speech and automatic speech recognition","authors":"Yi Ren, Xu Tan, Tao Qin","year":"2019","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Ruder et al., 2019] Sebastian Ruder, Ivan Vulic, and Anders Søgaard. A survey of cross-lingual word embedding mod- els. JAIR, 65:569-631, 2019.","title":"A survey of cross-lingual word embedding models","authors":"Sebastian Ruder, Ivan Vulic, Anders Søgaard","year":"2019","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Saeki et al., 2022a] Takaaki Saeki, Detai Xin, Wataru Nakata, et al. UTMOS: UTokyo-SaruLab system for VoiceMOS Challenge 2022. In Proc. Interspeech, pages 4521-4525, 2022.","title":"UTMOS: UTokyo-SaruLab system for VoiceMOS Challenge","authors":"Saeki","year":"2022","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Saeki et al., 2022b] Takaaki Saeki, Heiga Zen, Zhehuai Chen, et al. Virtuoso: Massive multilingual speech-text joint semi-supervised learning for text-to-speech. arXiv preprint arXiv:2210.15447, 2022.","title":"Virtuoso: Massive multilingual speech-text joint semi-supervised learning for text-to-speech","authors":"Saeki","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Seki et al., 2022] Kentaro Seki, Shinnosuke Takamichi, Takaaki Saeki, et al. Text-to-speech synthesis from dark data with evaluation-in-the-loop data selection. arXiv preprint arXiv:2210.14850, 2022.","title":"Text-to-speech synthesis from dark data with evaluation-in-the-loop data selection","authors":"Kentaro Seki, Shinnosuke Takamichi, Takaaki Saeki","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Shen et al., 2018] Jonathan Shen, Ruoming Pang, Ron J Weiss, et al. Natural TTS synthesis by conditioning WaveNet on mel spectrogram predictions. In Proc. ICASSP, pages 4779-4783, 2018.","title":"Natural TTS synthesis by conditioning WaveNet on mel spectrogram predictions","authors":"Jonathan Shen, Ruoming Pang, Ron J Weiss","year":"2018","venue":"Proc. ICASSP","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Snyder et al., 2018] David Snyder, Daniel Garcia-Romero, Gregory Sell, et al. X-vectors: Robust dnn embeddings for speaker recognition. In Proc. ICASSP, pages 5329-5333, 2018.","title":"X-vectors: Robust dnn embeddings for speaker recognition","authors":"David Snyder, Daniel Garcia-Romero, Gregory Sell","year":"2018","venue":"Proc. ICASSP","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Staib et al., 2020] Marlene Staib, Tian Huey Teh, Alexan- dra Torresquintero, et al. Phonological features for 0-shot multilingual speech synthesis. In Proc. Interspeech, pages 2942-2946, 2020.","title":"Phonological features for 0-shot multilingual speech synthesis","authors":"Marlene Staib, Tian Huey Teh, Alexandra Torresquintero","year":"2020","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Tachibana et al., 2018] Hideyuki Tachibana, Katsuya Uenoyama, and Shunsuke Aihara. Efficiently trainable text-to-speech system based on deep convolutional net- works with guided attention. In Proc. ICASSP, pages 4784-4788, 2018.","title":"Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention","authors":"Hideyuki Tachibana, Katsuya Uenoyama, Shunsuke Aihara","year":"2018","venue":"Proc. ICASSP","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[van der Maaten and Hinton, 2008] Laurens van der Maaten and Geoffrey E. Hinton. Visualizing data using t-sne. JMLR, 9(11):2579-2605, 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey E Van Der Maaten, Hinton","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. Attention is all you need. In Proc. NeurIPS, volume 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar","year":"2017","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Veaux et al., 2017] Christophe Veaux, Junichi Yamagishi, Kirsten MacDonald, et al. CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit. Uni- versity of Edinburgh. The Centre for Speech Technology Research (CSTR), 2017.","title":"CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit","authors":"Christophe Veaux, Junichi Yamagishi, Kirsten Macdonald","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Wang et al., 2021] Changhan Wang, Morgane Riviere, Ann Lee, et al. VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learn- ing and interpretation. In Proc. ACL, pages 993-1003, 2021.","title":"VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation","authors":"Wang","year":"2021","venue":"Proc. ACL","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Watanabe et al., 2018] Shinji Watanabe, Takaaki Hori, Shigeki Karita, et al. ESPnet: End-to-end speech process- ing toolkit. Proc. Interspeech, pages 2207-2211, 2018.","title":"ESPnet: End-to-end speech processing toolkit","authors":"Shinji Watanabe, Takaaki Hori, Shigeki Karita","year":"2018","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Wu and Dredze, 2019] Shijie Wu and Mark Dredze. Beto, Bentz, Becas: The surprising cross-lingual effectiveness of BERT. In Proc. EMNLP-IJCNLP, pages 833-844, 2019.","title":"Becas: The surprising cross-lingual effectiveness of BERT","authors":"Shijie Wu, Mark Dredze, Beto, Bentz","year":"2019","venue":"Proc. EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Xu et al., 2021] Guanghui Xu, Wei Song, Zhengchen Zhang, et al. Improving prosody modelling with cross- utterance BERT embeddings for end-to-end speech syn- thesis. In Proc. ICASSP, pages 6079-6083, 2021.","title":"Improving prosody modelling with crossutterance BERT embeddings for end-to-end speech synthesis","authors":"Guanghui Xu, Wei Song, Zhengchen Zhang","year":"2021","venue":"Proc. ICASSP","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zen et al., 2012] Heiga Zen, Norbert Braunschweiler, Sabine Buchholz, et al. Statistical parametric speech synthesis based on speaker and language factorization. TASLP, 20(6):1713-1724, 2012.","title":"Statistical parametric speech synthesis based on speaker and language factorization","authors":"Zen","year":"2012","venue":"TASLP","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zen et al., 2019] Heiga Zen, Viet Dang, Rob Clark, et al. LibriTTS: A corpus derived from LibriSpeech for text-to- speech. In Proc. Interspeech, pages 1526-1530, 2019.","title":"LibriTTS: A corpus derived from LibriSpeech for text-tospeech","authors":"Zen","year":"2019","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhang and Lin, 2020] Haitong Zhang and Yue Lin. Unsu- pervised learning for sequence-to-sequence text-to-speech for low-resource languages. Proc. Interspeech, pages 3161-3165, 2020.","title":"Unsupervised learning for sequence-to-sequence text-to-speech for low-resource languages","authors":"Lin ; Zhang, Haitong Zhang, Yue Lin","year":"2020","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhang et al., 2022] Guangyan Zhang, Kaitao Song, Xu Tan, et al. Mixed-Phoneme BERT: Improving BERT with mixed phoneme and sup-phoneme representations for text to speech. In Proc. Interspeech, pages 456-460, 2022.","title":"Mixed-Phoneme BERT: Improving BERT with mixed phoneme and sup-phoneme representations for text to speech","authors":"Zhang","year":"2022","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}