{"type":"ScholarlyArticle","title":"Recent Advances in Direct Speech-to-text Translation","authors":[{"name":"Chen Xu","slug":"chen-xu","given":"Chen","family":"Xu","affiliation":"School of Computer Science and Engineering , Northeastern University , Shenyang , China"},{"name":"Rong Ye","slug":"rong-ye","given":"Rong","family":"Ye","affiliation":"ByteDance"},{"name":"Qianqian Dong","slug":"qianqian-dong","given":"Qianqian","family":"Dong","affiliation":"ByteDance"},{"name":"Chengqi Zhao","slug":"chengqi-zhao","given":"Chengqi","family":"Zhao","affiliation":"ByteDance"},{"name":"Tom Ko","slug":"tom-ko","given":"Tom","family":"Ko","affiliation":"ByteDance"},{"name":"Mingxuan Wang","slug":"mingxuan-wang","given":"Mingxuan","family":"Wang","affiliation":"ByteDance"},{"name":"Tong Xiao","slug":"tong-xiao","given":"Tong","family":"Xiao","affiliation":"School of Computer Science and Engineering , Northeastern University , Shenyang , China; NiuTrans Research , Shenyang , China"},{"name":"Jingbo Zhu","slug":"jingbo-zhu","given":"Jingbo","family":"Zhu","affiliation":"School of Computer Science and Engineering , Northeastern University , Shenyang , China; NiuTrans Research , Shenyang , China"}],"abstract":"Recently, speech-to-text translation has attracted more and more attention and many studies have emerged rapidly. In this paper, we present a comprehensive survey on direct speech translation aiming to summarize the current state-of-the-art techniques. First, we categorize the existing research work into three directions based on the main challenges --- modeling burden, data scarcity, and application issues. To tackle the problem of modeling burden, two main structures have been proposed, encoder-decoder framework (Transformer and the variants) and multitask frameworks. For the challenge of data scarcity, recent work resorts to many sophisticated techniques, such as data augmentation, pre-training, knowledge distillation, and multilingual modeling. We analyze and summarize the application issues, which include real-time, segmentation, named entity, gender bias, and code-switching. Finally, we discuss some promising directions for future work.","doi":"10.24963/ijcai.2023/761","url":"https://ijcai.org/proceedings/2023/761","pdf":"https://ijcai.org/proceedings/2023/0761.pdf","markdown":"https://ijcai.org/proceedings/2023/761.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/761","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Survey Track","track":"Survey Track","number":761,"pages":{"first":6796,"last":6804},"keywords":["Survey: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Anastasopoulos and Chiang, 2018] Antonios Anastasopou- los and David Chiang. Tied multitask learning for neural speech translation. In NAACL, 2018.","title":"Antonios Anastasopoulos and David Chiang. Tied multitask learning for neural speech translation","authors":"Chiang Anastasopoulos","year":"2018","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ao et al., 2022] Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, and Furu Wei. SpeechT5: Unified-modal encoder-decoder pre-training for spoken language processing. In ACL, 2022.","title":"SpeechT5: Unified-modal encoder-decoder pre-training for spoken language processing","authors":"Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei","year":"2022","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bahar et al., 2019] Parnia Bahar, Albert Zeyer, Ralf Schlüter, and Hermann Ney. On using SpecAugment for end-to-end speech translation. In IWSLT, 2019.","title":"On using SpecAugment for end-to-end speech translation","authors":"Parnia Bahar, Albert Zeyer, Ralf Schlüter, Hermann Ney","year":"2019","venue":"IWSLT","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bapna et al., 2022] Ankur Bapna, Colin Cherry, Yu Zhang, Ye Jia, Melvin Johnson, Yong Cheng, Simran Khanuja, Ja- son Riesa, and Alexis Conneau. mslam: Massively multi- lingual joint pre-training for speech and text. CoRR, 2022.","title":"mslam: Massively multilingual joint pre-training for speech and text","authors":"Ankur Bapna, Colin Cherry, Yu Zhang, Ye Jia, Melvin Johnson, Yong Cheng, Simran Khanuja, Jason Riesa, Alexis Conneau","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bentivogli et al., 2020] Luisa Bentivogli, Beatrice Savoldi, Matteo Negri, Mattia A. Di Gangi, Roldano Cattoni, and Marco Turchi. Gender in danger? evaluating speech trans- lation technology on the MuST-SHE corpus. In ACL, 2020.","title":"Gender in danger? evaluating speech translation technology on the MuST-SHE corpus","authors":"Luisa Bentivogli, Beatrice Savoldi, Matteo Negri, Mattia A Di, Roldano Gangi, Marco Cattoni, Turchi","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bentivogli et al., 2021] Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, Mat- teo Negri, and Marco Turchi. Cascade versus direct speech translation: Do the differences still make a difference? In ACL, 2021. Bérard et al., 2016] Alexandre Bérard, Olivier Pietquin, Laurent Besacier, and Christophe Servan. Listen and trans- late: A proof of concept for end-to-end speech-to-text translation. In NIPS Workshop on end-to-end learning for speech and audio processing, 2016.","title":"Cascade versus direct speech translation: Do the differences still make a difference? In ACL","authors":"Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, Matteo Negri, Marco Turchi","year":"2016","venue":"NIPS Workshop on end-to-end learning for speech and audio processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chang and Lee, 2022] Chih-Chiang Chang and Hung-yi Lee. Exploring continuous integrate-and-fire for adaptive simultaneous speech translation. CoRR, 2022.","title":"Exploring continuous integrate-and-fire for adaptive simultaneous speech translation","authors":"Chang, Lee ; Chih-Chiang Chang, Hung-Yi Lee","year":"2022","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2020] Junkun Chen, Mingbo Ma, Renjie Zheng, and Liang Huang. MAM: masked acoustic model- ing for end-to-end speech-to-text translation. CoRR, 2020.","title":"MAM: masked acoustic modeling for end-to-end speech-to-text translation","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cheng et al., 2022] Yong Cheng, Yu Zhang, Melvin John- son, Wolfgang Macherey, and Ankur Bapna. Mu2slam: Multitask, multilingual speech and language models. CoRR, 2022.","title":"Mu2slam: Multitask, multilingual speech and language models","authors":"Yong Cheng, Yu Zhang, Melvin Johnson, Wolfgang Macherey, Ankur Bapna","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chuang et al., 2021] Shun-Po Chuang, Yung-Sung Chuang, Chih-Chiang Chang, and Hung-yi Lee. Investigating the reordering capability in CTC-based non-autoregressive end-to-end speech translation. In Findings of ACL, 2021.","title":"Investigating the reordering capability in CTC-based non-autoregressive end-to-end speech translation","authors":"Shun-Po Chuang, Yung-Sung Chuang, Chih-Chiang Chang, Hung-Yi Lee","year":"2021","venue":"Findings of ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dong et al., 2021] Qianqian Dong, Rong Ye, Mingxuan Wang, Hao Zhou, Shuang Xu, Bo Xu, and Lei Li. Lis- ten, understand and translate: Triple supervision decou- ples end-to-end speech-to-text translation. In AAAI, 2021.","title":"Listen, understand and translate: Triple supervision decouples end-to-end speech-to-text translation","authors":"Qianqian Dong, Rong Ye, Mingxuan Wang, Hao Zhou, Shuang Xu, Bo Xu, Lei Li","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fang et al., 2022] Qingkai Fang, Rong Ye, Lei Li, Yang Feng, and Mingxuan Wang. STEMM: Self-learning with speech-text manifold mixup for speech translation. In ACL, 2022.","title":"STEMM: Self-learning with speech-text manifold mixup for speech translation","authors":"Qingkai Fang, Rong Ye, Lei Li, Yang Feng, Mingxuan Wang","year":"2022","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gaido et al., 2021] Marco Gaido, Beatrice Savoldi, Luisa Bentivogli, Matteo Negri, and Marco Turchi. How to split: the effect of word segmentation on gender bias in speech translation. In Findings of ACL, 2021.","title":"How to split: the effect of word segmentation on gender bias in speech translation","authors":"Marco Gaido, Beatrice Savoldi, Luisa Bentivogli, Matteo Negri, Marco Turchi","year":"2021","venue":"Findings of ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gaido et al., 2022a] Marco Gaido, Matteo Negri, and Marco Turchi. Who are we talking about? handling person names in speech translation. In IWSLT, 2022.","title":"Who are we talking about? handling person names in speech translation","authors":"Gaido","year":"2022","venue":"IWSLT","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gaido et al., 2022b] Marco Gaido, Sara Papi, Matteo Negri, and Marco Turchi. Joint speech translation and named en- tity recognition. CoRR, 2022.","title":"Joint speech translation and named entity recognition","authors":"Gaido","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gangi et al., 2019] Mattia Antonino Di Gangi, Matteo Ne- gri, and Marco Turchi. Adapting transformer to end-to-end spoken language translation. In InterSpeech, 2019.","title":"Adapting transformer to end-to-end spoken language translation","authors":"Mattia Antonino, Di Gangi, Matteo Negri, Marco Turchi","year":"2019","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gulati et al., 2020] Anmol Gulati, James Qin, Chung- Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruom- ing Pang. Conformer: Convolution-augmented trans- former for speech recognition. In InterSpeech, 2020.","title":"Conformer: Convolution-augmented transformer for speech recognition","authors":"Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, Ruoming Pang","year":"2020","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Han et al., 2021] Chi Han, Mingxuan Wang, Heng Ji, and Lei Li. Learning shared semantic space for speech-to-text translation. In Findings of ACL, 2021.","title":"Learning shared semantic space for speech-to-text translation","authors":"Han","year":"2021","venue":"Findings of ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hsu et al., 2021] Wei-Ning Hsu, Benjamin Bolte, Yao- Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhut- dinov, and Abdelrahman Mohamed. Hubert: Self- supervised speech representation learning by masked pre- diction of hidden units. TASLP, 2021.","title":"Hubert: Selfsupervised speech representation learning by masked prediction of hidden units","authors":"Wei-Ning Hsu, Benjamin Bolte, Hubert Yao-Hung, Kushal Tsai, Ruslan Lakhotia, Abdelrahman Salakhutdinov, Mohamed","year":"2021","venue":"TASLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Huber et al., 2022] Christian Huber, Enes Yavuz Ugan, and Alexander Waibel. Code-switching without switching: Language agnostic end-to-end speech translation. CoRR, 2022.","title":"Code-switching without switching: Language agnostic end-to-end speech translation","authors":"Huber","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Inaguma et al., 2019] Hirofumi Inaguma, Kevin Duh, Tat- suya Kawahara, and Shinji Watanabe. Multilingual end- to-end speech translation. In ASRU, 2019.","title":"Multilingual endto-end speech translation","authors":"Hirofumi Inaguma, Kevin Duh, Tatsuya Kawahara, Shinji Watanabe","year":"2019","venue":"ASRU","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Inaguma et al., 2021a] Hirofumi Inaguma, Siddharth Dalmia, Brian Yan, and Shinji Watanabe. Fast-md: Fast multi-decoder end-to-end speech translation with non-autoregressive hidden intermediates. In ASRU, 2021.","title":"Fast-md: Fast multi-decoder end-to-end speech translation with non-autoregressive hidden intermediates","authors":"Inaguma","year":"2021","venue":"ASRU","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Inaguma et al., 2021b] Hirofumi Inaguma, Tatsuya Kawa- hara, and Shinji Watanabe. Source and target bidirectional knowledge distillation for end-to-end speech translation. In NAACL, 2021.","title":"Source and target bidirectional knowledge distillation for end-to-end speech translation","authors":"Inaguma","year":"2021","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Indurthi et al., 2021] Sathish Indurthi, Mohd Abbas Zaidi, Nikhil Kumar Lakumarapu, Beomseok Lee, Hyojung Han, Seokchan Ahn, Sangha Kim, Chanwoo Kim, and Inchul Hwang. Task aware multi-task learning for speech to text tasks. In ICASSP, 2021.","title":"Task aware multi-task learning for speech to text tasks","authors":"Sathish Indurthi, Abbas Mohd, Nikhil Zaidi, Beomseok Kumar Lakumarapu, Hyojung Lee, Seokchan Han, Sangha Ahn, Chanwoo Kim, Inchul Kim, Hwang","year":"2021","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Jia et al., 2019] Ye Jia, Melvin Johnson, Wolfgang Macherey, Ron J. Weiss, Yuan Cao, Chung-Cheng Chiu, Naveen Ari, Stella Laurenzo, and Yonghui Wu. Leveraging weakly supervised data to improve end-to-end speech-to-text translation. In ICASSP, 2019.","title":"Leveraging weakly supervised data to improve end-to-end speech-to-text translation","authors":"Ye Jia, Melvin Johnson, Wolfgang Macherey, Ron J Weiss, Yuan Cao, Chung-Cheng Chiu, Naveen Ari, Stella Laurenzo, Yonghui Wu","year":"2019","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Le et al., 2021] Hang Le, Juan Pino, Changhan Wang, Jiatao Gu, Didier Schwab, and Laurent Besacier. Lightweight adapter tuning for multilingual speech trans- lation. In ACL, 2021.","title":"Lightweight adapter tuning for multilingual speech translation","authors":"Le","year":"2021","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Li et al., 2021] Xian Li, Changhan Wang, Yun Tang, Chau Tran, Yuqing Tang, Juan Pino, Alexei Baevski, Alexis Conneau, and Michael Auli. Multilingual speech trans- lation from efficient finetuning of pretrained models. In ACL, 2021.","title":"Multilingual speech translation from efficient finetuning of pretrained models","authors":"Xian Li, Changhan Wang, Yun Tang, Chau Tran, Yuqing Tang, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli","year":"2021","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Liu et al., 2019] Yuchen Liu, Hao Xiong, Jiajun Zhang, Zhongjun He, Hua Wu, Haifeng Wang, and Chengqing Zong. End-to-end speech translation with knowledge dis- tillation. In InterSpeech, 2019.","title":"End-to-end speech translation with knowledge distillation","authors":"Yuchen Liu, Hao Xiong, Jiajun Zhang, Zhongjun He, Hua Wu, Haifeng Wang, Chengqing Zong","year":"2019","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Liu et al., 2020a] Yuchen Liu, Jiajun Zhang, Hao Xiong, Long Zhou, Zhongjun He, Hua Wu, Haifeng Wang, and Chengqing Zong. Synchronous speech recognition and speech-to-text translation with interactive decoding. In AAAI, 2020.","title":"Synchronous speech recognition and speech-to-text translation with interactive decoding","authors":"Liu","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Liu et al., 2020b] Yuchen Liu, Junnan Zhu, Jiajun Zhang, and Chengqing Zong. Bridging the modality gap for speech-to-text translation. CoRR, 2020.","title":"Bridging the modality gap for speech-to-text translation","authors":"Liu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Liu et al., 2021] Dan Liu, Mengge Du, Xiaoxi Li, Ya Li, and Enhong Chen. Cross attention augmented transducer networks for simultaneous translation. In EMNLP, 2021.","title":"Cross attention augmented transducer networks for simultaneous translation","authors":"Dan Liu, Mengge Du, Xiaoxi Li, Ya Li, Enhong Chen","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Park et al., 2019] Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. Specaugment: A simple data augmentation method for automatic speech recognition. In InterSpeech, 2019.","title":"Specaugment: A simple data augmentation method for automatic speech recognition","authors":"Daniel S Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D Cubuk, V Quoc, Le","year":"2019","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Pino et al., 2020] Juan Miguel Pino, Qiantong Xu, Xutai Ma, Mohammad Javad Dousti, and Yun Tang. Self- training for end-to-end speech translation. In InterSpeech, 2020.","title":"Selftraining for end-to-end speech translation","authors":"Pino","year":"2020","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Ren et al., 2020] Yi Ren, Jinglin Liu, Xu Tan, Chen Zhang, Tao Qin, Zhou Zhao, and Tie-Yan Liu. SimulSpeech: End- to-end simultaneous speech to text translation. In ACL, 2020.","title":"SimulSpeech: Endto-end simultaneous speech to text translation","authors":"Yi Ren, Jinglin Liu, Xu Tan, Chen Zhang, Tao Qin, Zhou Zhao, Tie-Yan Liu","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Savoldi et al., 2022] Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. Under the morphosyntactic lens: A multifaceted evaluation of gender bias in speech translation. In ACL, 2022.","title":"Under the morphosyntactic lens: A multifaceted evaluation of gender bias in speech translation","authors":"Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, Marco Turchi","year":"2022","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Schneider et al., 2019] Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. wav2vec: Unsuper- vised pre-training for speech recognition. In InterSpeech, 2019.","title":"wav2vec: Unsupervised pre-training for speech recognition","authors":"Steffen Schneider, Alexei Baevski, Ronan Collobert, Michael Auli","year":"2019","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Sperber and Paulik, 2020] Matthias Sperber and Matthias Paulik. Speech translation and the end-to-end promise: Taking stock of where we are. In ACL, 2020.","title":"Speech translation and the end-to-end promise: Taking stock of where we are","authors":"Paulik Sperber, Matthias Sperber, Matthias Paulik","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Sperber et al., 2019] Matthias Sperber, Graham Neubig, Jan Niehues, and Alex Waibel. Attention-passing models for robust and data-efficient end-to-end speech translation. TACL, 2019.","title":"Attention-passing models for robust and data-efficient end-to-end speech translation","authors":"Matthias Sperber, Graham Neubig, Jan Niehues, Alex Waibel","year":"2019","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Stentiford and Steer, 1988] Fred WM Stentiford and Mar- tin G Steer. Machine translation of speech. British Telecom technology journal, (2), 1988.","title":"Machine translation of speech","authors":"Steer ; Stentiford, W M Fred, Martin G Stentiford, Steer","year":"1988","venue":"British Telecom technology journal","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Tang et al., 2021a] Yun Tang, Juan Pino, Xian Li, Chang- han Wang, and Dmitriy Genzel. Improving speech transla- tion by understanding and learning from the auxiliary text translation task. In ACL, 2021.","title":"Improving speech translation by understanding and learning from the auxiliary text translation task","authors":"Tang","year":"2021","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Tang et al., 2021b] Yun Tang, Juan Pino, Changhan Wang, Xutai Ma, and Dmitriy Genzel. A general multi-task learn- ing framework to leverage text data for speech to text tasks. In ICASSP, 2021.","title":"A general multi-task learning framework to leverage text data for speech to text tasks","authors":"Tang","year":"2021","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Tang et al., 2022] Yun Tang, Hongyu Gong, Ning Dong, Changhan Wang, Wei-Ning Hsu, Jiatao Gu, Alexei Baevski, Xian Li, Abdelrahman Mohamed, Michael Auli, and Juan Pino. Unified speech-text pre-training for speech translation and recognition. In ACL, 2022.","title":"Unified speech-text pre-training for speech translation and recognition","authors":"Yun Tang, Hongyu Gong, Ning Dong, Changhan Wang, Wei-Ning Hsu, Jiatao Gu, Alexei Baevski, Xian Li, Abdelrahman Mohamed, Michael Auli, Juan Pino","year":"2022","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Tsiamas et al., 2022] Ioannis Tsiamas, Gerard I Gállego, José AR Fonollosa, and Marta R Costa-jussà. Shas: Approaching optimal segmentation for end-to-end speech translation. In InterSpeech, 2022.","title":"Shas: Approaching optimal segmentation for end-to-end speech translation","authors":"Ioannis Tsiamas, Gerard I Gállego, Marta R José Ar Fonollosa, Costa-Jussà","year":"2022","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neurips, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Neurips","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wang et al., 2020a] Changhan Wang, Juan Pino, Anne Wu, and Jiatao Gu. CoVoST: A diverse multilingual speech- to-text translation corpus. In IREC, 2020.","title":"CoVoST: A diverse multilingual speechto-text translation corpus","authors":"Wang","year":"2020","venue":"IREC","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Wang et al., 2020b] Chengyi Wang, Yu Wu, Shujie Liu, Ming Zhou, and Zhenglu Yang. Curriculum pre-training for end-to-end speech translation. In ACL, 2020.","title":"Curriculum pre-training for end-to-end speech translation","authors":"Wang","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Wang et al., 2021] Changhan Wang, Anne Wu, Juan Pino, Alexei Baevski, Michael Auli, and Alexis Conneau. Large-scale self-and semi-supervised learning for speech translation. In InterSpeech, 2021.","title":"Large-scale self-and semi-supervised learning for speech translation","authors":"Wang","year":"2021","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Weiss et al., 2017] Ron J. Weiss, Jan Chorowski, Navdeep Jaitly, Yonghui Wu, and Zhifeng Chen. Sequence-to- sequence models can directly translate foreign speech. In InterSpeech, 2017.","title":"Sequence-tosequence models can directly translate foreign speech","authors":"Ron J Weiss, Jan Chorowski, Navdeep Jaitly, Yonghui Wu, Zhifeng Chen","year":"2017","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Weller et al., 2022] Orion Weller, Matthias Sperber, Telmo Pires, Hendra Setiawan, Christian Gollan, Dominic Telaar, and Matthias Paulik. End-to-end speech translation for code switched speech. In Findings of ACL, 2022.","title":"End-to-end speech translation for code switched speech","authors":"Weller","year":"2022","venue":"Findings of ACL","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Wu et al., 2020] Anne Wu, Changhan Wang, Juan Miguel Pino, and Jiatao Gu. Self-supervised representations im- prove end-to-end speech translation. In InterSpeech, 2020.","title":"Self-supervised representations improve end-to-end speech translation","authors":"Anne Wu, Changhan Wang, Juan Miguel Pino, Jiatao Gu","year":"2020","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Wu et al., 2022] Felix Wu, Kwangyoun Kim, Shinji Watan- abe, Kyu Han, Ryan McDonald, Kilian Q Weinberger, and Yoav Artzi. Wav2seq: Pre-training speech-to-text encoder- decoder models using pseudo languages. CoRR, 2022.","title":"Wav2seq: Pre-training speech-to-text encoderdecoder models using pseudo languages","authors":"Felix Wu, Kwangyoun Kim, Shinji Watanabe, Kyu Han, Ryan Mcdonald, Kilian Q Weinberger, Yoav Artzi","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Xu et al., 2021] Chen Xu, Bojie Hu, Yanyang Li, Yuhao Zhang, Shen Huang, Qi Ju, Tong Xiao, and Jingbo Zhu. Stacked acoustic-and-textual encoding: Integrating the pre-trained models into speech translation encoders. In ACL, 2021.","title":"Stacked acoustic-and-textual encoding: Integrating the pre-trained models into speech translation encoders","authors":"Chen Xu, Bojie Hu, Yanyang Li, Yuhao Zhang, Shen Huang, Qi Ju, Xiao Tong, Jingbo Zhu","year":"2021","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Ye et al., 2021] Rong Ye, Mingxuan Wang, and Lei Li. End-to-end speech translation via cross-modal progressive training. In InterSpeech, 2021.","title":"End-to-end speech translation via cross-modal progressive training","authors":"Rong Ye, Mingxuan Wang, Lei Li","year":"2021","venue":"InterSpeech","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Ye et al., 2022] Rong Ye, Mingxuan Wang, and Lei Li. Cross-modal contrastive learning for speech translation. In NAACL, 2022.","title":"Cross-modal contrastive learning for speech translation","authors":"Rong Ye, Mingxuan Wang, Lei Li","year":"2022","venue":"NAACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}