{"type":"ScholarlyArticle","title":"Contrastive Learning for Sign Language Recognition and Translation","authors":[{"name":"Shiwei Gan","slug":"shiwei-gan","given":"Shiwei","family":"Gan","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Yafeng Yin","slug":"yafeng-yin","given":"Yafeng","family":"Yin","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Zhiwei Jiang","slug":"zhiwei-jiang","given":"Zhiwei","family":"Jiang","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Kang Xia","slug":"kang-xia","given":"Kang","family":"Xia","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Lei Xie","slug":"lei-xie","given":"Lei","family":"Xie","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Sanglu Lu","slug":"sanglu-lu","given":"Sanglu","family":"Lu","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"}],"abstract":"There are two problems that widely exist in current end-to-end sign language processing architecture. One is the CTC spike phenomenon which weakens the visual representational ability in Continuous Sign Language Recognition (CSLR). The other one is the exposure bias problem which leads to the accumulation of translation errors during inference in  Sign Language Translation (SLT). In this paper, we tackle these issues by introducing contrast learning, aiming to enhance both visual-level feature representation and semantic-level error tolerance. Specifically, to alleviate CTC spike phenomenon and enhance visual-level representation, we design a visual contrastive loss by minimizing visual feature distance between different augmented samples of frames in one sign video, so that the model can further explore features by utilizing numerous unlabeled frames in an unsupervised way. To alleviate exposure bias problem and improve semantic-level error tolerance, we design a semantic contrastive loss by re-inputting the predicted sentence into semantic module and comparing features of ground-truth sequence  and predicted sequence, for exposing model to its own mistakes. Besides, we propose two new metrics, i.e., Blank Rate and Consecutive Wrong Word Rate to directly reflect our improvement on the two problems. Extensive experimental results on current sign language datasets demonstrate the effectiveness of our approach, which achieves state-of-the-art performance.","doi":"10.24963/ijcai.2023/85","url":"https://ijcai.org/proceedings/2023/85","pdf":"https://ijcai.org/proceedings/2023/0085.pdf","markdown":"https://ijcai.org/proceedings/2023/85.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/85","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":85,"pages":{"first":763,"last":772},"keywords":["Computer Vision: CV: Action and behavior recognition","Computer Vision: CV: Vision and language"],"videos":[],"references":[{"position":1,"raw":"34.4 27.05 22.38 - 46.61 33.73 26.19 21.32 SimulSLT [Yin et al., 2021] (MM2021) ✓ 49.21 47.76 35.33 27.85 22.85 49.23 48.23 35.59 28.04 23.14","title":"MM","authors":null,"year":"2021","venue":"MM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"BN-TIN [Zhou et al., 2021a](CVPR2021) ✓ 50.29 51.11 37.90 29.80 24.45 49.54 50.8 37.75 29.72 24.32 Ours 51.60 51.04 39.49 30.69 26.73 51.97 51.96 40.33 31.60 27.21 Ours (Masking 5%+Padding 10%) 52.47 52.29 39.60 31.34 27.83 52.24 52.48 41.17 32.30 27.88 Table 4: Comparison with of SLT performance on Phoenix14T (F: face, M: mouth, H: hands, S: skeleton, P: pretraining on other datasets). Model CSL-daily DEV CSL-daily TEST ROUGE BLUE-1 BLEU-2 BLEU-3 BLEU-4 ROUGE BLUE-1 BLEU-2 BLEU-3 BLEU-4 SL-Luong [Camgoz et al., 2018](CVPR2018) 40.18 41.46 25.71 16.57 11.06 40.05 41.55 25.73 16.54 11.03 SL-Transf [Camgoz et al., 2020b](CVPR2020) 44.18 46.82 32.22 22.49 15.94 44.81 47.09 32.49 22.61 16.24","title":"Comparison with of SLT performance on Phoenix14T (F: face, M: mouth, H: hands, S: skeleton, P: pretraining on other datasets). Model CSL-daily DEV CSL-daily TEST ROUGE BLUE-1 BLEU-2 BLEU-3 BLEU-4 ROUGE BLUE-1 BLEU-2 BLEU-3 BLEU","authors":"Bn-Tin [ Zhou","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"BN-TIN [Zhou et al., 2021a](CVPR2021) 49.49 51.46 37.23 27.51 20.80 49.31 51.42 37.26 27.76 21.34 Ours 49.60 51.15 35.11 26.46 21.72 49.66 52.28 35.36 26.62 21.72 Ours (Masking 5%+Padding 10%) 50.34 51.97 37.10 27.53 21.79 50.73 52.31 37.37 27.89 21.81","title":"CVPR","authors":"Bn-Tin [ Zhou","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bengio et al., 2015] Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for se- quence prediction with recurrent neural networks. NIPS, 28, 2015.","title":"Scheduled sampling for sequence prediction with recurrent neural networks","authors":"Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Camgoz et al., 2017] Necati Cihan Camgoz, Simon Had- field, Oscar Koller, and Richard Bowden. Subunets: End- to-end hand shape and continuous sign language recogni- tion. In ICCV, pages 3075-3084. IEEE, 2017.","title":"Subunets: Endto-end hand shape and continuous sign language recognition","authors":"Simon Necati Cihan Camgoz, Oscar Hadfield, Richard Koller, Bowden","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Camgoz et al., 2018] Necati Cihan Camgoz, Simon Had- field, Oscar Koller, Hermann Ney, and Richard Bowden. Neural sign language translation. In CVPR, pages 7784- 7793, 2018.","title":"Neural sign language translation","authors":"Simon Necati Cihan Camgoz, Oscar Hadfield, Hermann Koller, Richard Ney, Bowden","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Camgoz et al., 2020a] Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. Multi-channel transformers for multi-articulatory sign language transla- tion. In ECCV, pages 301-319. Springer, 2020.","title":"Multi-channel transformers for multi-articulatory sign language translation","authors":"Camgoz","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Camgoz et al., 2020b] Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. Sign language transformers: Joint end-to-end sign language recognition and translation. In CVPR, pages 10023-10033, 2020.","title":"Sign language transformers: Joint end-to-end sign language recognition and translation","authors":"Camgoz","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen and He, 2021] Xinlei Chen and Kaiming He. Explor- ing simple siamese representation learning. In CVPR, pages 15750-15758, 2021.","title":"Exploring simple siamese representation learning","authors":"Chen, He ;, Xinlei Chen, Kaiming He","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597-1607. PMLR, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Chen et al., 2022] Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, and Stephen Lin. A simple multi-modality transfer learning baseline for sign language translation. In CVPR, pages 5120-5130, 2022.","title":"A simple multi-modality transfer learning baseline for sign language translation","authors":"Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, Stephen Lin","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Cheng et al., 2020] Ka Leong Cheng, Zhaoyang Yang, Qifeng Chen, and Yu-Wing Tai. Fully convolutional net- works for continuous sign language recognition. In ECCV, pages 697-714. Springer, 2020.","title":"Fully convolutional networks for continuous sign language recognition","authors":"Ka Leong Cheng, Zhaoyang Yang, Qifeng Chen, Yu-Wing Tai","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Cui et al., 2017] Runpeng Cui, Hu Liu, and Changshui Zhang. Recurrent convolutional neural networks for con- tinuous sign language recognition by staged optimization. In CVPR, pages 7361-7369, 2017.","title":"Recurrent convolutional neural networks for continuous sign language recognition by staged optimization","authors":"Cui","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Cui et al., 2019] Runpeng Cui, Hu Liu, and Changshui Zhang. A deep neural framework for continuous sign lan- guage recognition by iterative training. TMM, 21(7):1880- 1891, 2019.","title":"A deep neural framework for continuous sign language recognition by iterative training","authors":"Cui","year":"2019","venue":"TMM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Feng et al., 2021] Yang Feng, Shuhao Gu, Dengji Guo, Zhengxin Yang, and Chenze Shao. Guiding teacher forc- ing with seer forcing for neural machine translation. In ACL/IJCNLP (1), 2021.","title":"Guiding teacher forcing with seer forcing for neural machine translation","authors":"Yang Feng, Shuhao Gu, Dengji Guo, Zhengxin Yang, Chenze Shao","year":"2021","venue":"ACL/IJCNLP (1)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gan et al., 2021] Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Lei Xie, and Sanglu Lu. Skeleton-aware neural sign lan- guage translation. In MM, pages 4353-4361, 2021.","title":"Skeleton-aware neural sign language translation","authors":"Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Lei Xie, Sanglu Lu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Graves et al., 2006] Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In ICML, pages 369- 376, 2006.","title":"Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks","authors":"Alex Graves, Santiago Fernández, Faustino Gomez, Jürgen Schmidhuber","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Graves, 2012] Alex Graves. Sequence transduction with re- current neural networks. arXiv preprint arXiv:1211.3711, 2012.","title":"Sequence transduction with recurrent neural networks","authors":"Alex Graves","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Grill et al., 2020] Jean-Bastien Grill, Florian Strub, Flo- rent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhao- han Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learn- ing. NIPS, 33:21271-21284, 2020.","title":"Bootstrap your own latent-a new approach to self-supervised learning","authors":"Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar","year":"2020","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Guo et al., 2019] Dan Guo, Wengang Zhou, Anyang Li, Houqiang Li, and Meng Wang. Hierarchical recurrent deep fusion using adaptive clip summarization for sign language translation. TIP, 29:1575-1590, 2019.","title":"Hierarchical recurrent deep fusion using adaptive clip summarization for sign language translation","authors":"Dan Guo, Wengang Zhou, Anyang Li, Houqiang Li, Meng Wang","year":"2019","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Hadsell et al., 2006] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, volume 2, pages 1735-1742. IEEE, 2006.","title":"Dimensionality reduction by learning an invariant mapping","authors":"Raia Hadsell, Sumit Chopra, Yann Lecun","year":"2006","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Hao et al., 2021] Aiming Hao, Yuecong Min, and Xilin Chen. Self-mutual distillation learning for continuous sign language recognition. In ICCV, pages 11303-11312, 2021.","title":"Self-mutual distillation learning for continuous sign language recognition","authors":"Hao","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for un- supervised visual representation learning. In CVPR, pages 9729-9738, 2020.","title":"Momentum contrast for unsupervised visual representation learning","authors":"He","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[He et al., 2021] Tianxing He, Jingzhao Zhang, Zhiming Zhou, and James Glass. Exposure bias versus self- recovery: Are distortions really incremental for autore- gressive text generation? In Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Pro- cessing, pages 5087-5102, 2021.","title":"Exposure bias versus selfrecovery: Are distortions really incremental for autoregressive text generation?","authors":"Tianxing He, Jingzhao Zhang, Zhiming Zhou, James Glass","year":"2021","venue":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[He et al., 2022] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In CVPR, pages 16000-16009, 2022.","title":"Masked autoencoders are scalable vision learners","authors":"He","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Huang et al., 2018] Jie Huang, Wengang Zhou, Qilin Zhang, Houqiang Li, and Weiping Li. Video-based sign language recognition without temporal segmentation. In AAAI, 2018.","title":"Video-based sign language recognition without temporal segmentation","authors":"Jie Huang, Wengang Zhou, Qilin Zhang, Houqiang Li, Weiping Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Jin and Zhao, 2021] Tao Jin and Zhou Zhao. Contrastive disentangled meta-learning for signer-independent sign language translation. In MM, pages 5065-5073, 2021.","title":"Contrastive disentangled meta-learning for signer-independent sign language translation","authors":"Jin, Zhao ; ] Tao, Jin, Zhou Zhao","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Kan et al., 2022] Jichao Kan, Kun Hu, Markus Hagenbuch- ner, Ah Chung Tsoi, Mohammed Bennamoun, and Zhiy- ong Wang. Sign language translation with hierarchical spatio-temporal graph neural network. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 3367-3376, 2022.","title":"Sign language translation with hierarchical spatio-temporal graph neural network","authors":"Kan","year":"2022","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Koller et al., 2015] Oscar Koller, Jens Forster, and Hermann Ney. Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multi- ple signers. CVIU, 141:108-125, December 2015.","title":"Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers","authors":"Koller","year":"2015","venue":"CVIU","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Koller et al., 2016a] Oscar Koller, Hermann Ney, and Richard Bowden. Deep hand: How to train a cnn on 1 mil- lion hand images when your data is continuous and weakly labelled. In CVPR, pages 3793-3802, 2016.","title":"Deep hand: How to train a cnn on 1 million hand images when your data is continuous and weakly labelled","authors":"Koller","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Koller et al., 2016b] Oscar Koller, O Zargaran, Hermann Ney, and Richard Bowden. Deep sign: Hybrid cnn-hmm for continuous sign language recognition. In Proceedings of the British Machine Vision Conference 2016, 2016.","title":"Deep sign: Hybrid cnn-hmm for continuous sign language recognition","authors":"Koller","year":"2016","venue":"Proceedings of the British Machine Vision Conference 2016","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Koller et al., 2017] Oscar Koller, Sepehr Zargaran, and Her- mann Ney. Re-sign: Re-aligned end-to-end sequence mod- elling with deep recurrent cnn-hmms. In CVPR, pages 4297-4305, 2017.","title":"Re-sign: Re-aligned end-to-end sequence modelling with deep recurrent cnn-hmms","authors":"Koller","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Koller et al., 2019] Oscar Koller, Cihan Camgoz, Hermann Ney, and Richard Bowden. Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential parallelism in sign language videos. TPAMI, 2019.","title":"Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential parallelism in sign language videos","authors":"Koller","year":"2019","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Li et al., 2020] Dongxu Li, Chenchen Xu, Xin Yu, Kaihao Zhang, Benjamin Swift, Hanna Suominen, and Hongdong Li. Tspnet: Hierarchical feature learning via temporal se- mantic pyramid for sign language translation. In NIPS, volume 33, 2020.","title":"Tspnet: Hierarchical feature learning via temporal semantic pyramid for sign language translation","authors":"Li","year":"2020","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.","title":"Rouge: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text summarization branches out","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Min et al., 2021] Yuecong Min, Aiming Hao, Xiujuan Chai, and Xilin Chen. Visual alignment constraint for continu- ous sign language recognition. In ICCV, pages 11542- 11551, 2021.","title":"Visual alignment constraint for continuous sign language recognition","authors":"Min","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Niu and Mak, 2020] Zhe Niu and Brian Mak. Stochastic fine-grained labeling of multi-state sign glosses for contin- uous sign language recognition. In ECCV, pages 172-186. Springer, 2020.","title":"Stochastic fine-grained labeling of multi-state sign glosses for continuous sign language recognition","authors":"Zhe Niu, Brian Mak","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Orbay and Akarun, 2020] Alptekin Orbay and Lale Akarun. Neural sign language translation by learning tokenization. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pages 222-228. IEEE, 2020.","title":"Alptekin Orbay and Lale Akarun. Neural sign language translation by learning tokenization","authors":"Akarun Orbay","year":"2020","venue":"15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In ACL, pages 311-318, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Pu et al., 2018] Junfu Pu, Wengang Zhou, and Houqiang Li. Dilated convolutional network with iterative optimization for continuous sign language recognition. In IJCAI, vol- ume 3, page 7, 2018.","title":"Dilated convolutional network with iterative optimization for continuous sign language recognition","authors":"Pu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/123"},{"position":41,"raw":"[Pu et al., 2019] Junfu Pu, Wengang Zhou, and Houqiang Li. Iterative alignment network for continuous sign language recognition. In CVPR, pages 4165-4174, 2019.","title":"Iterative alignment network for continuous sign language recognition","authors":"Pu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Pu et al., 2020] Junfu Pu, Wengang Zhou, Hezhen Hu, and Houqiang Li. Boosting continuous sign language recogni- tion via cross modality augmentation. In MM, pages 1497- 1505, 2020.","title":"Boosting continuous sign language recognition via cross modality augmentation","authors":"Pu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Shi et al., 2018] Zhan Shi, Xinchi Chen, Xipeng Qiu, and Xuanjing Huang. Toward diverse text generation with in- verse reinforcement learning. IJCAI'18, page 4361-4367. AAAI Press, 2018.","title":"Toward diverse text generation with in-verse reinforcement learning","authors":"Zhan Shi, Xinchi Chen, Xipeng Qiu, Xuanjing Huang","year":"2018","venue":"IJCAI'18","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Subramanian et al., 2017] Sandeep Subramanian, Sai Ra- jeswar, Francis Dutil, Chris Pal, and Aaron Courville. Ad- versarial generation of natural language. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 241-251, Vancouver, Canada, August 2017. Asso- ciation for Computational Linguistics.","title":"Adversarial generation of natural language","authors":"Sandeep Subramanian, Sai Rajeswar, Francis Dutil, Chris Pal, Aaron Courville","year":"2017","venue":"Proceedings of the 2nd Workshop on Representation Learning for NLP","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Tang et al., 2021] Shengeng Tang, Dan Guo, Richang Hong, and Meng Wang. Graph-based multimodal sequen- tial embedding for sign language translation. TMM, 2021.","title":"Graph-based multimodal sequential embedding for sign language translation","authors":"Tang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Wei et al., 2020] Chengcheng Wei, Jian Zhao, Wengang Zhou, and Houqiang Li. Semantic boundary detection with reinforcement learning for continuous sign language recognition. TCSVT, 31(3):1138-1149, 2020.","title":"Semantic boundary detection with reinforcement learning for continuous sign language recognition","authors":"Wei","year":"2020","venue":"TCSVT","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yang et al., 2016] Wenwen Yang, Jinxu Tao, and Zhongfu Ye. Continuous sign language recognition using level building based on fast hidden markov model. Pattern Recognition Letters, 78:28-35, 2016.","title":"Continuous sign language recognition using level building based on fast hidden markov model","authors":"Wenwen Yang, Jinxu Tao, Zhongfu Ye","year":"2016","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Yin and Read, 2020] Kayo Yin and Jesse Read. Better sign language translation with STMC-transformer. In COL- ING, pages 5975-5989, Barcelona, Spain (Online), De- cember 2020.","title":"Kayo Yin and Jesse Read. Better sign language translation with STMC-transformer","authors":"Read Yin","year":"2020","venue":"COL-ING","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Yin et al., 2021] Aoxiong Yin, Zhou Zhao, Jinglin Liu, Weike Jin, Meng Zhang, Xingshan Zeng, and Xiaofei He. Simulslt: End-to-end simultaneous sign language transla- tion. In MM, pages 4118-4127, 2021.","title":"Simulslt: End-to-end simultaneous sign language translation","authors":"Aoxiong Yin, Zhou Zhao, Jinglin Liu, Weike Jin, Meng Zhang, Xingshan Zeng, Xiaofei He","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zhang et al., 2019] Zhihao Zhang, Junfu Pu, Liansheng Zhuang, Wengang Zhou, and Houqiang Li. Continuous sign language recognition via reinforcement learning. In ICIP, pages 285-289. IEEE, 2019.","title":"Continuous sign language recognition via reinforcement learning","authors":"Zhihao Zhang, Junfu Pu, Liansheng Zhuang, Wengang Zhou, Houqiang Li","year":"2019","venue":"ICIP","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhou et al., 2019] Hao Zhou, Wengang Zhou, and Houqiang Li. Dynamic pseudo label decoding for continuous sign language recognition. In ICME, pages 1282-1287. IEEE, 2019.","title":"Dynamic pseudo label decoding for continuous sign language recognition","authors":"Hao Zhou, Wengang Zhou, Houqiang Li","year":"2019","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhou et al., 2020] Hao Zhou, Wengang Zhou, Yun Zhou, and Houqiang Li. Spatial-temporal multi-cue network for continuous sign language recognition. In AAAI, pages 13009-13016, 2020.","title":"Spatial-temporal multi-cue network for continuous sign language recognition","authors":"Hao Zhou, Wengang Zhou, Yun Zhou, Houqiang Li","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Zhou et al., 2021a] Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. Improving sign language translation with monolingual data by sign back-translation. In CVPR, pages 1316-1325, 2021.","title":"Improving sign language translation with monolingual data by sign back-translation","authors":"Zhou","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Zhou et al., 2021b] Hao Zhou, Wengang Zhou, Yun Zhou, and Houqiang Li. Spatial-temporal multi-cue network for sign language recognition and translation. TMM, 2021.","title":"Spatial-temporal multi-cue network for sign language recognition and translation","authors":"Zhou","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Zuo and Mak, 2022] Ronglai Zuo and Brian Mak. C2slr: Consistency-enhanced continuous sign language recogni- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5131- 5140, 2022.","title":"Ronglai Zuo and Brian Mak. C2slr: Consistency-enhanced continuous sign language recognition","authors":"Mak Zuo","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}