{"type":"ScholarlyArticle","title":"Maximum Expected Likelihood Estimation for Zero-resource Neural Machine Translation","authors":[{"name":"Hao Zheng","slug":"hao-zheng","given":"Hao","family":"Zheng","affiliation":"# Beihang University , Beijing , China"},{"name":"Yong Cheng","slug":"yong-cheng","given":"Yong","family":"Cheng","affiliation":"+ Institute for Interdisciplinary Information Sciences , Tsinghua University , Beijing , China"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"† State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology , Tsinghua University , Beijing , China"}],"abstract":"While neural machine translation (NMT) has made remarkable progress in translating a handful of high-resource language pairs recently, parallel corpora are not always available for many zero-resource language pairs. To deal with this problem, we propose an approach to zero-resource NMT via maximum expected likelihood estimation. The basic idea is to maximize the expectation with respect to a pivot-to-source translation model for the intended source-to-target model on a pivot-target parallel corpus. To approximate the expectation, we propose two methods to connect the pivot-to-source and source-to-target models. Experiments on two zero-resource language pairs show that the proposed approach yields substantial gains over baseline methods. We also observe  that when trained jointly with the source-to-target model, the pivot-to-source translation model also obtains improvements over independent training.","doi":"10.24963/ijcai.2017/594","url":"https://ijcai.org/proceedings/2017/594","pdf":"https://ijcai.org/proceedings/2017/0594.pdf","markdown":"https://ijcai.org/proceedings/2017/594.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/594","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":594,"pages":{"first":4251,"last":4257},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Machine Translation"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2007] Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. Greedy layer-wise train- ing of deep networks. In Proceedings of NIPS. 2007.","title":"Greedy layer-wise training of deep networks","authors":"Yoshua Bengio, Pascal Lamblin, Dan Popovici, Hugo Larochelle","year":"2007","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bertoldi et al., 2008] Nicola Bertoldi, Madalina Barbaiani, Marcello Federico, and Roldano Cattoni. Phrase-based statistical machine translation with pivot languages. In IWSLT, 2008.","title":"Phrase-based statistical machine translation with pivot languages","authors":"Nicola Bertoldi, Madalina Barbaiani, Marcello Federico, Roldano Cattoni","year":"2008","venue":"IWSLT","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheng et al., 2016] Yong Cheng, Yang Liu, Qian Yang, Maosong Sun, and Wei Xu. Neural machine translation with pivot languages. arXiv preprint arXiv:1611.04928, 2016.","title":"Neural machine translation with pivot languages","authors":"Yong Cheng, Yang Liu, Qian Yang, Maosong Sun, Wei Xu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cohn and Lapata, 2007] Trevor Cohn and Mirella Lapata. Machine translation by triangulation: Making effective use of multi-parallel corpora. In Proceedings of ACL, 2007.","title":"Machine translation by triangulation: Making effective use of multi-parallel corpora","authors":"Trevor Cohn, Mirella Lapata","year":"2007","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Firat et al., 2016] Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, and Kyunghyun Cho. Zero-resource translation with multi-lingual neural ma- chine translation. In Proceedings of EMNLP, 2016.","title":"Zero-resource translation with multi-lingual neural machine translation","authors":"Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T Yarman Vural, Kyunghyun Cho","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Habash and Hu, 2009] Nizar Habash and Jun Hu. Improv- ing arabic-chinese statistical machine translation using en- glish as pivot language. In Proceedings of the Fourth Workshop on Statistical Machine Translation, 2009.","title":"Improving arabic-chinese statistical machine translation using english as pivot language","authors":"Hu Habash, Nizar Habash, Jun Hu","year":"2009","venue":"Proceedings of the Fourth Workshop on Statistical Machine Translation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Johnson et al., 2016] Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. Google's multilingual neural machine translation system: Enabling zero-shot translation. arXiv preprint arXiv:1611.04558, 2016.","title":"Google's multilingual neural machine translation system: Enabling zero-shot translation","authors":"Melvin Johnson, Mike Schuster, V Quoc, Maxim Le, Yonghui Krikun, Zhifeng Wu, Nikhil Chen, Fernanda Thorat, Martin Viégas, Greg Wattenberg, Corrado","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Koehn et al., 2007] Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Con- stantin, and Evan Herbst. Moses: Open source toolkit for statistical machine translation. In Proceedings of ACL, 2007.","title":"Moses: Open source toolkit for statistical machine translation","authors":"Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, Evan Herbst","year":"2007","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kočiský et al., 2016] Tomáš Kočiský, Gábor Melis, Edward Grefenstette, Chris Dyer, Wang Ling, Phil Blunsom, and Karl Moritz Hermann. Semantic parsing with semi- supervised sequential autoencoders. In Proceedings of EMNLP, 2016.","title":"Semantic parsing with semisupervised sequential autoencoders","authors":"Kočiský","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li and Jurafsky, 2016] Jiwei Li and Dan Jurafsky. Mutual information and diverse decoding improve neural machine translation. arXiv preprint arXiv:1601.00372, 2016.","title":"Mutual information and diverse decoding improve neural machine translation","authors":"Jurafsky ; Li, Jiwei Li, Dan Jurafsky","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Nakayama and Nishida, 2016] Hideki Nakayama and Noriki Nishida. Zero-resource machine translation by multimodal encoder-decoder network with multimedia pivot. arXiv preprint arXiv:1611.04503, 2016.","title":"Zero-resource machine translation by multimodal encoder-decoder network with multimedia pivot","authors":"Nishida Nakayama, Hideki Nakayama, Noriki Nishida","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Och and Ney, 2002] Franz Josef Och and Hermann Ney. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of ACL, 2002. [Papineni et al., 2002]","title":"Discriminative training and maximum entropy models for statistical machine translation","authors":"Ney ; Och, Josef Franz, Hermann Och, Ney","year":"2002","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In Proceedings of ACL, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shen et al., 2016] Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. Minimum risk training for neural machine translation. 2016.","title":"Minimum risk training for neural machine translation","authors":"Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, Yang Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Proceedings of NIPS. 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Utiyama and Isahara, 2007] Masao Utiyama and Hitoshi Isahara. A comparison of pivot methods for phrase-based statistical machine translation. In Proceedings of HLT- NAACL, 2007.","title":"A comparison of pivot methods for phrase-based statistical machine translation","authors":"Masao Utiyama, Hitoshi Isahara","year":"2007","venue":"Proceedings of HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wu and Wang, 2007] Hua Wu and Haifeng Wang. Pivot language approach for phrase-based statistical machine translation. Machine Translation, 2007.","title":"Pivot language approach for phrase-based statistical machine translation","authors":"Hua Wu, Haifeng Wang","year":"2007","venue":"Machine Translation","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zoph et al., 2016] Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. Transfer learning for low-resource neu- ral machine translation. In Proceedings of EMNLP, 2016.","title":"Transfer learning for low-resource neural machine translation","authors":"Barret Zoph, Deniz Yuret, Jonathan May, Kevin Knight","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}