{"type":"ScholarlyArticle","title":"Polygon-Net: A General Framework for Jointly Boosting Multiple Unsupervised Neural Machine Translation Models","authors":[{"name":"Chang Xu","slug":"chang-xu","given":"Chang","family":"Xu","affiliation":"College of Computer and Control Engineering , Nankai University"},{"name":"Tao Qin","slug":"tao-qin","given":"Tao","family":"Qin","affiliation":"Microsoft Research"},{"name":"Gang Wang","slug":"gang-wang","given":"Gang","family":"Wang","affiliation":"College of Computer and Control Engineering , Nankai University"},{"name":"Tie-Yan Liu","slug":"tie-yan-liu","given":"Tie-Yan","family":"Liu","affiliation":"Microsoft Research"}],"abstract":"Neural machine translation (NMT) has achieved great success. However, collecting large-scale parallel data for training is costly and laborious.  Recently, unsupervised neural machine translation has attracted more and more attention, due to its demand for monolingual corpus only, which is common and easy to obtain, and its great potentials for the low-resource or even zero-resource machine translation. In this work, we propose a general framework called Polygon-Net, which leverages multi auxiliary languages for jointly boosting unsupervised neural machine translation models. Specifically, we design a novel loss function for multi-language unsupervised neural machine translation. In addition, different from the literature that just updating one or two models individually, Polygon-Net enables multiple unsupervised models in the framework to update in turn and enhance each other for the first time. In this way, multiple unsupervised translation models are associated with each other for training to achieve better performance. Experiments on the benchmark datasets including UN Corpus and WMT show that our approach significantly improves over the two-language based methods, and achieves better performance with more languages introduced to the framework.","doi":"10.24963/ijcai.2019/739","url":"https://ijcai.org/proceedings/2019/739","pdf":"https://ijcai.org/proceedings/2019/0739.pdf","markdown":"https://ijcai.org/proceedings/2019/739.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/739","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":739,"pages":{"first":5320,"last":5326},"keywords":["Machine Learning: Deep Learning","Natural Language Processing: Machine Translation","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Artetxe et al., 2018] Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural ma- chine translation. In Proceedings of the Sixth International Conference on Learning Representations, April 2018.","title":"Unsupervised neural machine translation","authors":"Mikel Artetxe, Gorka Labaka, Eneko Agirre, Kyunghyun Cho","year":"2018","venue":"Proceedings of the Sixth International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bojanowski et al., 2017] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vec- tors with subword information. Transactions of the Asso- ciation for Computational Linguistics, 5:135-146, 2017.","title":"Enriching word vectors with subword information","authors":"Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov","year":"2017","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bojar et al., 2017] Ondřej Bojar, Rajen Chatterjee, Chris- tian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, et al. Findings of the 2017 conference on machine translation (wmt17). In Proceedings of the Sec- ond Conference on Machine Translation, pages 169-214, 2017.","title":"Findings of the 2017 conference on machine translation (wmt17)","authors":"Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva","year":"2017","venue":"Proceedings of the Second Conference on Machine Translation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carbonell et al., 2006] Jaime G Carbonell, Steve Klein, David Miller, Mike Steinbaum, Tomer Grassiany, and Jochen Frey. Context-based machine translation. 2006.","title":"Context-based machine translation","authors":"Steve Jaime G Carbonell, David Klein, Mike Miller, Tomer Steinbaum, Jochen Grassiany, Frey","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2017] Yun Chen, Yang Liu, Yong Cheng, and Victor O.K. Li. A teacher-student framework for zero- resource neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1925-1935, 2017.","title":"A teacher-student framework for zeroresource neural machine translation","authors":"Yun Chen, Yang Liu, Yong Cheng, O K Victor, Li","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Firat et al., 2016] Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, and Kyunghyun Cho. Zero-resource translation with multi-lingual neural ma- chine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 268-277, 2016.","title":"Zero-resource translation with multi-lingual neural machine translation","authors":"Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T Yarman Vural, Kyunghyun Cho","year":"2016","venue":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hassan et al., 2018] Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Fe- dermann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, et al. Achieving human parity on automatic chinese to english news translation. arXiv preprint arXiv:1803.05567, 2018.","title":"Achieving human parity on automatic chinese to english news translation","authors":"Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hesterberg, 1988] Timothy Classen Hesterberg. Advances in importance sampling. PhD thesis, Stanford University, 1988.","title":"Advances in importance sampling","authors":"Timothy Classen, Hesterberg","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hesterberg, 1995] Tim Hesterberg. Weighted average im- portance sampling and defensive mixture distributions. Technometrics, 37(2):185-194, 1995.","title":"Weighted average importance sampling and defensive mixture distributions","authors":"Tim Hesterberg","year":"1995","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Johnson et al., 2017] Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. Google's multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5:339-351, 2017.","title":"Google's multilingual neural machine translation system: Enabling zero-shot translation","authors":"Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean","year":"2017","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koehn et al., 2007] Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Con- stantin, and Evan Herbst. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pages 177-180. Association for Computational Linguistics, 2007.","title":"Moses: Open source toolkit for statistical machine translation","authors":"Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, Evan Herbst","year":"2007","venue":"Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lample et al., 2018a] Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. Unsuper- vised machine translation using monolingual corpora only. In Proceedings of the Sixth International Conference on Learning Representations, April 2018.","title":"Unsupervised machine translation using monolingual corpora only","authors":"Lample","year":"2018","venue":"Proceedings of the Sixth International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lample et al., 2018b] Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. Phrase-based & neural unsupervised machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5039- 5049, 2018.","title":"Phrase-based & neural unsupervised machine translation","authors":"Lample","year":"2018","venue":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for auto- matic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311-318. Association for Computational Linguistics, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of the 40th annual meeting on association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ravi and Knight, 2011] Sujith Ravi and Kevin Knight. De- ciphering foreign language. In Proceedings of the 49th Annual Meeting of the Association for Computational Lin- guistics: Human Language Technologies, pages 12-21, 2011.","title":"Deciphering foreign language","authors":"Sujith Ravi, Kevin Knight","year":"2011","venue":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ren et al., 2018] Shuo Ren, Wenhu Chen, Shujie Liu, Mu Li, Ming Zhou, and Shuai Ma. Triangular architec- ture for rare language translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 56-65, 2018.","title":"Triangular architecture for rare language translation","authors":"Wenhu Shuo Ren, Shujie Chen, Mu Liu, Ming Li, Shuai Zhou, Ma","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sennrich et al., 2016a] Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 86-96, 2016.","title":"Improving neural machine translation models with monolingual data","authors":"Sennrich","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sennrich et al., 2016b] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715-1725. Association for Computational Linguistics, 2016.","title":"Neural machine translation of rare words with subword units","authors":"Sennrich","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Sys- tems 30, pages 5998-6008. 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł Ukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems 30","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2018] Zhen Yang, Wei Chen, Feng Wang, and Bo Xu. Unsupervised neural machine translation with weight sharing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46-55, 2018.","title":"Unsupervised neural machine translation with weight sharing","authors":"Zhen Yang, Wei Chen, Feng Wang, Bo Xu","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ziemski et al., 2016] Michal Ziemski, Marcin Junczys- Dowmunt, and Bruno Pouliquen. The united nations par- allel corpus v1. 0. In LREC, 2016.","title":"The united nations parallel corpus v1. 0","authors":"Michal Ziemski, Marcin Junczys-Dowmunt, Bruno Pouliquen","year":"2016","venue":"LREC","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}