Polygon-Net: A General Framework for Jointly Boosting Multiple Unsupervised Neural Machine Translation Models

1 Introduction

Neural machine translationhas achieved great success, especially on the majority language pairs [Hassan et al., 2018]. To achieve good performance, large-scale labeled bilingual training corpus are required, since neural translation models usually have large numbers of parameters to be trained. However, these methods would lose their power for the lowresource languages that does not have enough parallel corpus and the zero-resource languages that have no parallel corpus.

Notice that monolingual data are very easy to obtain, and even for the low-resource or zero-resource languages, there are rich monolingual corpus on the internet and publications. Therefore, how to leverage monolingual corpus to improve neural machine translation models attracts more and more attention. Recently, the study of unsupervised machine translation [Carbonell et al., 2006; Ravi and Knight, 2011; Firat et al., 2016] provides a new approach for the translation of minority languages.

Core techniques of current top unsupervised NMT systems usually consist of two parts. The first part is to leverage prior knowledge of the two languages to get initial models so as to boost the training process, and the second part is to construct training objectives and principles for the unsupervised learning problem. The challenge is that we cannot explicitly optimize the model like in supervised settings without parallel data. The most popular and effective method is backtranslation [Sennrich et al., 2016a], which produces pseudo training pairs by mapping sentences in the target language space to the source language space. Then the outputs together with the source sentences forming as pairs are used as parallel data to train the translation model. In short, back-translation leverages the circuits of Source → Target → Source and Target → Source → Target to design supervisory signals.

The training objectives for unsupervised NMT are usually based on the duality of the Source → Target and Target → Source translation models, while the test is evaluated by the quality of the translations. A potential problem is the gaps between the optimization objectives and the true application scenario. Figure 1(a) illustrate that given the full hypothesis space F, the subspace constrained by the indirect training objectives could be very large. In that case, a fully converged translation model f1 optimized by the indirect objectives could be far from the golden translation model f ∗ . 1 Intuitively, if we add more constraints to the search space, it is more easy to search a better f2 which is closer to f ∗ .

We try to address this problem from a broader perspective. In addition to the two languages we are focusing on, many other languages also have monolingual corpus. We consider using monolingual corpus from other languages to provide more objectives and improve the performance of the translation models. In particular, We embed languages and translation models into a polygon where each node represents a language and each edge represents a translation model. We introduce new objectives and call the graph Polygon-Net.

1 An extreme case could be the translation mismatch. For example, the word "one" in the source language is translated to "2" in the target language, and the word "2" in the target language is translated to "one" in the source language, which satisfies the training principles but performs bad on test set.

Intuitively, a source sentence translated into the same target language along different routes (i.e. different pipelines of translation models) should be (semantically) consistent with each other. Inspired by that, we designed new unsupervised objectives by the probabilistic relations of different path using monolingual corpus other than the original two languages. A technical challenge of computing the objective is the infinite enumerations in the calculation of the probability of each route. We address this challenge by estimating the loss using importance sampling trick. In addition, since the feedback signals among multi-languages are very complicated, we reduce the multi-language situation to some trilingual problems. As a result, the models in the system are updated iteratively, so that multiple models can boost each other, and multiple unsupervised translation systems are combined to train to achieve better performance.

The main contributions of this paper can be summarized as follows: (1) We embed multi languages in to a system called Polygon-Net, and design novel objectives for multi languages using monolingual corpus to jointly boost multiple translation models. (2) To address the challenge of infinite enumeration in computing the objective, we propose to estimate the loss through importance sampling. (3) Experiments on UN Corpus and WMT dataset demonstrates the improvements of our method over baseline models built on two languages.

2 RelatedWork

Our work is related to studies on unsupervised NMT with monolingual data only. Recent top systems includes [Artetxe et al., 2018; Yang et al., 2018; Lample et al., 2018a; Lample et al., 2018b]. They first leverage prior knowledge of the two languages to get initial models which roughly map the source to the target language space. Common techniques includes sharing vocabularies or sub-words of the two languages to get a rough translation model, sharing encoders and decoders, to achieve semantic consistency, and using denoised language model as data-driven prior to the target sentences. After that they construct training objectives and principles for the unsupervised problem using back-translation. Our work is different from them since they all focus on two languages while we propose a method to introduce the monolingual corpus of third-party languages to obtain better models.

There are also some studies on multilingual NMT [Johnson et al., 2017] and design training objectives with auxiliary languages [Ren et al., 2018; Chen et al., 2017]. But they require either parallel data or pre-given translation models, which is different from our setting of using monolingual data only.

3 Model Description

Given a source language space X and a target language space Y , a translation model from language X to Y (denoted by HXY ) is usually represented by a conditional distribution P(y|x; hXY ), where x and y are sentences from language space X and Y respectively, and hXY is the model parameter. Most existing unsupervised neural machine translation methods only consider monolingual corpus of the source and target languages to learn the model. However, we show that the monolingual corpus of third-party languages can be introduced to obtain a better model.

We organize this section as follows. First, we give a brief introduction of the traditional bilingual unsupervised machine translation. Then we show how to introduce only one auxiliary language, i.e. three languages in the system in total, to do the unsupervised machine translation. Specifically, we introduce the objective function induced by multi-path feedback signals, importance sampling estimation for dealing with infinite enumerations, and round training strategy for updating all the models. Finally, we prove that training a system with more than three languages can be simplified to the previously described three-language situation. Based on that, the whole training process of Polygon-Net in a multilingual situation is introduced.

3.1 Traditional Bilingual Unsupervised NMT

The basic idea of traditional bilingual unsupervised NMT is that, if we input a sentence x to the translator from language X to Y along with the translator from language Y to X in the pipeline, the output should be (semantically) consistent with the original sentence x. Based on the above intuition, [Lample et al., 2018b] design a specific bilingual loss, which consists of two parts.

The first part is back-translation loss, defined in Eqn. (1), where H∗XY (x) = argmaxy P(y|x; hXY ) and H∗YX (y) = argmaxx P(x|y; hYX). Back-translation loss characterizes the disagreement between the original sentence and the backtranslation sentence. Note that unlike parameter hXY , the order of subscript in loss does not matter.

Language model loss is the other part of bilingual loss, which is applied to work as data-driven prior to the target sentences:

where M is a pre-determined noise model with some words dropped and swapped, and hXX (hY Y ) is the parameter of model which combines hXY 's (hYX's) encoder and hYX's (hXY 's) decoder. Then the final bilingual loss is defined as

where β is a hyperparameter controlling the tradeoff between the back-translation loss and the language modeling loss.

3.2 Trilingual Training

We now introduce the auxiliary languages to help doing the unsupervised NMT. We first study the simplest case (only using one auxiliary languages) in this section, and leave the multi-auxiliary-language case in Section 3.3.

Suppose that X and Y are the languages of our interests, and the auxiliary language is denoted by Z. Let us consider two semantic paths: Z → Y and Z → X → Y as shown in Figure 2. Intuitively, a sentence translated into the same final language through different paths should get results (semantically) consistent with each other. In particular, for each source sentence z ∈ Z and each target sentence y ∈ Y , we have

In practice, the above relation should approximately hold for good translation models. Thus we can define the following loss.

We remark that the expectation of y can be approximately calculated by taking the average of one or more random sampled sentences from distribution P (y|z; hZY ).

Importance Sampling

Since language space X is too large, it is impossible to directly compute the sum of P(x|z; hZX)P(y|x; hXY ) in Eqn. (5). A naive way to address this issue is to build an approximate estimator by sampling enough sentences from the entire language space. However, for most x ∈ X, the value of P (x|z; hZX) and P (y|x; hXY ) would be almost zero. This is because, only a few sentences from Y or Z are semantically similar to a certain sentence x. To overcome the above issue, we make an identity transform as follows:

where K is the number of samples and xi is the sample drawn from distribution P(x|y; hYX). In this way, the expectation in Eqn. (6) can be approximately calculated by taking the average of samples's values as Eqn. (7). In other words, we use model HY X to sample sentences in X which are strongly related to y and z semantically, such that the probability value of P(x|z; hZX)P(y|x; f) in Eqn. (6) is large enough.

This procedure is exactly the technique of importance sampling [Hesterberg, 1988; Hesterberg, 1995]. Therefore, the loss function in Eqn. (5) can be revised as

where xi is sampled from distribution P(x|y; hY X).

We remark that, in the training process, one can compute ∇hXY Ltri(1) Z ,∇hZX Ltri(1) Z and ∇hZY Ltri(1) Z respectively and update models hXY , hZX and hZY iteratively to make the training process more stable.

Similarly, we can define the symmetrical loss as

where yi is sampled from distribution P(y|x; hXY ), and K is also the number of samples.

The total loss of using Z as the auxiliary language combines the two parts of loss together:

In the same way, we can also define Ltri X and Ltri Y .

Round Training Strategy

With the help of the auxiliary language, the unsupervised learning feedback signal is added to improve the performance of HXY and HYX. In the meanwhile, HZX and HZY are also improved. Here we propose round training strategy, which takes X, Y and Z as the auxiliary language in turns and iteratively updates all the models in the system, so that the models can help each other to improve the performance.

The final objective function for trilingual system is

where γ is used to balance bilingual and trilingual loss.

3.3 Polygon-Net: Extension toMore Languages

In this subsection, we extend one auxiliary language to multi auxiliary languages to improve the unsupervised NMT models. We construct a directed graph to illustrate our framework (called Polygon-Net).

As shown in Figure 3 (a), each node represents a language and each directed edge represents a translation model from the starting point to the ending point. By the same idea of Eqn. (4), given a source sentence zi ∈ Zi and a target sentence zj ∈ Zj where Zi and Zj are two languages, the translations through two different paths should have the same result as shown in Figure 3 (b). Thus we could define a loss to characterize the difference between every two paths like Eqn. (5). However, the computational complexity would be high. In particular, if we take sentence z1 ∈ Z1 as the source sentence and zn ∈ Zn as the target sentence, then the probability of getting zn from z1 through any semantic path Z1 → Z2 → · · · → Zn is calculated by

The computational complexity is O(∏n-1 i=2 |Zi|) where | · | is the cardinality. Moreover, the computational complexity increases exponentially with the number of nodes.

Algorithm 1: Multilingual Training Process for Polygon-Net

• 1 Require Monolingual corpus of n languages Z1, · · · , Zn, sample size K.

2 repeat

• 3 Sample three languages X, Y , and Z from {Z1, Z2, · · · , Zn}, and get three mini-batches of monolingual sentences from each language;
• 4 For each sentence y in the mini-batch of language Y , sample K sentences ˆ x1, ˆ x2, · · · , ˆ xK according to translation model hY X;
• 5 For each sentence x in the mini-batch of language X, sample K sentences ˆ y1, ˆ y2, · · · , ˆ yK according to translation model hXY ;
• 6 Calculate the training objective LXYZ according to Eqn. (9) and compute the gradient of LXYZ with respect to hXY ,hYX,hZX and hZY respectively;
• 7 Update the models: hXY ← hXY -α∇hXY LXYZ hY X ← hY X - α∇hY X LXYZ hZX ← hZX - α∇hZX LXYZ hZY ← hZY -α∇hZY LXYZ

8 until model converges;

Instead, we can sum up the loss for each language triplet to be the loss for Polygon-Net as in Figure (4). There are several advantages for the above loss. First, Section 3.2 can be directly applied to calculate the loss. Second, the computational complexity is much lower. Third, if the above loss is zero, then the loss for any two different paths will also be zero. For example, let us consider two paths: Z1 → Z2 → · · · → Zn and Z1 → Zn. We have

where each equality holds because the loss for each triplet is zero. This means that optimizing the loss for triplets could help optimize the loss for every two paths. Therefore, we use the triplet loss for the Polygon-Net. The final objective function for multilingual system is

where γ is a coefficient to balance bilingual loss and trilingual loss.

4 Experimental Results

We conduct a serious of experiments to evaluate our PolygonNet framework. In this section, we first describe the experimental setups and baseline models. And then, to prove the effectiveness of our method, we apply Polygon-Net framework to the three-language situation and compare the performance with another unsupervised baselines on two datasets. After that, we apply our proposed method to four and five languages to evaluate its generalization ability to larger systems, and analyze the impact of introducing more languages. At last, we make some studies and discussions on the selection of hyperparameters and time cost.

4.1 Settings and Baselines

We compare our proposed Polygon-Net framework with other recent work on unsupervised NMT.

[Artetxe et al., 2018] Unsupervised embedding mappings based encoder-decoder NMT system with denoising and back-translation.

[Lample et al., 2018a] The method takes sentences from monolingual corpora in two different languages and maps them into one common latent space. By learning to reconstruct in both languages from the shared latent feature space, the model effectively learns to translate without using any labeled data.

[Lample et al., 2018b] The model leverages a careful initialization of the parameters, the denoising effect of language models, iterative back-translation and sharing latent representations from two different languages.

Note that the work of [Lample et al., 2018b] is a very strong baseline, which can achieve about 5 to 10 BLUE [Papineni et al., 2002] scores improvement over [Artetxe et al., 2018; Yang et al., 2018; Lample et al., 2018a] on English, French and German language pairs on WMT.

To ensure fair comparison and convincing evaluation, we follow the settings of [Lample et al., 2018b] in our experiments. Specifically, we use Moses scripts [Koehn et al., 2007] for tokenization. We pre-processed the corpus with byte pair encoding (BPE) [Sennrich et al., 2016b] since it has been proved to be an effective approach to handle the large vocabulary issue in NMT. We use NMT models of [Lample et al., 2018b] built on transformer [Vaswani et al., 2017] cells. To be fair, the encoders and decoders of our PolygonNet and baseline pair-wise unsupervised NMT models are all equipped with 6-layer Transformer with word embedding size 512 and hidden state size 512. The sentences come from different language are distinguished with a language identifier at first token. Before feeding corpus to the NMT models, we first learn BPE embeddings from monolingual corpora in different languages jointly by fastText [Bojanowski et al., 2017], the joint BPE embeddings are used for initializing NMT models. All encoders and decoders are shared across the two languages. Adam optimizer with an initial learning rate 10-4 , β1 = 0.5, and a batch size of 32 is applied to all experiments.

4.2 Polygon-Net on Three Languages

In order to verify our proposed multilingual machine translation training framework, we conduct experiments on two different multilingual datasets:

UN Corpus [Ziemski et al., 2016] Parallel corpus composed of United Nations documents. We consider three language pairs: English-French, English-Spanish, French-Spanish. We randomly shuffle the UN corpus and then sample 10 million sentences to construct monolingual corpus of English (en), French (fr), and Spanish (es) respectively. Standard test sets of UN corpus are used for evaluation.

WMT [Bojar et al., 2017] We consider three language pairs: English-French, English-German, French-German. For each language of English (en), French (fr), and German (de), we randomly selected 10 million sentences from WMT monolingual News Crawl datasets from years 2007 through 2010. Since there is no test set for French-German after 2013, we use the test set of WMT'13 for evaluation. Considering [Lample et al., 2018b] reported their results on newstest 2014 for en-fr, and newstest 2016 for en-de, to ensure the credibility of our reproduced baseline [Lample et al., 2018b], we compared our results with the original reported results on these test sets, which is 17.5 (our implementation) vs. 17.2 (reported) for en → de, 21.2 vs. 21.0 for de → en, 24.2 vs. 24.2 for fr → en and 24.8 vs. 25.1 for en → fr.

In Table 1, we report the performance of different unsupervised NMT methods. From the figure we can observe that Polygon-Net outperforms all the baseline models on all language pairs on two datasets. In most of the 12 groups of results, Polygen-Net achieves more than 1 point improvement over the strongest bilingual baseline. The results show that with the help of introducing auxiliary languages, PolygonNet perform better than methods trained with objectives of two languages, which agrees with our intuition that the constraints from other languages can enhance the performance of target bilingual translation.

4.3 Expand toMore Languages

After demonstrating the effectiveness of Polygon-Net on trilingual NMT task, we try to introduce more languages to prove the generality of Polygon-Net further. We consider the other two monolingual corpus in Spanish (es) and Czech (cs) from WMT News Crawl datasets. For Spanish, the News Crawl dataset from 2007 through 2010 only consists of 3.5 million sentences, thus we augment Spanish corpus with the monolingual dataset from the News Crawl dataset from 2007 through 2012, resulting in 10 million sentences for each language. Similarly, the multilingual-aligned test sets from WMT'12 and WMT'13 are used for evaluation.

We report the impact of introducing 3/4/5 languages to the system in Table 2. The results show that Polygon-Net achieves better results with more languages added to the system, which matches our intuition that providing more constraints and objectives could help improve the performance.

The complete results of five-language Polygon-Net is listed in Table 3. Polygon-Net achieves improvement on all language pairs over the strong bilingual baseline. Another interesting observation is that Polygon-Net is helpful to enhance not only the performance of NMT models among nonmainstream languages like es-cs, but also can improve models among mainstream languages like en - fr thanks to the constraints from non-mainstream languages.

4.4 Discussions

Impact of Sample Size K We discuss the selection of sample size K. We conduct experiments on UN Corpus and report the results in Table 4. By increasing K, Polygon-Net achieves more accurate estimation of Eqn. (6), resulting in better performance. To balance the performance and computation cost, we set k as 2 for all the experiments in this paper Time Cost The mainly extra time cost of Polygon-Net compared to bilingual baseline [Lample et al., 2018b] comes from importance sampling to draw sentences from translation models. However, since we introduce more language to tighten the hypothesis space, according to our experience, all the models in the system can converge with fewer iterations. Empirically, the overall time cost of the three-language PolygonNet is about 1.5 times of bilingual baseline. And the time cost of five-language Polygon-Net is about 2 times of bilingual baseline. Therefore, the time cost is practicable for training multilingual MT systems.

5 Conclusion

We proposed Polygon-Net framework, which leverages multi auxiliary languages for jointly boosting unsupervised neural machine translation models. Experimental results verified the effectiveness of our method.

Acknowledgments

This work is partially supported by National Science Foundation of China (61872201, 61702521, 61602266, U1833114); Science and Technology Development Plan of Tianjin (17JCYBJC15300, 16JCYBJC41900, 18ZXZNGX00140, 18ZXZNGX00200); and the Fundamental Research Funds for the Central Universities and SAFEA: Overseas Young Talents in Cultural and Educational Sector.