Differentially Private Optimal Transport: Application to Domain Adaptation

1 Introduction

Optimal Transport (OT) [Villani, 2008] is a geometric theory that allows us to put a distance (e.g. Wasserstein distance, earth mover distance) on the space of probability measures. As probability measures occur in many scenarios in machine learning, OT has received a lot of attention during the past few years in this community [Cuturi, 2013; Frogner et al., 2015; Arjovsky et al., 2017]. For instance, probability measures take the form of color histograms when comparing images, bags of words in document analysis, activation maps in brain imaging, etc. Perhaps, the most exciting recent application of OT in machine learning comes from the wide use of generative models and the need of some notion of distance to measure the divergence between generated data and the actual generative model. Very recently, OT has been shown to be a useful and intuitive tool to address domain adaptation tasks [Courty et al., 2017b; Courty et al., 2017a; Shen et al., 2018], where the goal is to transfer knowledge from a source domain to a target domain. In such a scenario, regularized OT aims at finding a transportation of minimal cost aligning labeled source data on unlabeled target examples. Then a classifier can be learned from the source examples and deployed on the target domain. OT for domain adaptation has also received recent attention from a theoretical perspective. In [Redko et al., 2017], the authors show how the Wasserstein distance can be used to derive generalization bounds on the target error. All things considered, OT can be seen nowadays as a theoretically grounded competitive framework to perform transfer learning.

Optimizing the transportation that aligns the source and target distributions boils down to finding a coupling matrix that moves the source data to the target ones and allows us to induce the Wasserstein distance. Therefore, this procedure requires to share data from both sources at some point of the learning process that may appear to be an unacceptable solution when source and/or target data contain sensitive information (e.g. about the medical history of patients). The contributions of this paper lie in the setting of differentially private OT and its application to Domain Adaptation. Differential privacy [Dwork et al., 2006] provides a strong theoretical-grounded guarantee for the privacy of individuals against attackers and has become the standard for formal privacy in machine learning. The basic idea of differential privacy is to introduce randomness in the communication that preserves privacy even against an adversary possessing arbitrary side information and having access to the communication. It turns out that there has been a rich amount of work on differentially private machine learning, such as in logistic regression [Chaudhuri and Monteleoni, 2009], principal component analysis [Hardt and Roth, 2013], boosting [Dwork et al., 2010], support vector machines [Rubinstein et al., 2009], or more recently on deep learning [Abadi et al., 2016; Shokri and Shmatikov, 2015] and semi-supervised deep learning [Papernot et al., 2017]. However, it is worth noting that privacy preserving domain adaptation is surprisingly under-developed. Recently, [Wang et al., 2018] and [Guo et al., 2018] proposed two similar methods, combining hypothesis transfer learning with private logistic regression. However, the first model needs to have access to a publicly available auxiliary dataset as a bridge to transfer knowledge from the source to the target, while the second trains on a fully labeled target data; both are strong constraints that do not hold in the standard (both unsupervised and semi-supervised) domain adaptation setting.

The objective of this paper is to fill the gap mentioned above and address the challenging privacy preserving problem when OT is used to transfer knowledge from a source to a target. To do so, we present two new algorithms. The first one, called Differentially Private Optimal Transport (DPOT), makes use of the Johnson-Lindenstrauss transform [Johnson and Lindenstrauss, 1984] which linearly projects a set of examples onto a small feature space by a random matrix that preserves the pairwise distances. Adapted to the context of OT, DPOT allows us to jointly compute the coupling matrix and the Wasserstein distance between two domains under the differential privacy constraint, by adding noise on the random projection. As far as we know, DPOT is the first differentially private optimal transport algorithm. Experimental results show that DPOT preserves the Wasserstein distance on the space of probability measures compared to a non private OT approach. We then build upon DPOT the first complete differentially private domain adaptation model (DPDA) where the learner can benefit from labeled source data to improve a different but related target task while still ensuring the privacy of each source of data. We provide theoretical guaranty of our method and demonstrate its efficiency empirically on various benchmarks.

The rest of this paper is organized as follows: Section 2 presents the required background in Optimal Transport, Domain Adaptation and Differential Privacy. In Section 3, based on the Johnson-Lindenstrauss transform, we introduce DPOT. Section 4 is devoted to the presentation of DPDA, the first Differentially Private Domain Adaptation algorithm based on Optimal Transport for which differential privacy guarantees are derived. In Section 5, we first show that DPOT allows us to efficiently approximate the true Wasserstein distance that would be obtained without privacy constraint. Then, we report the results of an extensive experimental study performed on various benchmarks (VisDA, Office-Home and OfficeCaltech datasets) that demonstrate the efficiency of DPDA compared to non-private strategies.

2 Background and RelatedWork

2.1 Optimal Transport

Let µ1 and µ2 be two probability measures. Given some cost function c, optimal transport (OT) seeks a transportation plan of minimal cost that moves µ1 to µ2. Let Π be the space of joint probability distributions with marginals µ1 and µ2. In the relaxed formulation of Kantorovitch [1942], the optimal transportation takes the form of a distribution (or coupling) γ0 ∈ Π that minimizes the following quantity:

where c(xs , xt ) is this paper the Euclidean distance between xs ∈ Ωs and xt ∈ Ωt. The quantity W(µ1, µ2) is called the Wasserstein distance between the two distributions µ1 and µ2. When µ1, µ2 are defined as empirical measures between two datasets Xs and Xt, then

where 〈., .〉F is the Frobenius dot product and C is the cost matrix between Xs and Xt. Then, the Wasserstein distance between Xs and Xt is W(Xs,Xt) = 〈γ0,C〉F .

2.2 Domain Adaptation

Domain adaptation [Pan and Yang, 2010] aims at using available labeled data from a source domain (e.g. synthetic data) to facilitate the learning process in a target domain (e.g. real data) with a different underlying distribution. In unsupervised domain adaptation, we assume to have access to only unlabeled target data, while in semi-supervised tasks, a small amount of supervision is available, but not large enough to learn well only from the labeled target data. Most recent state-of-the-art methods either learn a common latent feature space between the two domains, e.g. DANN [Ganin et al., 2016], or map the samples of one domain to the other, e.g. ADDA [Tzeng et al., 2017].

Optimal transport was recently applied successfully to domain adaptation, where the Wasserstein distance is used as a divergence measure between the two domains that a domain adaptation algorithm aims at minimizing. In [Courty et al., 2017b], the source samples are transported to the target domain using the coupling matrix γ0. In [Courty et al., 2017a], the authors optimize both the coupling matrix and a target prediction function, while [Shen et al., 2018] minimize the domain divergence in the latent space using the WassersteinGAN [Arjovsky et al., 2017].

2.3 Differential Privacy

Differential Privacy was introduced by Dwork et. al. [2006] and constitutes nowadays a standard for privacy guarantees.

Definition 1 (Differential Privacy, see eg. [Abadi et al., 2016]). A randomized mechanismM : X n → Rd satisfies (ε, δ)-differential privacy if for any two datasets X, X′ ∈ X n differing by a single element and for any set of possible output O ⊆ Range(M):

There has been a rich amount of work on differentially private machine learning, such as in logistic regression [Chaudhuri and Monteleoni, 2009], principal component analysis [Hardt and Roth, 2013], boosting [Dwork et al., 2010] and support vector machine [Rubinstein et al., 2009]. Private deep learning methods were also introduced recently; [Abadi et al., 2016] provided a differentially private SGD method for training a deep neural network, where at each step, the gradient is clipped and some noise is added to protect privacy. On the other hand, [Shokri and Shmatikov, 2015] proposed a distributed selective SGD: several models are trained independently using SGD and they selectively upload the gradients to a global one. Similarly, [Papernot et al., 2017] train several teachers with private data, then predict pseudo-labels via a majority vote for unlabeled public data that are used by a student to train.

It is worth noticing that privacy preserving domain adaptation has not received much attention. Deep learning methods such as [Abadi et al., 2016; Papernot et al., 2017] are designed for standard classification tasks with the assumption that all data are drawn from the same underlying distribution, which is not the case in domain adaption (although technically, these algorithms can be deployed out of the box for domain adaptation through training a source-only model) Private multiparty learning methods, such as [Pathak et al., 2010; Hamm et al., 2016] could also be applied to semisupervised domain adaptation, but they face two drawbacks: (1) the amount of target labeled data is usually very limited for a meaningful aggregation, and (2) these models assume that the domain distributions are identical and so ignore the domain shift. As already mentioned, [Wang et al., 2018] and [Guo et al., 2018] proposed two methods that combine hypothesis transfer learning with private logistic regression. However, the first model needs to have access to a publicly available auxiliary dataset as a bridge to transfer knowledge from the source to the target, while the second trains on a fully labeled target data; both are strong constraints that do not hold in the standard (both unsupervised and semi-supervised) domain adaptation setting.

2.4 Johnson-Lindenstrauss Transform

Let N(0, σ)k×ℓ be a k × ℓ matrix where each entry is drawn i.i.d. from N(0, σ). A variant of the famous JohnsonLindenstrauss Lemma [Johnson and Lindenstrauss, 1984] states that a linear transformation of a set of n data points to a much smaller subspace by such a random matrix has a high probability to retain the pairwise distances within (1 ± η)- factor for some η ∈ [0, 0.5] (see eg. [Blocki et al., 2012]).

Theorem 1. Let v1, ..., vn be a set of n points in Rk . For any η, ℓ > 0 and an N(0, 1ℓ )k×ℓ matrix M, with probability at least 1- 2 exp (ℓη2 /8) the following holds for every i, j:

The Johnson-Lindenstrauss transform is widely used across many areas of computer science such as computational speedups, machine learning, information retrieval. The interested reader may find an extensive overview of the topic in the monograph written by Vempala [2005]. Due to its randomness, the Johnson-Lindenstrauss transform was also used in differential privacy. [Blocki et al., 2012] showed that it preserves the differential privacy, as long as all singular values of the database are sufficiently large. [Kenthapadi et al., 2013] showed that by simply applying the Johnson-Lindenstrauss transform together with some noise, one can publish the pairwise distances of elements while protecting the privacy of the original data.

As OT resorts to a pairwise distance matrix to find the optimal coupling and the Wasserstein distance, we suggest in the next sections to benefit from the Johnson-Lindenstrauss transform and the idea introduced in [Kenthapadi et al., 2013] to build the first differentially private OT method (DPOT); then, we use it to design a new privacy-preserving domain adaptation algorithm (DPDA).

Algorithm 1 Differentially Private Optimal Transport

Input: Xs, Xt, and σ, ℓ > 0

• 1: Source generates anN(0, 1ℓ )k×ℓ matrixM and anN(0, σ)k×ℓ noise matrix ∆.
• 2: Source sends {M, ˜Xs +∆}, where ˜Xs =XsM
• 3: Target computes ˜ C = c( ˜Xs+∆, ˜Xt)-ℓσ2 where ˜Xt =XtM
• 4: Solve Problem (1) with cost matrix ˜ C and return ˜ γ0 and ˜W (Xs, Xt).

3 Differentially Private Optimal Transport

Let Xs and Xt be a source and a target dataset of size ns × k and nt × k (where k is the number of features) respectively. A standard OT procedure aims at finding the Wasserstein distance W and the empirical transportation plan γ0 between Xs and Xt.

Usually, one has access to both Xs and Xt to calculate the cost matrix C, typically as the matrix of pairwise Euclidean distances c(Xs,Xt) between samples of the source and target sets. Then we solve the optimization Problem (1) to get γ0 and W. However, under privacy constraints, source and target parties may not be willing to release their data, making optimal transport a challenging task. To overcome this problem, let us use the result proved in Kenthapadi et al. [2013] stating that by applying the Johnson-Lindenstrauss transform with some additional Gaussian noise, one can publish the pairwise distances of elements privately (see Kenthapadi et al. [2013] for more details).

We exploit this idea in the context of OT where the objective is to align Xs on Xt. Therefore, the data we want to publish safely here is the source set Xs. Algorithm 1 describes the pseudo-code of our Differentially Private Optimal Transport (DPOT) algorithm. The main steps of DPOT are the following: In Step 1, we generate a random matrix M according to [Johnson and Lindenstrauss, 1984] and a noise matrix ∆ according to [Kenthapadi et al., 2013]. Step 2 boils down to publishing both M and the source data ˜Xs+∆ after a random projection of Xs by M in a subspace of size ℓ and the addition of the noise ∆. To benefit from the privacy guarantees of [Kenthapadi et al., 2013] over the pairwise (source/target) distances, one compute the distance matrix ˜ C in Step 3 between the published source data ˜Xs + ∆ and the projected target examples ˜Xt = XtM. Note that ℓσ2 is subtracted from each entry to cancel the bias caused by ∆ (indeed, σ2 is known by the target party but not ∆). Step 4 is devoted to the resolution of Problem (1).

Kenthapadi et al. [2013], Theorem 1, showed that the mechanism of publishing M and the noisy data ˜Xs + ∆ satisfies the (ϵ, δ)-differential privacy of Definition 1, for any ε, δ > 0 and σ ≥ w √2(ln( 1 2δ )+ε) ε where w = max1≤i≤k (∑ℓj=1 M 2 ij ) 1 2 , that is the ℓ2-norm sensitivity of M, which is tightly concentrated around 1 [Kenthapadi et al., 2013]. Therefore, Algorithm 1 comes directly with the following privacy guarantee.

Theorem 2. Algorithm 1 is (ε, δ)-differentially private for any ε, δ > 0 and σ ≥ w √2(ln( 1 2δ )+ε) ε .

In Section 5, we will report experimental results showing that by satisfying the previous constraint on the noise σ, the Wasserstein distance ˜W(Xs,Xt) computed by Algorithm 1 remains very close to the true W(Xs,Xt), while preserving the privacy of the data. Therefore, we can now benefit from ˜W(Xs,Xt) and ˜ γ0 to transfer knowledge from a source domain to a target one to deal with domain adaptation in a differential privacy setting.

4 Differentially Private Domain Adaptation

Extending DPOT for domain adaptation requires (i) to transport the source data to the target domain thanks to a private barycentric mapping, (ii) to transfer the source labels while ensuring privacy, (iii) to add some appropriate domain adaptation regularizers for computing the coupling matrix.

Barycentric mapping. Once the coupling matrix ˜ γ0 is computed, the source samples can be moved to the target domain using the geodesics of the Wasserstein metric. The barycentric mapping [Reich, 2013] from each source sample xsi ∈ Xs to its corresponding image ̂ xsi in the target domain is as follows

The barycentric image ̂ xsi can be understood as the empirical optimal location on the target domain to transport xsi w.r.t. the coupling matrix ˜ γ0. Let ̂Xs be the set of barycentric images of the source data. It is now possible to train a model from the transported labeled source data ( ̂Xs, Ys) and use it in the target domain [Courty et al., 2017b].

Private barycentric mapping. Generally in domain adaptation, the target agent must have access to the labeled data from the source. However, under privacy constraints, this is not possible anymore. Fortunately, by combining DPOT algorithm with the barycentric mapping, we can privately transport the source data in the target domain. Indeed, as shown in Equation 2, only ˜ γ0 and Xt are involved in the calculation of the barycentric mapping which does not depend on the source data. The private barycentric images can be conveniently rewritten as ̂Xs = ns˜ γ0Xt as shown in [Perrot et al., 2016] in a non private scenario.

Transmitting source labels. To perform domain adaptation in both unsupervised and semi-supervised settings, the target party needs to have access to the source labels Ys. To enhance the privacy of our domain adaptation algorithm, we also aim at adding some noise on the labels. To do so, we follow the principle of Histogram Queries mentioned in [Dwork et al., 2014]: The source first reorders its data (Xs, Ys) such that all samples of label 1 appear first, then samples of label 2 and so on. Hence Ys is now equivalent to a vector v(Ys) of length q counting the number of samples for each label, where q is the number of labels. The source party can now safely publish a noisy version of v(Ys) as a histogram query by adding a Laplacian noise Lap( 1 ε′ )q , which is (ε′ , 0)- differentially private [Dwork et al., 2014].

Regularization techniques. There are various techniques in the literature to regularize the coupling matrix for domain adaptation tasks. In our method, we utilize two of them. The first one is the entropic regularization [Cuturi, 2013] that allows us to transform Problem 1 into a strictly convex problem and is defined as follows:

The second one is the group Lasso regularization [Courty et al., 2017b],

where ‖ · ‖2 is the ℓ2-norm, Is contains the indices of rows in γ related to source samples associated to label s, γ(Is, j) is then a vector containing the coefficients of the j th column of γ associated to label s. This regularization encourages a coupling where a given target sample receives masses from source samples having the same labels. Thus, the coupling matrix is now computed by the following equation:

where λs, λc are hyper-parameters. In the semi-supervised setting, we can further exploit the available target label data by prohibiting masses from being transported between samples with different labels, which does not require any additional hyper-parameter [Courty et al., 2017b].

Algorithm 2 Differentially Private Domain Adaptation

Input: (Xs, Ys), Xt and σ, ε′ , ℓ > 0

• 1: Source reorders (Xs, Ys) and sends noisy v(Ys) +Lap( 1 ε′ )q
• 2: Source runs Steps 1,2 of Algorithm 1 on the ordered Xs
• 3: Target follows Steps 3,4 of Algorithm 1 solving Problem (3) and returns ˜ γ0 and ˜W
• 4: Target computes the barycentric images ̂Xs = ns˜ γ0Xt
• 5: Target trains his own model using ( ̂Xs, ˜ Ys), where ˜ Ys = v(Ys) + Lap( 1 ε′ )q , together with labeled data in Xt (if any)

Differentially Private Domain Adaptation Algorithm. We now have all the necessary blocks for our differentially private domain adaptation algorithm DPDA, see the pseudocode in Algorithm 2, where the two parties collaboratively compute a regularized coupling matrix using Eq. (3), which allows the target to obtain the barycentric images of the source data for his own training. We have shown that both mechanisms of transmitting {M, ˜Xs +∆} and the noisy vector v(Ys) from the source are differentially private. According to the Composition Theorem ([Dwork et al., 2014] Theorem 3.16), the composition of differentially private mechanisms is also differentially private. This means that Algorithm 2 is also differentially private.

The whole workflow of our differentially private method is shown is Figure 2.

5 Experiments

5.1 Private Optimal Transport with DPOT

In this section, we empirically validate our private optimal transport method DPOT by analyzing the relation between the privacy budget (ε, δ) and the accuracy of the Wasserstein distance ˜W computed by DPOT. The error between ˜W and the true Wasserstein distance W is measured here by Err = ∣∣ ˜W-W W ∣∣. We perform an experiment on the OfficeHome dataset [Venkateswara et al., 2017], using two domains with a moderate number of samples, Clipart (4365 samples) and Product (4439 samples). We follow the experimental protocol used in [Abadi et al., 2016] and [Papernot et al., 2017] by setting the privacy budget to δ = 1 1.2ns , where ns is the size of the source domain, and using a some moderate ε with 1-digit.

Besides, it is widely known that minibatch subsampling can enhance privacy [Abadi et al., 2016; Balle et al., 2018]. Recently, [Abadi et al., 2016] introduced the privacy accountant theorem to get a tight bound on the total privacy budget (ε, δ) (i.e. the allowed leakage amount of privacy during the whole training process) when training with multibatches. In the following, we will perform experiments on two settings: whole batch and minibatch to demonstrate the effect of using minibatches on the privacy budget.

Whole batch setting. In the whole batch setting, the noiseratio σw can be calculated directly from the privacy budget (ε, δ) by Theorem 2. For example, when ε = 4, 5, 10, we get σw = 1.25, 1.04, 0.61, respectively. In Figure 2 (left), we plot the evolution of the error Err between ˜W and W given a noise-ratio σw on the two whole datasets (we calculate Err over 20 runs and report the average). Keeping in mind that we want ε as small as possible, the figure shows that Err = 0.08 at ε = 10 (maximum acceptable value according to the literature), while when the privacy budget is reduced to ε = 4, the error reaches 22%. In order to reduce the approximation error, the next experiment shows that using minibatches allows us to drastically reduce the privacy budget.

Minibatch setting. In the minibatch setting, the relationship between the noise-ratio σw and the privacy budget (ε, δ) can be calculated via the privacy accountant [Abadi et al., 2016]. For δ = 1 1.2ns and a minibatch of size 128, the noiseratio σw depends on both ε and the number of minibatches we wish to run. For example, if σw = 1 then for ε = 3, 5, 7 we can run 200, 600, 1200 minibatches respectively, while when σw = 0.7, we can run only 4, 60, 200 batches for ε = 3, 5, 7. We can note in Figure 2 (right) that the error Err of DPOT (average over 1000 randomminibatches), for σw = 0.7, 1, 1.3, is 7%, 10% and 15%, respectively. This is substantially lower than running on the whole batch. Empirically, we found that the effect of the noise-ratio σw on the accuracy of DPOT in the minibatch setting is the same across datasets, but the privacy budget (or the number of allowed minibatches for a given budget) would be different since the size of the dataset plays an important role. For example, as the Synthetic domain of VisDA dataset [Peng et al., 2017] has 150k samples, we can run 10k batches for ε = 1, σw = 1 or 30k batches for ε = 2, σw = 0.8.

5.2 Private Domain Adaptation with DPDA

The previous experimental study showed that minibatches allow us to improve the quality of the outputs of DPOT with a smaller privacy budget. For this reason, the next experiments are performed only in this setting.

Benchmarks. We evaluate our method on three domain adaptation benchmarks from the classical Office-Caltech dataset [Saenko et al., 2010] to the more recent and challenging VisDA [Peng et al., 2017] and Office-Home [Venkateswara et al., 2017] datasets. The three datasets offer a wide range of scenarios: Office-Caltech is a very small dataset with 150-1000 samples per domain; VisDA contains a large collection (200k samples in total) of high quality images, while Office-Home has a moderate size (4000 samples/domain) but many (65) classes. In Office-Caltech and Office-Home, there are four different domains each, coming from different sources, while VisDA is a specific dataset for simulation-to-real adaptation, where the source domain contains 150k synthetic images rendering of 3D models from different angles and the target domain contains 50k samples of natural images. We conduct the experiments on both unsupervised and semi-supervised settings.

Baselines. Our baselines include the state-of-the-art private semi-supervised method PATE [Papernot et al., 2017], where the teachers hold source data and the student holds target data, the non-private optimal transport domain adapation method OTDA [Courty et al., 2017b] which we build our model DPDA upon, and another state-of-the-art non-private domain adaptation method ADDA [Tzeng et al., 2017] using a domain-adversarial technique. We do not add [Wang et al., 2018] and [Guo et al., 2018] as baselines since both only work on binary-classification and the first requires an auxiliary public dataset while the second requires fully labeled target data.

For the Office-Caltech dataset, we use the common DeCAF features as input, while for VisDA and Office-Home, we use the NASNet model [Zoph et al., 2018] with weights pre-trained on ImageNet as a base model to extract features from the images. All methods are written in Keras [Chollet, 2015] with the same target model architecture (a 3-layer neural network) for fair comparison. The coupling matrices are computed using the POT library [Flamary and Courty, 2017]. As already said, we train all the models using minibatches.

Hyper-parameters. For OTDA and our method DPDA, we set the hyper-parameters λe and λg of Eq. (3) to 0.01 and 0.1, respectively. In all benchmarks, we set the dimension of the subspace of our method ℓ = k 10 and the noise-ratio σw = 1.1. For the privacy budget, we again follow the standard of [Abadi et al., 2016; Papernot et al., 2017] by setting δ = 1 1.2ns , ε = 2 for VisDA and ε = 8 for the other datasets, except ε = 20 if the source is DSLR or Webcam in Office-Caltech since they have too few samples (150-200 in total). For PATE and DPDA, we use the privacy accountant tool [Abadi et al., 2016] to keep track the privacy budget after each step as explained in Section 5.1. We run each test 3

times and report the average.

Results. The results are presented in Tables 1, 2 and 3. In both Office-Caltech and VisDA benchmarks, our model DPDA performs at the same level as non-private state-of-theart methods and even gets 1 point higher than ADDA in the VisDA benchmark, while PATE significantly lags behind. If we reduce the privacy budget for the source domains DSLR, Webcam in the Office-Caltech benchmark to ε = 10, then the performances of DPDA and PATE drop by 3-5 points. In the Office-Home dataset, DPDA is 2-3 points lower than non-private baselines but still safely outperforms PATE by 3 points. On the other hand, in the semi-supervised scenario on the Office-Home benchmark , when only 1 labeled target sample per class is allowed, DPDA manages to reduce the gap with OTDA significantly from 3 points (55 vs 58) to 1 point (60 vs 61). This means that DPDA benefits more from some labeled target data than OTDA. Behind the good behavior of DPDA to deal with domain adaptation tasks, the results reported in these tables confirm that DPOT is a very performing differentially private optimal transport method.

6 Conclusion

In this paper, we have proposed the first differentially private approach for optimal transport. The proposed algorithm DPOT is able to preserve the Wasserstein distance despite the noisy perturbations introduced used to ensure differential privacy. We have then designed the first complete differentially private domain adaptation model based upon DPOT. Our methods are justified by strong theoretical guarantees and the experimental evaluations illustrated that our approach allows one to obtain good results while ensuring a high privacy level of the data. Future work includes extensions of our framework to other applications of optimal transport, for instance, in differentially private distributed learning.

Acknowledgements

Work supported by the ACADEMICS grant of the IDEXLYON, project of the Universit´e de Lyon, PIA operated by ANR-16-IDEX-0005