Towards Accurate and Robust Domain Adaptation under Noisy Environments

1 Introduction

Conventional learning theories assume that the learning machines are under a static environment where we draw training and test examples from an identical distribution. If the training and test distributions substantially differ, the learning machines would probably lose generalization ability [Valiant, 1984]. However, we expect our learning machines can perform well in crucial non-stationary environments where the test domain is similar yet distinct from the training domain. Unsupervised domain adaptation is the key machine learning topic to deal with this problem [Ben-David et al., 2007].

Prominent theoretical advance and effective algorithm have been achieved in domain adaptation. Ben-David et al. [2007; 2010a] propose a pioneering H∆H distance and give rigorous generalization bounds to measure the distribution shift.

∗Co-corresponding author

A series of seminal studies then extend the H∆H distance to discrepancy distance [Mansour et al., 2009], to domain disagreement distance [Germain et al., 2013], to generalized discrepancy distance [Cortes et al., 2019], or to margin disparity discrepancy [Zhang et al., 2019], etc. With significant theoretical findings, domain adaptation algorithms have made significant advances. Previous studies have explored various techniques concerning statistic moment matchingbased algorithms [Pan et al., 2010; Long et al., 2017], gradual transition-based algorithms [Gopalan et al., 2011], and pseudo labeling-based algorithms [Sener et al., 2016; Saito et al., 2017]. More interestingly, adversarial learningbased algorithms introducing a domain discriminator for minimizing distribution discrepancy, yields state of the art performance on many visual tasks [Ganin and Lempitsky, 2015; Tzeng et al., 2017; Long et al., 2018].

While previous works have achieved significant successes, they easily fail in realistic scenarios. The reason is that they implicitly assume an ideal learning environment where the source data are noise-free, which is difficult to hold in practice. Domain adaptation in noisy environments relaxes the assumption of clean data in standard domain adaptation to more realistic scenarios, where labels and features of examples from source domain are noisy. However, this more general yet challenging problem is under-explored so far. There is only one pioneering work [Shu et al., 2019] that proposes an online curriculum learning algorithm to handle this problem. Although this work makes some progress, how to conduct theoretical analysis and construct a robust algorithm for unsupervised domain adaptation under noisy environments is still an open problem.

In this paper, we give a theoretical analysis of unsupervised domain adaptation when trained with noisy labels and features. Our theoretical analysis reveals that label noise worsens the expected risk of the source distribution; thus, we define a conditional-weighted empirical risk and propose an offline curriculum learning, which also provides some effective remedial measures for the online curriculum learning [Shu et al., 2019]. Meanwhile, our theoretical analysis reveals that feature noise mainly aggravates the distribution discrepancy between source and target distribution; thus, we propose a novel proxy margin discrepancy by introducing a proxy distribution to provide an optimal solution for distribution discrepancy minimization. We finally transform our methods into an adversarial network that performs dynamic joint optimization between them. A

series of empirical studies on synthetic and real datasets show that our algorithm remarkably outperforms previous methods.

2 Domain Adaptation in Noisy Environments

2.1 Learning Set-Up

In domain adaptation, a sample of m labeled training examples {(xi, yi)}mi=1 is drawn according to a source distribution Q defined on X ×Y, where X is the feature set and Y is the label set. Y is {1, . . . ,K} in multi-class classification. Meanwhile, a sample of n unlabeled test examples {(xi)}ni=1 is drawn according to a target distribution P that somewhat differs from Q. Assume that the distributions Q and P should not be dissimilar substantially [Ben-David et al., 2010b].

For domain adaptation in noisy environments, we relax the assumption of clean data in standard domain adaptation to that source distribution may be corrupted with feature noise and label noise independently. The source distribution is corrupted into a noisy source distribution Qn. We denote by ˜x the noisy features, and pf the probability of corrupting one feature with harmful noises e, i.e., p(˜xi =xi+e)=pf and p(˜xi =xi)= 1-pf . We denote by ˜ y the noise labels corrupted with a noise transition matrix T ∈ RK×K where Tij = p(˜ y = j|y = i) denotes the probability of labeling an i-th class example as j.

We denote by L : Y × Y → R a loss function defined over pairs of labels. For multi-class classification, we denote by f : X → RK a scoring function, which induces a labeling function hf : X → Y where hf : x → argmaxy∈Y[f(x)]y. For any distribution Q on X ×Y and any labeling function h ∈ H, we denote ϵQ(h) = E(x,y)∼QL(h(x), y) the expected risk. Our objective is to select a hypothesis f out of a hypothesis set F with a small expected risk ϵP (hf ) on the target distribution.

2.2 Theoretical Analysis

For in-depth analysis, we derive an upper bound of expected target risk in noisy environments according to the triangle inequality. Full proofs will be provided in the longer version.

Proposition 2.1. For any hypothesis h ∈ H, the bound of target expected risk ϵP (hf ) in noisy environments is given by

where λ=ϵQn (h∗ )+ϵP (h∗ ) is the ideal combined error of h∗ :

Proof Sketch: According to the triangle inquality, we have

This bound is informative if the three terms are small enough; however, noises do worsen all of them as follows.

The first term, expected risk of the noisy source distribution, is given by ϵQn (h) = E(˜x,˜ y)∼Qn L(h(˜x), ˜ y). Label noise worsens the generalization ability of h in the first term, and the more label noises, the worse generalization ability. However, it is uncertain how feature noise behaves. Ilyas et al. [2019] concluded that deep neural networks (DNN) trained with adversarial perturbations might have better generalization and transfer-ability. Our empirical studies also find little difference between reserving and eliminating feature-noise examples when minimizing empirical source risk using DNNs.

The second term represents the measure of distribution discrepancy between the noisy source distribution and target distribution. Feature noise is dependent of it, but label noise is not. The following theorem reveals the dependence.

Theorem 2.1. Assume the source distribution Q be corrupted by feature noise into a new distribution R while target distribution P is unaffected. Assume feature noise is harmful and independent of the source and target distributions. Feature noise will enlarge the distribution discrepancy (disc) between source and target distribution, i.e. disc(Q, P) ≤ disc(R, P). Proof Sketch: Given distributions QX and PX over feature space X , assuming H denotes a symmetric hypothesis class, the H-divergence [Ben-David et al., 2007] is derived into

When injecting harmful feature noise into source examples, the infimum would become lower, thus dH(Q, P ) ≤ dH(R, P ).

The third term λ is assumed to be small enough [Ben-David et al., 2010b]; however, if feature-noise and label-noise are heavy, the ideal hypothesis error of target distribution will be unbounded, and this bound would be uninformative.

In summary, the above analysis suggests two critical aspects of achieving robust domain adaptation. Firstly, we should achieve a robust empirical risk minimization over the noisy source distribution by mitigating the negative influence of label noise. Secondly, we should concern a robust distribution discrepancy by reducing the impact of feature noise.

3 The ProposedMethods

Based on our analysis, we present the offline curriculum learning for the robust empirical risk minimization (Sec. 3.1), and then introduce the robust proxy margin discrepancy (Sec. 3.2).

3.1 Offline CurriculumLearning

To improve the empirical source risk, a natural idea is to eliminate label noise examples to recover a clean distribution. The pioneering work [Shu et al., 2019] adopts an online curriculum learning that defines a loss value threshold γ to eliminate noisy examples in each training epoch t ∈ T by

where wt i indicates whether or not to keep the i-th example, and θtf is the parameter of hypothesis f in the t-th epoch. ℓti = L(f(˜xi; θtf ), ˜ yi) minimized by stochastic gradient descent (SGD). Symbol 1 is the indicator function. γ is set according to cross-validation. The reason behind its success is the small loss criterion that reveals clean examples have smaller loss values than noisy examples [Han et al., 2018].

While online curriculum learning is intuitive, there are still four serious issues. Firstly, when source data are classimbalanced, the magnitude of loss values will exist substantial differences across different classes; online curriculum learning will cause biased estimation. Secondly, it is in the expectation that correct examples will have smaller losses than incorrect ones in every epoch. However, the SGD algorithmmakes the loss value of each example fluctuate in different epochs, which would cause the online curriculum learning unstable and unreliable to some extent. Thirdly, setting a fixed threshold of γ by cross-validation, is unpractical in domain adaptation. We cannot access the annotated data of the target domain. It is also unreasonable because it will lead DNNs to overfit on label-noise examples when the loss values of label-noise examples drop gradually with the increasing of the epoch. Finally, online curriculum learning treats feature and label noise examples indiscriminately and removing all of them in the whole course. However, eliminating feature-noise examples is needless when minimizing empirical source risk according to our analysis because lots of valuable features would be wasted in such a way. To resolve the above issues, we define the robust conditional-weighted empirical risk.

Definition 3.1 (Conditional-Weighted Empirical Risk). Denote by T the epoch number to filter out noisy examples, and denote by γk the loss value threshold of class k ∈ K, we define the conditional-weighted empirical risk over noisy source distribution for any classifier f ∈ F by

To optimize this risk, we propose the offline curriculum learning, which consists of two steps. In the first step, we construct the early training curriculum to filter out noisy examples. In short, we first learn a classifier f and collect examples' loss values in T epochs. We then average each example's loss value (in Eq. (9)) and rank them by class in ascending order. We use the loss value of the (mk×pk)-th example as the loss value threshold γk of class k. mk is the example number of class k, and pk = 1 - rk in which rk denotes the label-noise rate of class k. In practice, we find it is safer to set pk = max{1-1.2rk, 0.8(1-rk)}. Finally, we conduct an offline exam by comparing each example's averaged loss value of ¯ ℓi with γ˜ yi to decide each example's weight w(˜xi, ˜ yi).

In the second step, we minimize Eq. (7) using the trusted examples (w(˜xi, ˜ yi)=1) and use them to do domain adaptation. We choose cross-entropy loss, which can be optimized by SGD efficiently. Denote by σ the softmax function, i.e., for

Offline curriculum learning (OCL) provides point-to-point remedial measures for online curriculum learning. Firstly, OCL ranks and selects small-loss examples class by class, fully considering the class-conditional information and successfully achieving an unbiased estimation. Secondly, OCL examines the average loss value of each example along the whole curriculum process for comparison, avoiding the randomness of loss values in different epochs. Thirdly, OCL gets an efficient threshold of γ by leveraging the estimated labelnoise rate, which can be estimated by existing methods [Patrini et al., 2017]. We argue that we cannot expect to obtain a reliable classifier behind learning guarantees without known rough noise information. Finally, OCL only mitigates the negative influence of label noise but keeps valuable information on feature-noise examples to get better generalization. One might wonder how to separate label-noise and feature-noise examples. Our empirical studies point out that in the training process of DNNs with SGD, as for examples with the same noisy label, the ones with feature noises only will have smaller losses than the ones with false labels. Furthermore, the selected trusted source data by OCL directly supports shrinking the ideal combined hypothesis error of λ; such establishes a valid learning guarantee for the expected target risk.

3.2 ProxyMargin Discrepancy

Recall that the measure of distribution discrepancy appeared as a critical term in the target error bound; thus, we especially propose the proxy margin discrepancy to mitigate feature noise.

Theorem 2.1 implies that a better guarantee would hold if we could select, instead of Qn, other proxy distribution Q′ with a smaller discrepancy disc(Q′ , P). We thus introduce the proxy discrepancy using the Q′ to give an optimal solution for distribution discrepancy minimization, by shrinking the discrepancy between source, proxy, and target distributions.

Definition 3.2 (Proxy Discrepancy, PD). Given a proxy distribution Q′ , a noisy source distribution Qn, and a target distribution P , for any measure of distribution discrepancy disc, we define the Proxy Discrepancy and its empirical version by

In practice, we need to select an efficient disc. In the seminal works, Zhang et al. [2019] propose the novel margin disparity discrepancy (MDD) that embeds margin loss into a disparity discrepancy with informative generalization bound. Definition 3.3 (Margin Disparity Discrepancy, MDD). Given a hypothesis set F, and a specific classifier f ∈ F, the MDD and its empirical version induced by f ′ ∈ F are defined by

where disp(ρ) (f, f ′ ) is the margin disparity defined below. Let ρf ′ denote the margin of a hypothesis f ′ and Φρ denote ρ-margin loss, the margin disparity is defined by

Definition 3.4 (Proxy Margin Discrepancy, PMD). With the definition of MDD and a proxy distribution Q′ , we define the Proxy Margin Discrepancy and its empirical version by

To reveal the capacity of proxy distribution, we derive a generalization bound for PMD based on the bound of MDD.

Theorem 3.1. Based on the definition of Rademacher Complexity [Mansour et al., 2009], for any δ > 0, with probability 1-2δ over a sample of size u drawn from Q′ , a sample of size m drawn from Q, and a sample of size n dram from P, the following holds simultaneously for any scoring function f:

where R denote Rademacher complexity, ΠFH is defined as {x → f (x, h(x))|h ∈ H, f ∈ F}, where h is induced by f .

To achieve effective optimization of PMD, we adopt a deep adversarial learning network using cross-entropy loss instead of the margin loss of Φρ according to [Zhang et al., 2019]. As shown in Fig. 1, it consists of a representation function ψ and two classifiers f and f ′ with coincide to Definition 3.3. Theorem 3.1 implies searching for an empirical proxy distribution with large size of u is crucial. To achieve that, we assume that ˆQ′ is a better distribution that can be derived from noisy source distribution ˆQn, and let classifier f search for it by

where Q denotes the set of distributions with support of trusted source examples by supp(w ˆQn). When the size of ˆQ′ is zero, optimizing Eq. (16) is meaningless. Thus we limit the size of ˆQ′ by | ˆQ′ | ≥ τ|w ˆQn| where τ ∈ [0, 1] controls the ratio

Algorithm 1: Robust Domain Adaptation Algorithm.

input : parameters θψ, θf , θf ′ , learning rate η, fixed

pk, fixed τ , max epoch T , max iteration Nmax output : θψ, θf , θf ′ 1 /* stage 1: filter out label-noise examples */ 2 initialize parameters θf 3 for t = 1, 2, . . . , T do 4 collect loss values [ℓti ]mi=1 of source examples ˆQn 5 update θf = θf - η∇ℓ(f, ψ( ˆQn)) 6 end 7 average each example' loss value ¯ ℓi = 1 T ∑Tt=1 ℓti 8 rank average loss values by class in ascending order 9 assign w=1 to the top mk × pk examples of class k 10 obtain trusted data ˆQ(w=1), untrusted ˆQ′′ (w=0) 11 /* stage 2: conduct domain adaptation */ 12 initialize parameters θψ, θf , θf ′ , τ ′ = 1 13 for n = 1, 2, . . . , Nmax do 14 fetch batch ¯Q from ˆQ, ¯ P from ˆ P 15 update θf = θf - η∇ℓ(f, ψ( ¯Q′ )) 16 update θψ = θψ - η∇ℓ(f, ψ( ¯Q)) 17 obtain ¯Q′ = argmin ¯Q′ :| ¯Q′ |≥τ ′ | ¯Q| ℓ(f, ψ( ¯Q)) 18 update θf ′ = θf ′ - η∇d(ρ) Λ (ψ( ¯Q′ ), ψ( ¯ P )) 19 update θψ = θψ - η∇d(ρ) Λ (ψ( ¯Q′ ), ψ( ¯ P )) 20 update τ ′ =min{ n Nmax , τ}

21 end

between ˆQ′ and ˆQn. Since the early stage of DNNs is unstable, we design an incremental learning to gradually increase the size of ˆQ′ . We let τ ′ =min{ n Nmax , τ}, instead of τ. Here n, Nmax are the current and maximum iterations, respectively.

Meanwhile, since the optimization of PMD is a minmax game, we let the representation function ψ to minimize PMD and the classifier f ′ to maximize it. To avoid the problem of vanishing gradients, on the target domain, we use a modified cross-entropy loss: dispP (f ′ , f) = log[1σf(ψ(x))(f ′ (ψ(x))]. The final optimization of PMD is by

where α ≜ exp ρ is designed to attain the margin ρ.

In summary, we give a novel perspective when facing feature noises in domain adaptation. Briefly speaking, PMD injects a proxy distribution to bridge noisy source distribution and target distribution, successfully alleviating the negative impact of feature noise. The theory-induced adversarial network performs proxy distribution search and disentangles the min-max game simultaneously, successfully learning domaininvariant representation in noise environments. The optimization process uses incremental learning to control the size of proxy distribution, providing a stable solution for PMD.

3.3 Robust Joint Optimization

Finally, we unify the learning with the robust conditionalweighted empirical risk (Ew) and proxy margin discrepancy in a joint min-max problem. We state the final problem as

where β is the trade-off coefficient between source error and discrepancy. In practice, we split the optimization process of Eq. (19) into two stages: filter out label-noise examples and conduct domain adaptation, as shown in Algorithm 1.

4 Experiments

We evaluate our algorithm on three datasets against state of the art methods. The code is at https://github.com/zhyhan/RDA.

4.1 Setup

Office-Home is a more challenging domain adaptation dataset consisting of 15,599 images with 65 unbalanced classes. It consists of four more diverse domains: Artistic images, Clip Art, Product images, and Real-world images.

Following the protocol in the pioneering work [Shu et al., 2019], we create corrupted counterparts on the above two clean datasets as follows. We make three types of corruption: label corruption, feature corruption, and mixed corruption on source domains. Label corruption uniformly changes the label of each image into a random class with probability pnoise. Feature corruption refers to each image corrupted by Gaussian blur and salt-and-pepper noise with probability pnoise. Mixed corruption refers to each image corrupted by label corruption and Noise Level feature corruption with probability pnoise/2 independently. While past literature mainly studied label corruption, we also study mixed corruption with distribution shifts.

Figure 3: An analysis of ten methods in various mixed noise levels.

Bing-Caltech is a real noisy dataset created with Bing and Caltech-256 datasets with 256 classes. The Bing dataset consists of rich mixed noises because it was collected by retrieving from the Bing image search engine using the category labels of Caltech-256. We naturally set Bing as a source domain, while Caltech-256 as the target domain.

We compare our designed Robust Domain Adaptation (RDA) algorithm with state of the art methods: ResNet50 [He et al., 2016], Self-Paced Learning (SPL) [Kumar et al., 2010], MentorNet [Jiang et al., 2018], Deep Adaptation Network (DAN) [Long et al., 2015], Residual Transfer Network (RTN) [Long et al., 2016], Domain Adversarial Network (DANN) [Ganin and Lempitsky, 2015], Adversarial Discriminative Domain Adaptation (ADDA) [Tzeng et al., 2017], Margin Disparity Discrepancy based algorithm (MDD) [Zhang et al., 2019], and Transferable Curriculum Learning (TCL) [Shu et al., 2019]. Note that TCL is the pioneering work with the same setup of datasets as ours.

We implement our algorithm in Pytorch. We use ResNet50 as the representation function with parameters pre-trained from ImageNet. The main classifier and adversarial classifier are both 2-layer neural networks. We set the early training epoch T to 30. α is set to 3 and β is set to 0.1 according to [Zhang et al., 2019]. We use mini-batch SGD with the Nesterov momentum 0.9. The initial learning rate of the classifiers f and f ′ is 0.001, which is ten times than of the representation function ψ. The set of rk and τ depends on noise rates.

4.2 Results

Table 1 reports the results on Office-31 under 40% corruption of label, feature, and both. Our algorithm significantly outperforms existing domain adaptation methods and noise label methods (MentorNet, SPL) on almost all tasks. Note that the standard domain adaptation methods (DANN, MDD, etc.) suffer from over-fitting under noises while our algorithm performs stably positive adaptation, which demonstrates its robustness and generalization ability. While the pioneering work TCL uses additional entropy minimization to enhance its performance, our algorithm outperforms it by a large margin. Table 2 reports the results on the real dataset Bing-Caltech and the synthetic dataset Office-Home under 40% mixed corruption, where we make a remarkable performance boost.

4.3 Analysis

Ablation Study. Table 3 reports the results of our ablation study. Abandoning either the Offline Curriculum Learning (del-OCL) or the Proxy Margin Discrepancy (del-PMD) results in a significant decline, which demonstrates the efficacy and robustness of both methods in noisy environments.

Offline CurriculumLearning. We dissect the strengths of offline curriculum learning. Table 4 reports the results of three curriculummodes: online, offline, ours (Offline Curriculum Learning) on Office-31 under 40% label corruption. The online mode performs worse than the offline mode, which performs worse than us, proving the necessity of considering the averaged loss and class priors. Furthermore, we also justify our claim that we need to treat label-noise and feature-noise examples separately. As shown in Table 5, our method gets better performance than TCL that treats them indiscriminately. Table 5 also reports that reserving feature-noise examples (ours-res) generally gets better results than deleting them (ours-del and TCL) when optimizing empirical source risk.

Proxy Margin Discrepancy. While Table 1 has demonstrated the strengths of our proxy margin discrepancy of mitigating the negative influence from feature-noise examples, we provide a broader spectrum for more in-depth analysis. Table 6

reports the results of representative methods on wider featurenoise levels. Our method maintains stable performance while DANN and TCL drop rapidly with the increase of noise levels.

Noise Levels. Fig. 3 reports the results in various mixed noise levels on A→W task. Our method outperforms all the compared methods at each level, which demonstrates that our method is more robust under severe noisy environments. In particular, our methods achieve the best result when the noise level is 0%, which proves that our method can also fit into the standard domain adaptation scenarios.

Loss Distribution. Fig. 2 exhibits the loss distributions of the three types of noises of 40% on the Office-31 Amazon dataset. Fig. 2 (a, b) verifies that correct examples have smaller losses than incorrect examples. Fig. 2 (c) verifies that featurenoise examples generally have smaller losses than label-noise examples, which fits our intuition that label noise is more harmful than feature noise.

Feature Visualization. Fig. 4 illustrates the t-SNE embeddings [Donahue et al., 2014] of the learned representations by DANN, TCL, and ours, on 40% mixed corruption of A→W task. While the learned features of DANN, and TCL are mixed up, ours is more discriminative in every class on the target domain, which verifies that our method can learn domaininvariant representations in noise environments.

5 Conclusion

We presented a new analysis of domain adaptation in noisy environments, an under-explored but more realistic scenario. We also proposed an offline curriculum learning and a new proxy discrepancy tailored to label and feature noises, respectively. Our methods are more robust for achieving real-world applications in noisy non-stationary environments. The theoryinduced algorithm yields the state of the art results in all tasks.

Acknowledgements

This work is supported by the National Natural Science Foundation of China (61876098, 61573219, 61701281), the National Key R&D Program of China (2018YFC0830100, 2018YFC0830102), and the Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.