Self-adaptive Re-weighted Adversarial Domain Adaptation

1 Introduction

Unsupervised Domain Adaptation (UDA) [Pan and Yang, 2010] task aims to recognize the unlabeled target domain data, leveraging a su�ciently labeled, related but di�erent source domain. The key issue of UDA is to reduce distribution di�erence between the two domains, such that the learned classifier from source domain can well classify target domain samples. Generally, maximum mean discrepancy (MMD) [Long et al., 2015], as a non-parametric metric, is commonly used to measure the dissimilarity of distributions. Recently, adversarial learning [Bousmalis et al., 2016] has been successfully brought into UDA to reduce distribution discrepancy, in which domain-invariant or domain-confused feature representation is usually learned. Unlike many previous MMD-based methods, domain-adversarial neural netwithin a unified training paradigm. The goal of adversarial domain adaptation is to confuse the features between domains, so that domain-invariant representations are ultimately obtained.

� Contact Author

However, as discussed in MCD [Saito et al., 2018], it does not really guarantee safe domain alignment. i.e., the alignment of category space between domains is ignored in alleviating domain shift. Target samples that are close to the decision boundary or far from their class centers could be misclassified by the classifier trained in source domain [Wen et al., 2016]. Thus, previous domain adversarial adaptation methods that only match the domain distributions without exploiting the inner structures may be prone to under transfer (underfitting) or negative transfer (overfitting).

To address this problem, [Saito et al., 2017] attempt to include the target samples into the learning of their models. Specifically, some method [Chen et al., 2019] proposed to leverage pseudo-labels which is progressively guaranteed by an Easy-to-Hard strategy to learn target discriminative representations. These methods encourage a low-density separation between classes in the target domain.

However, as the domain bias exists, the pseudo labeled samples are not always correct. In order to obtain the high accurate labels of target samples, a much closer domain distribution is expected as the source classifier can generalize well on such domain-invariant target representations. In this paper, to tackle the aforementioned challenge, we take a twostep approach to learn domain invariant representations.

Firstly, the structure of adversarial network is adopted in our method. Although existing adversarial learning methods aim to reduce domain distribution discrepancy, they still suffer from a major limitation: these approaches mainly consider the marginal distribution while ignoring the conditional distribution, the classifier learned from source domain might be incapable of confidently distinguishing target samples. i.e., the joint distributions of feature and category are not well aligned across domains. Thus, these methods may lead to either under transfer or negative transfer.

To promote positive transfer and combat negative transfer, we propose to recognize the transferability of each sample and re-weight these samples to force the underlying domain distributions closer. To push further along this line, we propose a self-adapted re-weighted adversarial DA approach shown in Fig. 1. Our method considers the transferable degree from the perspective of conditional distribution, therefore it can adapt better on target domain than previous approaches which only consider marginal distribution.

In information theory, the entropy is an uncertainty measure which can also be borrowed to quantify the adaptation. Di�erent from previous methods which employ the conditional entropy directly, we utilize the entropy criterion to generate the weight and measure the degree of domain adaptation. Noteworthily, the conditional entropy is constructed by the conditional distribution. i.e., our model does not just reduce conditional distributions between two domains directly, but dynamically leverages the conditional distribution to re-weight the samples. The inner reason lies that if the sample can get a high prediction by the conditional entropy, it can be regarded as a poorly-aligned sample, otherwise a well-aligned sample. The weights for those well aligned features are decreasing while for those poorly aligned features increasing in adversarial loss self-adaptively, then a better domain-level alignment can be achieved. If the distribution bias is reduced, the precisely pseudo-labeled samples in target domain can be chosen.

Secondly, as the pseudo labels are not always correct, they are not directly leveraged to train the classifier. Instead, they are employed to train the generalized feature representations. In our method, not only global domain-level aligning strategy, but also the metric loss is employed to learn the discriminative distance in the confusing domain. In our mechanism, triplet loss utilizes source samples and pseudo-labeled target samples to keep the samples align well in class-level. As a result, our model can learn better domain alignment features in the collaborative alignment adversarial training process and these features are not only domain invariant but also class discriminative for semantic alignment. This will have a more confident guarantee that the joint error of the ideal source and target hypothesis is low. The DA becomes possible as presented in Ben-David's theorem [Ben-David et al., 2010].

The main contributions and novelties of this paper are summarized as follows.

• � Our model attempts to learn the target generalized model to promote positive transfer and combat negative trans-
• fer. The network leveraging the joint distributions is much more proper than only the marginal distribution.
• � In order to achieve the better domain confusion, we present a self-adaptive re-weighted adversarial domain adaptation approach through entropy from the perspective of conditional distribution.In our method, the adversarial network forces to reduce domain discrepancy by re-weighting the samples. The weights of the adversarial loss for well aligned features are decreased while increasing for those poorly aligned, such that a better domain-level alignment can be achieved.
• � Besides the domain-level alignment, triplet loss is employed to enforce the features have better inter-class separation and intra-class compactness utilizing source samples and pseudo-labeled target samples. As a result, the feature representations which are not only domain invariant for domain alignment but also class discriminative for semantic alignment can be learned.

2 RelatedWork

Training CNN for UDA can be conducted through various strategies. Matching distributions of the middle features [Long et al., 2015; Long et al., 2017; Zellinger et al., 2017] in CNN is considered to be e�ective for an accurate adaptation. These works pay attention to first-order or high-order statistics alignment.

Recent research on deep domain adaptation further embeds domain-adaptation modules in deep networks to boost transfer performance. In [Ganin et al., 2016], DANN is proposed for domain adversarial learning, in which a gradient reversal layer is designed for confusing features from two domains. This method can be regarded as the baseline of adversarial learning methods. Tzeng etal. proposed an ADDA method [Tzeng et al., 2017] which combines discriminative modeling, untied weight sharing and a GAN loss. Long etal. [Long et al., 2018] also present a conditional adversarial domain adaptation (CDAN) that conditions the discriminative information conveyed into the predictions of classifier. Wang et al. [Wang et al., 2019] propose a TADA focus the adaptation model on transferable regions or images both by local attention and global attention.

However, these methods are based on the theory that the predicted error is bounded by the distribution divergence. They do not consider the relationship between target samples and decision boundaries. To tackle these problems, multiple classifiers instead of the discriminator are considered and these methods become another branch. Saito et al. propose a ATDA method [Saito et al., 2017] by tri-training three classifiers equally to give pseudo-labels to unlabeled samples. Later, he also proposes a new approach called MCD [Saito et al., 2018] that uses two di�erent classifiers to align those easily misclassified target samples through adversarial learning in CNN. Recently, Zhang et al. proposed SymNet [Zhang et al., 2019] based on a symmetric design of source and target task classifiers, meanwhile, an additional classifier that shares with layer neurons was constructed.

In our method, we propose a di�erent strategy to address these problems. Not only the source data but also the target samples are leveraged to align domain features and class relations.

3 Self-adaptive Re-weighted Adversarial DA

An overview of our method is depicted in Fig. 2. In UDA, we suppose Ds = f(x s i ; y s i )g ns i=1 and Dt = fx t j g nt j=1 to be the labeled source data and unlabeled target data, drawn from different distributions respectively. Our goal is to predict the target label ˆy t = argmaxGy( f (x t )) and minimize the target risk �t(Gy) = E(x t ;y t )�Dt [Gy( f (x t )) , y t ], where Gy(�) represents the softmax output and f (�) refers to the feature representation.

Our method aims to construct a target generalized network. The re-weighted adversarial domain adaptation forces a close global domain-level alignment. Simultaneously, triplet loss is leveraged to train the class-discriminative representations utilizing source samples and pseudo-labeled samples. Then the classifier gradually increase accuracies on the target domain.

Preliminaries: Domain Adversarial Network. In DA setting, domain adversarial networks have been successfully explored to minimize the cross-domain discrepancy by extracting transferable features. The procedure is a two-player game: the first player is the domain discriminator Gd trained to distinguish the source domain from the target domain, and the second player is the feature extractor f (�) trained to confuse the domain discriminator. The objective function of domain adversarial network is as follows:

where �f ; �d represent the parameters of feature network and domain discriminator, respectively. �y is the parameter of source classifier. di is the domain label of sample xi.

Self-adaptive Re-weighted Adversarial DA. In practical domain adaptation problems, however, the data distributions of the source domain and target domain usually embody complex multimode structures. Thus, previous domain adversarial adaptation methods that only match the marginal distributions without exploiting the multimode structures maybe prone to either under transfer or negative transfer. To promote positive transfer and combat negative transfer, we should find a technology to reveal the transferable degree of samples and then re-weight them to force the underlying distribution closer.

As mentioned earlier, not all images are equally transferable in domain adaptation network and some images are more transferable than others. Therefore, we propose a method to measure adaptable degree using the certainty estimate. In information theory, the entropy functional is an uncertainty measure which nicely meets our need to quantify the adaptation and is depicted in Fig. 3. Therefore, we utilize the entropy criterion to estimate weights and improve the domain confusion by relaxing the alignment on these well aligned samples and focusing the alignment on these poorly aligned. If the sample can get a low entropy, it can be regarded as a well-aligned transferable sample, else it is a poorly aligned sample. The conditional distribution leveraged in the entropy is not considered in the standard adversarial DA methods. We adopt the conditional entropy as the indicator to weight the adversarial loss and the adversarial loss is extended as

where C is the number of classes and pc is the probability of predicting an sample to class c.

To make the best of conditional distribution, entropy minimization principle is adopted to enhance discrimination of learned models for target data by following [Long et al., 2016]. In order to reduce wrong classified samples due to domain shift, the entropy minimization loss is used to update both the feature network and classifier.

Class-level Alignment. So far, we only consider the global domain-level confusion, the discriminative power between classes is not involved [Hong et al., 2015]. For one thing, samples with same labels should be pulled together in the embedding space. For another, samples with di�erent labels should be separated apart. Naturally, metric learning [Yang et al., 2017] as an e�ective method can be implemented to achieve our goal. Triplet loss [Schro� et al., 2015] tries to enforce a margin m between each pair of samples from one class to all other classes. It allows samples to enforce the distance and thus discriminate to other classes. In our paper, we select triplet loss to train the class-level aligned features.

Intuitively, a target generalized classifier is much more proper than the source classifier for the target domain. We propose to assign pseudo-labels to target samples and train the network as if they were true labels. Noteworthily, as the pseudo labels are not always correct, they are not directly leveraged to train the classifier. In order to make the best of pseudo-labeled samples, we leverage the source samples and pseudo-labeled target samples to construct the sample-pairs in metric learning. We follow the sampling strategy in [Deng et al., 2018] to randomly select samples.

The pseudo label ˆy t i of x t i is predicted based on maximum posterior probability using the source cross-entropy loss. It is progressively updated during optimization. Additionally, we only select target images with predicted scores above a high threshold T for building the semantic relations based on the intuitive consideration that the image with the high predicted score is more likely to be classified correctly. We empirically set the threshold T as a constant.

In the confusing domain, given an anchor image xa, a positive image xp, and a negative image xn, the minimized loss is as:

Overall Training Loss. With Eq. (1), Eq. (2), Eq. (3) and Eq. (4), the overall training loss of our model is given by,

The optimization problem is to find the parameters ^ �f ; ^ �y and ^ �d that jointly satisfy

4 Experiment

In this section, several benchmark datasets, not only the toy datasets as USPS+MNIST datasets, but also O�ce-31 dataset [Saenko et al., 2010], ImageCLEF-DA [Long et al., 2017] dataset and O�ce-Home [Venkateswara et al., 2017] dataset, are adopted for evaluation.

Handwritten Digits Datasets. USPS (U) and MNIST (M) datasets are toy datasets for domain adaptation. They are standard digit recognition datasets containing handwritten digits from 0 � 9. USPS consists of 7,291 training images and 2,007 test images of size 16 � 16. MNIST consists of 60,000 training images and 10,000 test images of size 28 � 28. We construct two tasks: U ! M and M ! U and we follow the experimental settings of [Ho�man et al., 2018].

O�ce-31 Dataset. This dataset is a most popular benchmark dataset for cross-domain object recognition. The dataset consists of daily objects in an o�ce environment and includes three domains such as Amazon (A), Webcam (W) and Dslr (D). There are 2,817 images in domain A, 795 images in W and 498 images in domain D making total 4,110 images. With each domain worked as source and target alternatively, 6 cross-domain tasks are formed, e.g., A!D etc.

ImageCLEF-DA Dataset. The ImageCLEF-DA is a benchmark for ImageCLEF 2014 domain adaptation challenge. It contains 12 common categories shared by three public datasets: Caltech-256 (C), ImageNet ILSVRC 2012 (I) and Pascal VOC 2012 (P). In each domain, there are 50 images per class and totally 600 images are constructed. Images in ImageCLEF-DA are of equal size, making it a good alternative dataset. We evaluate all methods across three transfer domains and build 6 cross-domain tasks: e.g., I ! P etc.

O�ce-Home Dataset. This is a new and challenging dataset for domain adaptation, which consists of 15,500 images from 65 categories coming from four significantly different domains: Artistic images (Ar), Clip Art (Cl), Product images (Pr) and Real-World images (Rw). With each domain worked as source and target alternatively, there are 12 DA tasks on this dataset. The images of these domains have substantially di�erent appearance and backgrounds, and the number of categories is much larger than that of O�ce31 and ImageCLEF-DA, making it more di�cult to transfer across domains.

Results. In our experiment, the target labels are unseen by following the standard evaluation protocol of UDA [Long et al., 2017]. Our implementation is based on the PyTorch framework. For the toy datasets of handwritten digits, we utilize the LeNet. For other datasets, we use the pre-trained ResNet-50 as backbone network. We adopt the progressive training strategies as in CDAN [Long et al., 2018]. In the process of selecting pseudo-labeled samples, the threshold T is empirically set as the constant 0.9. The margin m and N0 in triplet loss are set as 0.3 and 3 following the setting as usual, respectively.

We evaluate the rank-1 classification accuracy for comparison. For handwritten digits, as there are plenty of di�erent configutations in these datsets, in order to give the fair comparison, we only show some the recent results with the same backbone and training/test split. we compared with ADDA [Tzeng et al., 2017], CoGAN [Liu and Tuzel, 2016], UNIT [Liu et al., 2017], CYCADA [Ho�man et al., 2018] and CDAN [Long et al., 2018]. For other datasets, our compared baseline methods include DAN [Long et al., 2015], DANN [Ganin et al., 2016], JAN [Long et al., 2017], CDAN [Long et al., 2018]and SAFN [Xu et al., 2019]. Besides, on O�ce-31 dataset, we compare with TCA [Pan et al., 2011], GFK [Gong et al., 2012], DDC [Tzeng et al., 2014], RTN [Long et al., 2016], ADDA [Tzeng et al., 2017], MADA [Cao et al., 2018], GTA [Sankaranarayanan et al., 2019], MCD [Saito et al., 2018], iCAN [Zhang et al., 2018], TADA [Wang et al., 2019] and SymNet [Zhang et al., 2019]. On ImageCLEF-DA dataset, RTN [Long et al., 2016], MADA [Cao et al., 2018] and iCAN [Zhang et al., 2018] are compared. On O�ce-Home dataset, TADA [Wang et al., 2019] and SymNet [Zhang et al., 2019] are compared.

5 Discussion

Ablation Study. The ablation analysis results under di�erent model variants with some loss removed are presented in Table 5. The baseline of ResNet-50 denotes that only the source classifier based on cross-entropy loss is trained. DANN is another baseline, in which the cross-entropy loss and domain alignment are taken into account, and the performance is increased from 74.3% to 82.1%. Besides, we additionally optimize the entropy minimization loss of target samples over their feature extractors and denote them as ResNet-50 (Em) and DANN (Em), respectively. They two can be regarded as another baselines compared with our method.

Our model benefits from both the novel re-weighted domain adaptation and class-level alignment cross domains. To investigate how di�erent components in our model, we add every item alternately. We can verify the items from two aspects. On the one hand, we do not re-weight the adversarial loss and the DANN (Em) can be regarded as the baseline. Then we add the cross-domain weight Hp into the adversarial loss, the training setting of which is denoted as "DANN (Em+Hp)", the performance is increased from 87.2% to 88.8% after adding re-weight itemHp. On the other, we add the metric triplet loss Ltri into the DANN baseline, the training setting of which is denoted as "DANN (Em +Ltri)" and the performance is increased from 87.2% to 89.5%. Furthermore, the item of "DANN (Em +Ltri)" can be regarded as another baseline compared with "Ours" to prove the e�ectiveness of re-weight item Hp and the performance is increased to 90.2%. Additionally, because we also take some pseudo target labels into consideration, so for validating the triple loss with target samples, we have experimentally demonstrated the e�ectiveness of pseudo labels in metric loss. The performance is decreased from 89.5% to 88.6% after removing the target pseudo labels, i.e., "DANN (Em +Ltri�s)".

Quantitative Distribution Discrepancy. A-distance

[Ben-David et al., 2010] which jointly formulates source and target risk, are used to measure the distribution discrepancy after domain adaptation. A-distance is defined as dA = 2(1�2�), where � is the classification error of a binary domain classifier (e.g., SVM) for discriminating the source and target domains. Therefore, with the increasing discrepancy between two domains, the error � becomes smaller. Figure 4 (a) shows A-distance on di�erent tasks by using di�erent models. From Figure 4 (a), it is obvious that a large A-distance denotes a large domain discrepancy. The distribution discrepancy analysis based on A-distance in O�ce-31 dataset on tasks A ! W and W ! D is conducted by using ResNet, DANN and our complete model, respectively.

We can observe that A-distance between domains after using our model is smaller than that of other two baselines, which suggests that our model is more e�ective in reducing the domain discrepancy gap. By comparing the distribution discrepancy between A ! W and W ! D, obviously, W ! D has a much smaller A-distance than A !W. From the classification accuracy in Table 2, the recognition rate of W!D is 100.0%, which is higher than A !W (95.2%). Therefore, the reliability of A-distance is demonstrated.

Convergence. Figure 4 (b) shows the convergence of ResNet, DANN, our baseline method only with re-weight item, i.e., DANN (Em+Hp), our baseline method with only source labels for triplet loss, i.e., DANN (Em+Ltri�s), our baseline method with only triplet loss, i.e., DANN (Em+Ltri), and our complete model Ours, respectively. We choose the task A !W in O�ce-31 dataset as an example and the test errors of di�erent methods with the increasing number of iterations are shown in Figure 4 (b).

Feature Visualization. We visualize the domain invariant features learned by ResNet, DANN, Ours (w=o Hp), and our complete model for further validating the e�ectiveness. For feature visualization, t-SNE visualization method is employed on the source domain and target domain in the A!W task from O�ce-31 dataset. The results of feature visualization for ResNet (traditional CNN), DANN (with adversarial learning), Ours (w=o Hp) (i.e., our model without re-weight), and our complete model are illustrated in Figure 5.

Note that Figure 5 (a)-(d) represent the results of source features from 31 classes marked in di�erent colors, from which we observe that Ours (w=o Hp) and our model can reserve better discrimination than other two baselines as the two consider the discriminative power. The features of two domains are visualized in Figure 5 (e)-(h). It is obvious that the features learned by ResNet across source and target domains can not be well aligned, without considering the feature distribution discrepancy. In DANN, by aligning the domain distribution, the distribution discrepancy of learned features between two domains can be improved. However, the class discrepancy of features fromDANN is not improved, as DANN does not take the class level distribution into account. In our method, compared with Ours (w=o Hp), it can alleviate domain discrepancy to some extent by our re-weight. From the classification accuracies in Table 5, Ours (95.2%) is a little better than Ours (w=o Hp) (93.8%). From the results, the features learned by our model can be well aligned between two domains, but reserve more class discrimination including intra-class compactness and inter-class separability.

6 Conclusion

To promote positive transfer and combat negative transfer in DA problem, we propose a self-adaptive re-weighted adversarial approach that tries to enhance domain alignment from the perspective of conditional distribution. For alleviating the domain bias issue, on one hand, considering that not all images are equally transferable in domain adaptation network and some images are more transferable than others, we propose a method to reduce domain-level discrepancy by reweighting the transferable samples. Our method reduces the weights of the adversarial loss for aligned features while increasing the adversarial forces for those poorly aligned adaptively. On the other, triplet loss is employed on the confusing domain to ensure the distance of the intra-class sample pairs closer than the inter-class pairs to achieve class-level alignment. Therefore, the high accurate pseudo-labeled target samples and semantic alignment can be captured simultaneously in this co-training process. The experimental results verify that the proposed model outperforms state-of-the-arts in various UDA tasks.

Acknowledgements

This work was supported by the National Science Fund of China under Grants (61771079), Chongqing Youth Talent Program, and the Fundamental Research Funds of Chongqing (No. cstc2018jcyjAX0250).