--- title: "Exploring Safety Supervision for Continual Test-time Domain Adaptation" authors: - "Xu Yang" - "Yanan Gu" - "Kun Wei" - "Cheng Deng" doi: "10.24963/ijcai.2023/183" url: "https://ijcai.org/proceedings/2023/183" pdf: "https://ijcai.org/proceedings/2023/0183.pdf" venue: "Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence" year: 2023 pages: "1649-1657" track: "Main Track" keywords: - "Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning" - "Computer Vision: CV: Representation learning" --- # Exploring Safety Supervision for Continual Test-time Domain Adaptation Xu Yang, Yanan Gu, Kun Wei, Cheng Deng *Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence*, pages 1649-1657. DOI: [10.24963/ijcai.2023/183](https://doi.org/10.24963/ijcai.2023/183) ## Abstract Continual test-time domain adaptation aims to adapt a source pre-trained model to a continually changing target domain without using any source data. Unfortunately, existing methods based on pseudo-label learning suffer from the changing target domain environment, and the quality of generated pseudo-labels is attenuated due to the domain shift, leading to instantaneous negative learning and long-term knowledge forgetting. To solve these problems, in this paper, we propose a simple yet effective framework for exploring safety supervision with three elaborate strategies: Label Safety, Sample Safety, and Parameter Safety. Firstly, to select reliable pseudo-labels, we define and adjust the confidence threshold in a self-adaptive manner according to the test-time learning status. Secondly, a soft-weighted contrastive learning module is presented to explore the highly-correlated samples and discriminate uncorrelated ones, improving the instantaneous efficiency of the model. Finally, we frame a Soft Weight Alignment strategy to normalize the distance between the parameters of the adapted model and the source pre-trained model, which alleviates the long-term problem of knowledge forgetting and significantly improves the accuracy of the adapted model in the late adaptation stage. Extensive experimental results demonstrate that our method achieves state-of-the-art performance on several benchmark datasets. ## Full text ## 1 Introduction Deep neural networks have achieved remarkable success in visual tasks when training and test data follow the same distribution. These networks, however, suffer from the generalization problem in the presence of domain shift. For example, a classification network pre-trained in the normal, natural images domain may not recognize the corrupted images due to domain shift [Dong et al., 2020; Yang et al., 2022]. Thus, an adaptation is necessary to transfer knowledge from the source domain to the target one by reducing the domain shift. However, the label of the target domain is usually unavailable, so the problem is known as Unsupervised Domain Adaptation (UDA) [Ganin and Lempitsky, 2015; Prabhu et al., 2021; Sun et al., 2019; Dong et al., 2021]. In addition, the source data is often inaccessible during inference time due to privacy or business problems, making the adaptation problem more challenging but more realistic. Therefore, such adaptation problem becomes SourceFree/Test-Time domain Adaptation (TTA) [Chen et al., 2022; Yang et al., 2021; Kurmi et al., 2021; Kundu et al., 2020; Liu et al., 2021] where only the source model and unlabeled target data are available in the adaptation process. ∗Corresponding author. Existing TTA methods usually solve the domain shift problem by updating the adapted model parameters using the generated pseudo-labels or entropy regularization. These selftraining-based methods are effective when the distribution of data in the target domain is fixed, but when the distribution of the target domain is constantly changing [Wang et al., 2022; Prabhu et al., 2021], these methods become unstable. Noise problem [Wang et al., 2022] caused by the change of target domain distribution seriously affects the adaptation process. CoTTA [Wang et al., 2022] first defines this kind of problem as Continual Test-Time Domain Adaptation, where a source pre-trained model needs to adapt to a stream of continually changing target test data without using any source data. It uses a weight-average teacher network to improve the quality of generated pseudo-labels. However, the accurate optimization of a network requires the joint action of label and loss function. The pseudo-labels generated by existing methods become noisier in the changing target domain environment because of domain shifts in the adaptation process. Thus, refining pseudo-label learning requires high-quality pseudolabels and a reliable label selection strategy. In this paper, we propose a simple yet effective framework for continual test-time domain adaptation, which refines the pseudo-label learning process for exploring safety supervision from three perspectives: Label Safety, Sample Safety, and Parameter Safety, to alleviate the instantaneous and long-term impact of noisy pseudo-labels. In terms of label safety, noisy pseudo-labels will immediately affect the loss calculation in network optimization. Thus, we define and adjust the confidence threshold in a self-adaptive manner to restrain the noisy pseudo-labels in the test-time learning status. Notably, instead of simply presenting a constant hyper-parameter and treating all classes equally, we select an independent threshold for each class through global and local strategies to choose more reliable pseudo-labels as supervision information. Based on such thresholds, we construct a soft-weighted contrastive learning module for the sample safety of contrastive learning module, which pulls the reliable same-class samples closer and discriminates against uncorrelated samples. Moreover, the Fourier transform is introduced into the contrastive learning process as a strong augmentation to explore domain-invariant predictions. In this way, we can alleviate instantaneous negative learning and improve learning efficiency. From the perspective of long-term impact, as the model continually adapts to the target domain with changing distributions, the knowledge from the pre-trained model is constantly forgotten due to error accumulation caused by the noisy pseudo-labels. To slow down the accumulation of errors and combat knowledge forgetting, we propose a Soft Weight Alignment (SWA) strategy, which continuously distills knowledge from the source pre-trained model. In detail, we normalize the distance between the parameters of the adapted model and the pre-trained model. In doing so, we continuously distill knowledge from the source pre-trained model and can effectively alleviate the long-term knowledgeforgetting problem. Extensive experimental results demonstrate that our method achieves state-of-the-art performance on several datasets. To sum up, our contributions are as follows: - We design a novel yet efficient continual test-time domain adaptation method for exploring safety supervision. The learnable thresholds are tuned in a local and global manner according to the test-time learning status to enhance label safety, sample safety, and parameter safety, further alleviating instantaneous negative learning and long-term knowledge-forgetting problems. - We introduce a soft-weighted contrastive learning module, which pulls the reliable same-class samples closer and alleviates the false negative samples. The learnable thresholds and pseudo-labels are utilized to select more discriminative positive and precise negative pairs for contrastive learning. Moreover, we employ the Fourier transform to augment the positives, exploring domain-invariant predictions. - We propose a Soft Weight Alignment strategy, which continuously distills knowledge from the source pretrained model. We normalize the distance between the adapted and pre-trained models' parameters. In doing so, we continuously distill knowledge from the source pre-trained model and can effectively alleviate the longterm knowledge-forgetting problem. ## 2 RelatedWork ## 2.1 Domain Adaptation Domain adaptation refers to the goal of learning a concept from labeled data in the source domain that performs well on different but related target domains. The critical problem of domain adaptation lies in the misalignment between the feature and label spaces of the source and target domains. To solve this problem, some domain adaptation methods guide the deep model to learn domain invariant representation and classifiers. Specifically, some works [Ganin and Lempitsky, 2015; Tzeng et al., 2017; Ganin et al., 2016] utilize adversarial training to align feature distribution with a domain discriminator, and some works constrain the cross-domain feature space by entropy constraint [Grandvalet and Bengio, 2004; Saito et al., 2019] or maximum prediction rank [Cui et al., 2020; Yang et al., 2019]. ## 2.2 Test-Time Domain Adaptation Recently, some works on test-time domain adaptation focus on a more challenging setting where only the source model and unlabeled target data are available. Some testtime domain adaptation methods [Kurmi et al., 2021; Li et al., 2020] utilize generative models to achieve the feature alignment between the source and target domain in the absence of source data. Kurmi et al. adopt the trained classifier [Kurmi et al., 2021] to generate samples from the source classes. This method learns the joint distribution of data by using the energy-based modeling of the trained classifier. Yeh et al. propose the method of Source-data-free Feature Alignment (SoFA) [Yeh et al., 2021] to extract features with class semantics, thus realizing domain adaptation in the absence of the source data. In addition, some methods achieve test-time domain adaptation by finetuning the source model with the help of target data and do not require explicit domain alignment. Test entropy minimization (TENT) [Wang et al., 2020] introduces entropy minimization as a test-time optimization objective, which estimates normalization statistics and optimizes channel-wise affine transformations to update online on each batch. Source HypOthesis Transfer (SHOT) [Liang et al., 2020] aims to learn the optimal target-specific feature learning module to fit the source hypothesis. It is worth mentioning that SHOT requires using source data to train a specialized source model, so it cannot support an arbitrary source pre-trained model. Zhou et al. present Bayesian Adaptation for Covariate Shift (BACS) [Zhou and Levine, 2021] to obtain both improved accuracy and well-calibrated uncertainty estimates when faced with domain shift. Most test-time adaptation methods only consider the offline scenario, where the full set of test data is provided during the training process. Further, CoTTA [Wang et al., 2022] extends test-time adaptation from offline scenario to online continual scenario. It considers a more challenging but more realistic problem named Continual Test-Time Domain Adaptation, where a source pre-trained model needs to adapt to a stream of continually changing target test data without using any source data. In this paper, we also focus on such a more realistic setting than standard test-time adaptation. ## 3 ProposedMethod ## 3.1 ProblemDefinition Following [Wang et al., 2022], we consider a continual testtime domain adaptation setting, where a pre-trained model needs to adapt to a continually changing target domain in an online fashion when the absence of source data. Consider a pre-trained model Fθ(x) with parameter θ trained on the source data (X S , Y S ). Unlabeled target domain data X T is provided sequentially and the data distribution of X T is continually changing. At inference time t, when the unlabeled target data XT (t) = [xT1 (t), ..., xTB (t)] is sent to the model Fθt , where B is the number of samples. The model Fθt needs to make the prediction Fθt (XT (t)) and adapts itself accordingly for the next input (θt → θt+1). It is worth noting that the total evaluation process is online, and the model only has access to the data XT (t) of the current time step t. The framework is shown in Fig. 1. Figure 1: This is the flow of our method. At the time t, the model receives an incoming batch of images from the unlabeled target data flow. After data augmentation, these incoming images are fed into both the teacher and student networks. The average outputs of the augmented images in the teacher network are utilized as self-adaptive thresholding pseudo-labels to supervise the student network. In addition, the parameters of the student network are softly aligned with those of the anchor network to prevent knowledge forgetting in the adaptation process. ![Image](https://ijcai.org/proceedings/2023/fulltext/183/image_000000_cac225b08131b4db5de49856aae2672389d6bda3a8f922ee2f92c7177c1bd258.png) ## 3.2 Self-adaptive Thresholding Cross-Entropy Loss In this section, we detail the self-adaptive thresholding crossentropy loss. In the process of adapting to a continually changing target domain, the quality of pseudo-labels decreases significantly because of the distribution shift. To improve the quality of pseudo-labels, following [Wang et al., 2022], we use a weight-averaged teacher model Fˆ θ to generate the pseudo-labels. At the time t = 0, the teacher network is initialized by the pre-trained source model. Then, at time t, the teacher model Fˆ θ generates the pseudo-label ˆ yT (t) to help the learning process of the student model (adapted model) Fθt . Specifically, we compute the cross-entropy loss between the output of the student model and the pseudo-label ˆ yT (t). The learning process can be denoted as follows: ˆ yT bk (t) is the soft predication of the teacher model of class k ∈ K, and yT bk (t) is the prediction of the student model. During this process, the accumulation of erroneous pseudo-labels can severely disturb the model's predictive performance. This work focuses on pseudo-labeling using crossentropy loss with a confidence threshold. Instead of simply presenting a constant hyperparameter and treating all classes equally, we advocate that the key to determining thresholds is that thresholds should reflect the test learning status. Thus, we present self-adaptive thresholding that automatically defines and adaptively adjusts the confidence threshold for each class by leveraging the current predictions during test-time training. The global threshold should represent the model's confidence in the test data, reflecting the overall learning status. We set the global threshold τt as the average confidence from the model on test data, and estimate the global confidence as the exponential moving average (EMA) at each time step, where t represents the t-th time step. The global threshold τt is defined and adjusted as: In addition to the global threshold, the local threshold is utilized to modulate the global threshold in a class-specific fashion to account for the intra-class diversity and the possible class adjacency. We compute the expectation of the model's predictions on each class k to estimate the classspecific learning status: After integrating the global and local thresholds, we can obtain the final self-adaptive threshold of each class k. Finally, we adopt the self-adaptive threshold to select reliable samples for supervision, and the training objective of Eq. 1 can be denoted as follows: ## 3.3 Soft-weighted Contrastive Learning Such pseudo-labels can be further utilized to promise the model more substantial representation power, while previous methods have not achieved it. Under the context of contrastive learning, in particular, these semantic class structures can give helpful guidance in selecting contrastive pairs with similar semantics to improve training efficiency. Specifically, the similarity of the samples b and q can be calculated with cosine similarity using pseudo-labels. We adopt the weighted similarity matrix wt to guide the traditional contrastive loss, which can be written as follows: where zT b (t) = Fθ(xTb (t)). We then introduce the components of the objective in detail. Fourier Augmentation. Some work [Xu et al., 2021; Lu et al., 2022] have demonstrated that Fourier phase information contained high-level semantics and was not easily affected by domain shifts, which may be a kind of universal domain-invariant features. Thus, we introduce the Fourier transform as a new augmentation and attempt to learn domain-invariant representations. The Fourier transformation FFT(x) for a single-channel two-dimensional data x is formulated as: where u and v are indices, H and W are the height and the width, respectively. For data xTb (t) with several channels, the Fourier transformation for each channel is computed independently to obtain the corresponding phase information ˜xTb (t). Positives. Except for introducing Fourier transform as a positive sample, we attempt to present more positive samples by utilizing the correlation between samples during the instantaneous learning process. According to the pseudo-labels output by the teacher model, we select reliable samples of the same class with b as the positive sample set through the learned threshold. Negtatives. The traditional contrastive loss strives to maximize the cosine distances between b and every q in the batch. Instead, we argue that not pushing away same-class pairs helps learn better semantically meaningful clusters. Specifically, we adopt the labels to exclude same-class pairs from all negative pairs: The objective of the contrastive loss can be written as: Similar to [Wang et al., 2022], we also use the data augmentation strategy to refine the pseudo-label learning process. Specifically, we combine the original images with its N augmented images into a batch and send it to the student network. Large batch size is conducive to the update of the batch normalization layer and the convergence of the network, especially for the student model in an online update fashion. We use the average predictions for all augmented samples as the pseudo-labels to supervise the learning process of the student model. After the update of student model (θt → θt+1). The teacher model is updated by exponential moving average [Polyak and Juditsky, 1992; Tarvainen and Valpola, 2017] using the weights of the student model: where α represents the smoothing factor. ## 3.4 Soft Weight Alignment With the help of the pseudo-labels, the pre-trained model can quickly adapt to the target domain and make a more accurate prediction of the target data. However, due to the difference in the data distribution of the source and target domains, there are noisy pseudo-labels in the adaptation process, which leads to instantaneous negative learning. Furthermore, as the model continually adapts to the target domain with changing distributions, the instantaneous errors accumulate, and the knowledge from the pre-trained model is constantly forgotten. For example, the model may not recover after encountering some hard samples, which is detrimental to the test of subsequent data, even if the latter data is not severely shifted. To slow down the accumulation of errors and combat knowledge forgetting, we propose a Soft Weight Alignment (SWA) strategy, which continuously distills knowledge from the source pre-trained model. Given a pre-trained model Fθ and its two derivative models: anchor model Fθa and student model Fθt , and both Fθt and Fθa are initialized by Fθ. In the continual test-time domain adaptation process, the parameters of the anchor model are fixed, and the parameters of the student network are constantly updated. In fact, the anchor model Fθa is the pretrained model Fθ0 at time t = 0. To maintain the knowledge gained from the pre-training model, we soft-align the parameters of the student network Fθt and the anchor network Fθa : where θla and θlt are parameters of anchor model Fθa and student model Fθt at layer l, respectively. BN represents the Batch Normalization (BN) layer. SWA forces the parameters of the student model to be soft consistent with those of the anchor model, thereby enhancing the anti-forgetting ability of the student model. Specifically, the indicator function l /∈ BN equals 1 if layer l of the anchor or student network is not the BN layer, and 0 otherwise. Therefore, we use SWA to constrain the update of the student network. The total loss function LT can be formulated as follows: where β1 and β2 are the balance hyper-parameters of the total loss function. ## 4 Experiments In this section, we review the proposed method on several benchmark tasks: CIFAR10-to-CIFAR10C (Standard and Gradual), CIFAR100-to-CIFAR100C, and ImageNetto-ImageNet-C. We first introduce several commonly used datasets and our method's implementation details; then, we present several commonly used comparison methods; finally, we report and analyze the results to validate the effectiveness of our approach. ## 4.1 Datasets We use CIFAR10, CIFAR100, and ImageNet as the source domain datasets, and CIFAR10C, CIFAR100C, and ImageNet-C as the corresponding target domain datasets, respectively. The target domain datasets were originally created to evaluate the robustness of classification networks [Hendrycks and Dietterich, 2019]. Each target domain dataset contains 15 types of corruption with 5 levels of severity. Following [Wang et al., 2022], for each corruption, we use 10000 images for both CIFAR10C and CIFAR100C datasets and use 5000 images for ImageNet-C. ## 4.2 Implementation Detail Following [Wang et al., 2022], the corrupted images are provided to the network online, which means these images can be used to update the model only once in the adaptation process. In addition, different from traditional test-time adaptation methods, which adapt to each corruption type data individually, we adjust the source model to each corruption type sequentially. We evaluate the adaptation performance immediately after encountering each corruption type data. The total type of corruption is set as 15, and the corruption level is set to the highest level of 5 (except for the gradual experiments on CIFAR10-to-CIFAR10C). For CIFAR10-to-CIFAR10C, we use a pre-trained WideResNet28 [Zagoruyko and Komodakis, 2016] model from the RobustBench benchmark[Croce et al., 2020]. We use Adam to optimize the network and set the learning rate to 1e-3. The data augmentation strategy is the same as [Wang et al., 2022], including color jitter, gaussian blur, gaussian noise, random affine, and random horizontal flip, N = 8. CIFAR100-toCIFAR100C, we use a pre-trained ResNeXt-29 [Xie et al., 2017] from [Hendrycks et al., 2019], N = 4. For ImageNetto-ImageNet-C, we use the standard pre-trained ResNet-50 from RobustBench [Croce et al., 2020], N = 4. The experiments on ImageNet-to-ImageNet-C are performed under ten diverse corruption orders. The smoothing factor α is set as 0.99. ## 4.3 Baselines We compare our method with several state-of-the-art continual test-time adaptation algorithms, the details of these methods are as follows: 1) Source: It directly uses the pre-trained model for adaptation without any specific method for domain adaptation; 2) BN Stats Adapt: Batch Normalization Statistics Adaptation method keeps the pre-trained model weights and uses the Batch Normalization statistics from the input data of the input batch for the prediction [Li et al., 2016; Schneider et al., 2020]; 3) Pseudo-Label [Lee and others, 2013]: This method picks up the class which has the maximum predicted probability as the pseudo-labels to update the model; 4) TENT [Wang et al., 2020]: Test entropy minimization, a test-time entropy minimization scheme to reduce generalization error by reducing the entropy of model predictions on test data; 5) TENT-continual is a continual learning version of TENT; 6) CoTTA [Wang et al., 2022]: Continual Test-Time Adaptation, which reduces the error accumulation by using weight-averaged and augmentation-averaged predictions and avoids catastrophic forgetting by stochastically restoring a small part of the source pre-trained weights. ## 4.4 CIFAR10-to-CIFAR10C Performance Evaluation. Table 1 shows the classification error rate for the standard CIFAR10-to-CIFAR10 task. Source method shows the highest average error rate. It depends mainly on the distance between the current corruptiontype data distribution and the source domain distribution. BN Stats Adapt method is simple and effective, dramatically reducing the evaluation error compared with the Source method. Compared with BN Stats Adapt and Pseudo-Label, the performance of TENT-continual is not improved or even decreased. It is mentioned that TENT-continual outperforms the BN Stats Adapt in earlier stages of the adaptation. However, after adapting to several types of corruption, the performance of TENT-continual decreases rapidly. This shows that the TENT-continual can not deal with the error accumulation or forgetting problem in continuous adaptation, and may be unstable under long-term continuous adaptation. CoTTA takes the error accumulation into account to further improve the performance. Our method achieves the best results in the average error value and most of the corruption-type data. Table 1: Classification error rate (%) for the standard CIFAR10-to-CIFAR10C continual test-time adaptation task. All results are evaluated with the largest corruption severity level 5 in an online fashion. Bold text indicates the best performance. | Time - Method | t ---------------------------------------------------------------------→ - Gaussian | t ---------------------------------------------------------------------→ - shot | t ---------------------------------------------------------------------→ - impulse | t ---------------------------------------------------------------------→ - defocus | t ---------------------------------------------------------------------→ - glass | t ---------------------------------------------------------------------→ - motion | t ---------------------------------------------------------------------→ - zoom | t ---------------------------------------------------------------------→ - snow | t ---------------------------------------------------------------------→ - frost | t ---------------------------------------------------------------------→ - fog | t ---------------------------------------------------------------------→ - brightness | t ---------------------------------------------------------------------→ - contrast | t ---------------------------------------------------------------------→ - elastic trans | t ---------------------------------------------------------------------→ - pixelate | t ---------------------------------------------------------------------→ - jpeg | t ---------------------------------------------------------------------→ - Mean | |------------------------------------|---------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------| | Source | 72.3 | 65.7 | 72.9 | 46.9 | 54.3 | 34.8 | 42.0 | 25.1 | 41.3 | 26.0 | 9.3 | 46.7 | 26.6 | 58.5 | 30.3 | 43.5 | | BN Stats Adapt | 28.1 | 26.1 | 36.3 | 12.8 | 35.3 | 14.2 | 12.1 | 17.3 | 17.4 | 15.3 | 8.4 | 12.6 | 23.8 | 19.7 | 27.3 | 20.4 | | Pseudo-Label | 26.7 | 22.1 | 32.0 | 13.8 | 32.2 | 15.3 | 12.7 | 17.3 | 17.3 | 16.5 | 10.1 | 13.4 | 22.4 | 18.9 | 25.9 | 19.8 | | TENT-continual [Wang et al., 2020] | 24.8 | 20.5 | 28.5 | 14.5 | 31.7 | 16.2 | 15.0 | 19.2 | 17.6 | 17.4 | 11.4 | 16.3 | 24.9 | 21.6 | 26.0 | 20.4 | | CoTTA [Wang et al., 2022] | 24.6 | 21.9 | 26.5 | 11.9 | 27.8 | 12.4 | 10.6 | 15.2 | 14.4 | 12.8 | 7.4 | 11.1 | 18.7 | 13.6 | 17.8 | 16.5 | | Ours | 23.9 | 20.5 | 24.5 | 11.2 | 26.3 | 11.8 | 10.1 | 14.0 | 12.7 | 11.5 | 7.6 | 9.5 | 17.6 | 12.0 | 15.8 | 15.2 | Table 2: Ablation experiments for the CIFAR10-to-CIFAR10C task. 'AugTeach' means the pseudo-labels are generated by the teacher model. 'STA' means the self-weighted thresholding cross-entropy loss. 'SWC' means Soft-weighted contrastive learning, and 'SWA' means Soft Weight Alignment strategy. | Time - Method | t ---------------------------------------------------------------------→ - Gaussian | t ---------------------------------------------------------------------→ - shot | t ---------------------------------------------------------------------→ - impulse | t ---------------------------------------------------------------------→ - defocus | t ---------------------------------------------------------------------→ - glass | t ---------------------------------------------------------------------→ - motion | t ---------------------------------------------------------------------→ - zoom | t ---------------------------------------------------------------------→ - snow | t ---------------------------------------------------------------------→ - frost | t ---------------------------------------------------------------------→ - fog | t ---------------------------------------------------------------------→ - brightness | t ---------------------------------------------------------------------→ - contrast | t ---------------------------------------------------------------------→ - elastic trans | t ---------------------------------------------------------------------→ - pixelate | t ---------------------------------------------------------------------→ - jpeg | t ---------------------------------------------------------------------→ - Mean | |---------------------------|---------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------| | baseline | 27.3 | 23.9 | 32.1 | 12.7 | 30.2 | 14.1 | 12.3 | 17.4 | 17.0 | 15.5 | 9.5 | 15.0 | 20.4 | 15.9 | 20.9 | 19.0 | | +AugTeach | 25.6 | 22.5 | 27.8 | 12.6 | 29.5 | 14.3 | 12.5 | 17.3 | 16.7 | 15.4 | 9.5 | 15.1 | 20.9 | 15.9 | 19.7 | 18.4 | | +AugTeach+SAT | 24.5 | 20.5 | 25.2 | 12.0 | 26.8 | 13.1 | 11.5 | 16.0 | 14.6 | 13.2 | 8.3 | 11.8 | 19.3 | 14.1 | 16.4 | 16.5 | | +AugTeach+SAT+SWC w/o FFT | 24.4 | 20.4 | 25.2 | 11.4 | 26.0 | 12.5 | 10.8 | 15.2 | 13.6 | 12.7 | 8.1 | 11.4 | 18.5 | 13.5 | 16.0 | 15.9 | | +AugTeach+SAT+SWC | 23.5 | 20.5 | 24.4 | 11.2 | 26.2 | 12.1 | 10.3 | 14.3 | 13.1 | 12.0 | 7.5 | 9.3 | 17.9 | 12.9 | 16.3 | 15.4 | | +AugTeach+SAT+SWC+SWA | 23.9 | 20.5 | 24.5 | 11.2 | 26.3 | 11.8 | 10.1 | 14.0 | 12.7 | 11.5 | 7.6 | 9.3 | 17.6 | 12.0 | 15.8 | 15.2 | Table 3: Gradually changing setup results on CIFAR10-to-CIFAR10C. The severity level changes gradually between the lowest and the highest. Results are the mean over ten diverse corruption-type sequences. Bold text indicates the best performance. | Avg. Error (%) | Source | BNAdapt | TENT-continual [Wang et al., 2020] | CoTTA [Wang et al., 2022] | Ours | |------------------|----------|-----------|--------------------------------------|-----------------------------|-----------| | CIFAR10C | 24.8 | 13.7 | 29.2 | 10.4 ± 0.3 | 8.0 ± 0.5 | Compared with the strongest baseline CoTTA, the average error value of our method is reduced from 16.5% to 15.2%. Ablation Studies. In addition, we also conducted ablation experiments to prove the effectiveness of each module of our method. The results are shown in Table 2. All modules we propose are helpful for performance gains. Following [Wang et al., 2022], baseline is a combination of BN Stats Adapt and Pseudo-Label. Then we add the 'AugTeach' strategy to the baseline model, and the average error value dropped from 19.0% to 18.4%. Further, the addition of the 'STA' strategy reduced the model's average error rate from 18.4% to 16.5%, and 'SWC' loss further reduces the average error to 15.5%. It is worth mentioning that after adding FFT augmentations, the test model achieved better results. Next, the addition of 'SWA' has led to an increase in overall performance and a significant reduction in the error of subsequent adaptations. Inevitably, adding this additional constraint will slow down the adaptation, for example, the addition of 'SWA' makes the performance of the first three corrupted data worse than not adding. Finally, We randomly selected a batch of data in the middle and late stages, and the visualization results are shown in Fig. 3. The results demonstrate that the proposed label selection strategy is effective for instantaneous parameter learning. Figure 2: The influence of λ1 and λ2 ![Image](https://ijcai.org/proceedings/2023/fulltext/183/image_000001_55503950ae22b5a436c5bfad1c0a61f8f1986409ecff1a9c4a25fe491b069b95.png) Parameters Analysis. We also investigate the parameter sensitivity in the proposed method. Fig. 2 represents the change of accuracies with different loss weights, which indicates that our method is insensitive to the parameters λ1 and λ2 in the range of [0.1,2]. Gradually Changing Setup. Following [Wang et al., 2022], we also consider a gradually changing setup. For the standard setup, corruption types change abruptly in the highest severity. For the gradually changing setup, the corruption types change is gradual. Table 4: Classification error rate (%) for the standard CIFAR100-to-CIFAR100C continual test-time adaptation task. All results are evaluated with the largest corruption severity level 5 in an online fashion. Bold text indicates the best performance. | Time - Method | t ---------------------------------------------------------------------→ - Gaussian | t ---------------------------------------------------------------------→ - shot | t ---------------------------------------------------------------------→ - impulse | t ---------------------------------------------------------------------→ - defocus | t ---------------------------------------------------------------------→ - glass | t ---------------------------------------------------------------------→ - motion | t ---------------------------------------------------------------------→ - zoom | t ---------------------------------------------------------------------→ - snow | t ---------------------------------------------------------------------→ - frost | t ---------------------------------------------------------------------→ - fog | t ---------------------------------------------------------------------→ - brightness | t ---------------------------------------------------------------------→ - contrast | t ---------------------------------------------------------------------→ - elastic trans | t ---------------------------------------------------------------------→ - pixelate | t ---------------------------------------------------------------------→ - jpeg | t ---------------------------------------------------------------------→ - Mean | |------------------------------------|---------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------| | Source | 73.0 | 68.0 | 39.4 | 29.3 | 54.1 | 30.8 | 28.8 | 39.5 | 45.8 | 50.3 | 29.5 | 55.1 | 37.2 | 74.7 | 41.2 | 46.4 | | BN Stats Adapt | 42.1 | 40.7 | 42.7 | 27.6 | 41.9 | 29.7 | 27.9 | 34.9 | 35.0 | 41.5 | 26.5 | 30.3 | 35.7 | 32.9 | 41.2 | 35.4 | | Pseudo-Label | 38.1 | 36.1 | 40.7 | 33.2 | 45.9 | 38.3 | 36.4 | 44.0 | 45.6 | 52.8 | 45.2 | 53.5 | 60.1 | 58.1 | 64.5 | 46.2 | | TENT-continual [Wang et al., 2020] | 37.2 | 35.8 | 41.7 | 37.7 | 50.9 | 48.5 | 48.5 | 58.2 | 63.2 | 71.4 | 72.0 | 83.1 | 88.6 | 91.6 | 95.1 | 61.6 | | CoTTA [Wang et al., 2022] | 40.1 | 37.7 | 39.7 | 26.8 | 38.0 | 27.9 | 26.5 | 32.9 | 31.7 | 40.4 | 24.6 | 26.8 | 32.5 | 28.1 | 33.8 | 32.5 | | Ours | 39.4 | 36.4 | 37.4 | 25.0 | 36.0 | 26.6 | 25.0 | 29.1 | 28.4 | 35.0 | 23.5 | 25.1 | 28.5 | 25.8 | 29.6 | 30.0 | Table 5: Average error of standard ImageNet-to-ImageNet-C experiments over 10 diverse corruption sequences. All results are evaluated with the largest corruption severity level 5 in an online fashion. | Avg. Error (%) | Source | BNAdapt | TENT [Wang et al., 2020] | CoTTA [Wang et al., 2022] | Ours | |------------------|----------|-----------|----------------------------|-----------------------------|------------| | ImageNet-C | 82.4 | 72.1 | 66.5 | 63.0 ± 2.3 | 62.1 ± 2.3 | Figure 3: Visualization of the discriminative capability of the sample features on CIFAR10C. Colors represent sample classes. ![Image](https://ijcai.org/proceedings/2023/fulltext/183/image_000002_aded03c8b0d387642d9d282a9cfd4b746316a7d9ff40b6163cd7408d6af6a4c7.png) Specifically, the change process can be expressed as follows: . . .2-→1 ︸ ︷︷ ︸ change ----→ t-1 and before ︸ ︷︷ ︸ type 1-→2-→3-→4-→5-→4-→3-→2-→1 corruption type t, gradually changing severity ︸ ︷︷ ︸ type 1-→2. . . t+1 and after - , where the number represents the corruption severity. The corruption type changes when the severity level is the lowest. In addition, the severity level changes within each type are also gradual. ## 4.5 CIFAR100-to-CIFAR100C As shown in Table 4, TENT-continual shows the highest average error rate, and its performance is even worse than the Source model of doing nothing. Pseudo-Label also offers a similar performance degradation curve. In the early stage of adaptation, the performance of Pseudo-Label is better than BN stats adapt and Source, but lower than these two methods in the later stage. This phenomenon is also caused by error accumulation. CoTTA considers the problem of error accumulation and reduces the error to 32.5%. Further, we propose a self-adaptive threshold loss to mitigate the instan- change ----→ taneous impact of noisy pseudo-labels on optimization objective, and offer a SWA strategy to alleviate the long-term knowledge forgetting problem in continuous adaptation. The performance of our method is better than CoTTA on all corruption types of data, and the average error value is reduced to 30.0%. ## 4.6 ImageNet-to-ImageNet-C We also make experiments on ImageNet dataset. Following [Wang et al., 2022], we conduct ImageNet-to-ImageNet-C experiments over ten diverse corruption type sequences in severity level 5. The average result of ten experiments is shown in Table 5. ImageNet is more complex than CIFAR100 and CIFAR-10, and the overall average test error is also greater. Our method outperforms other competing methods and reduces the average test error to 62.1%. ## 5 Conclusion In this paper, we propose a simple yet effective framework for continual test-time domain adaptation, which refines the pseudo-label learning process from the perspective of the instantaneous and long-term impact of noisy pseudo-labels. Firstly, we propose a self-adaptive thresholding Cross-Entropy loss to optimize the adaptation process, which facilitates learning the adapted model. Secondly, the learned thresholds and pseudo-labels are utilized to select more discriminative positive and precise negative pairs for contrastive learning. Finally, we propose a Soft Weight Alignment strategy to normalize the distance between the parameters of the adapted model and the source pre-trained model, which improves the classification accuracy in the late stage of adaptation. Extensive experimental results demonstrate that our method achieves state-of-the-art performance on various benchmark datasets. ## Acknowledgements Our work was supported by Joint Fund of Ministry of Education of China (8091B022149), Key Research and Development Program of Shaanxi (2021ZDLGY01-03), National Natural Science Foundation of China (62132016, 62171343, 62071361 and 62201436), and Fundamental Research Funds for the Central Universities (ZDRC2102 and ZYTS23135). ## Contribution Statements Xu Yang and Yanan Gu contribute equally. They jointly designed the experiments and wrote the manuscript. Kun Wei and Cheng Deng contributed to the writing of the manuscript. Cheng Deng supervise the project. ## References 1. [Chen et al., 2022] Dian Chen, Dequan Wang, Trevor Dar- rell, and Sayna Ebrahimi. Contrastive test-time adaptation. arXiv preprint arXiv:2204.10377, 2022. 2. [Croce et al., 2020] Francesco Croce, Maksym An- driushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670, 2020. 3. [Cui et al., 2020] Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. Towards dis- criminability and diversity: Batch nuclear-norm maxi- mization under label insufficient situations. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3941-3950, 2020. 4. [Dong et al., 2020] Jiahua Dong, Yang Cong, Gan Sun, Bi- neng Zhong, and Xiaowei Xu. What can be transferred: Unsupervised domain adaptation for endoscopic lesions segmentation. In IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 4022-4031, June 2020. 5. [Dong et al., 2021] Jiahua Dong, Yang Cong, Gan Sun, Zhen Fang, and Zhengming Ding. Where and how to transfer: Knowledge aggregation-induced transferability perception for unsupervised domain adaptation. pages 1-1, 2021. 6. [Ganin and Lempitsky, 2015] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backprop- agation. In International conference on machine learning, pages 1180-1189. PMLR, 2015. 7. [Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The jour- nal of machine learning research, 17(1):2096-2030, 2016. 8. [Grandvalet and Bengio, 2004] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimiza- tion. Advances in neural information processing systems, 17, 2004. 9. [Hendrycks and Dietterich, 2019] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network ro- bustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019. 10. [Hendrycks et al., 2019] Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781, 2019. 11. [Kundu et al., 2020] Jogendra Nath Kundu, Naveen Venkat, R Venkatesh Babu, et al. Universal source-free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4544- 4553, 2020. 12. [Kurmi et al., 2021] Vinod K Kurmi, Venkatesh K Subrama- nian, and Vinay P Namboodiri. Domain impression: A source data free domain adaptation method. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 615-625, 2021. 13. [Lee and others, 2013] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896, 2013. 14. [Li et al., 2016] Yanghao Li, Naiyan Wang, Jianping Shi, Ji- aying Liu, and Xiaodi Hou. Revisiting batch normal- ization for practical domain adaptation. arXiv preprint arXiv:1603.04779, 2016. 15. [Li et al., 2020] Rui Li, Qianfen Jiao, Wenming Cao, Hau- San Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9641-9650, 2020. 16. [Liang et al., 2020] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pages 6028-6039. PMLR, 2020. 17. [Liu et al., 2021] Yuang Liu, Wei Zhang, and Jun Wang. Source-free domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1215-1224, 2021. 18. [Lu et al., 2022] Wang Lu, Jindong Wang, Haoliang Li, Yiqiang Chen, and Xing Xie. Domain-invariant feature exploration for domain generalization. arXiv preprint arXiv:2207.12020, 2022. 19. [Polyak and Juditsky, 1992] Boris T Polyak and Anatoli B Juditsky. Acceleration of stochastic approximation by averaging. SIAM journal on control and optimization, 30(4):838-855, 1992. 20. [Prabhu et al., 2021] Viraj Prabhu, Shivam Khare, Deeksha Kartik, and Judy Hoffman. Sentry: Selective entropy opti- mization via committee consistency for unsupervised do- main adaptation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 8558-8567, 2021. 21. [Saito et al., 2019] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi- supervised domain adaptation via minimax entropy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8050-8058, 2019. 22. [Schneider et al., 2020] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, 33:11539-11551, 2020. 23. [Sun et al., 2019] Yu Sun, Eric Tzeng, Trevor Darrell, and Alexei A Efros. Unsupervised domain adaptation through self-supervision. arXiv preprint arXiv:1909.11825, 2019. 24. [Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. Advances in neural information pro- cessing systems, 30, 2017. 25. [Tzeng et al., 2017] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE confer- ence on computer vision and pattern recognition, pages 7167-7176, 2017. 26. [Wang et al., 2020] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726, 2020. 27. [Wang et al., 2022] Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. arXiv preprint arXiv:2203.13591, 2022. 28. [Xie et al., 2017] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 1492-1500, 2017. 29. [Xu et al., 2021] Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. A fourier-based frame- work for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14383-14392, 2021. 30. [Yang et al., 2019] Xu Yang, Cheng Deng, Feng Zheng, Junchi Yan, and Wei Liu. Deep spectral clustering us- ing dual autoencoder network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 31. [Yang et al., 2021] Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8978-8987, 2021. 32. [Yang et al., 2022] Xu Yang, Cheng Deng, Tongliang Liu, and Dacheng Tao. Heterogeneous graph attention net- work for unsupervised multiple-target domain adaptation. IEEE Transactions on Pattern Analysis and Machine In- telligence, 44(4):1992-2003, 2022. 33. [Yeh et al., 2021] Hao-Wei Yeh, Baoyao Yang, Pong C Yuen, and Tatsuya Harada. Sofa: Source-data-free feature alignment for unsupervised domain adaptation. In Pro- ceedings of the IEEE/CVF Winter Conference on Applica- tions of Computer Vision, pages 474-483, 2021. [Zagoruyko and Komodakis, 2016] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016. 34. [Zhou and Levine, 2021] Aurick Zhou and Sergey Levine. Training on test data with bayesian adaptation for covari- ate shift. arXiv preprint arXiv:2109.12746, 2021.