Meta-Cognitive Resonance Label Correction for Instance-Dependent Noise

1 Introduction

Deep Neural Networks (DNNs) have demonstrated remarkable capabilities in visual recognition tasks, with success attributed to the availability of large-scale, high-quality annotated datasets [Chen et al., 2019]. However, collecting massive datasets with precise annotations is prohibitively expensive and time-consuming. In real-world scenarios, web crawlers are frequently employed, inevitably introducing noisy labels [Fabian Benitez-Quiroz et al., 2016; Yan et al., 2014]. Directly training on such contaminated data leads to severe performance degradation, as high-capacity DNNs tend to overfit noisy samples by memorizing erroneous correlations rather than learning generalizable patterns [Arpit et al., 2017; Zhang et al., 2021]. This motivates the development of Learning with Noisy Labels (LNL) methods that are tailored to large-scale, low-quality datasets.

∗Wenjian Wang is the corresponding author.

Existing LNL methods focus on Class-Conditional Noise (CCN) and often rely on the small-loss criterion for sample selection [Zhao et al., 2022; Li et al., 2020]. However, their performance deteriorates significantly in the presence of Instance-Dependent Noise (IDN), which more faithfully reflects real-world labeling processes. For instance, in the Clothing1M dataset, visually similar classes induce ambiguous decision boundaries, resulting in highly asymmetric noise distributions [Chen et al., 2021]. Under such challenging IDN scenarios, the failure to distinguish semantically similar instances becomes a fundamental bottleneck.

To reveal why existing methods fail on IDN, we analyze the training dynamics on CIFAR-100. We evaluate noise identification using the AUC metric based on the small-loss criterion. A larger AUC indicates a clearer separability between the loss distributions of noisy and clean samples, and thus a stronger noise detection capability. As shown in Fig. 1(a), we observe a vicious cycle, starting from epoch 65, the deterioration of feature quality (K-Nearest Neighbors accuracy) occurs simultaneously with a sharp decline in noise detection (AUC). This implies that noise memorization destroys the structure of the representation space, blinding the model to noise patterns and invalidating the small-loss assumption.

Most existing methods aggravate this issue by training feature extractors from scratch on noisy data, while jointly updating model weights and pseudo-label distributions within a unified framework [Wu et al., 2021; Zheng et al., 2021]. This tightly coupled training paradigm inevitably yields corrupted representations [Tu et al., 2023]. Under conventional CNNs (e.g., ResNet) trained with noisy supervision, the extracted features lack sufficient robustness and tend to collapse semantically similar IDN samples together [Garg et al., 2024], thereby hindering subsequent noise identification and correction. We conduct a decoupling analysis comparing End-toEnd (E2E) baselines with decoupled paradigms as shown in Fig. 1(b). In E2E settings, models suffer from severe feature collapse, where higher capacity (ResNet-50) paradoxically yields lower noise detection AUC (0.63) than ResNet34 (0.66) due to accelerated noise memorization. In contrast, decoupled approaches like SimCLR [Chen et al., 2020] and frozen CLIP dramatically restore the AUC to 0.96 and 0.99, respectively. These results indicate that isolating feature learning from label corruption is critical for breaking this vicious cycle.

Building upon these insights, we propose the Metacognitive Resonance Label Correction (MecReC) framework. By utilizing CLIP as an anchor, MecReC effectively prevents feature collapse under noisy supervision. The main contributions are as follows.

• Decoupled label correction paradigm. To break the vicious cycle where noisy labels corrupt feature learning, we propose a decoupled architecture that isolates label correction from representation learning. By leveraging frozen CLIP features as an anchor, the correction process is guided by robust open-world priors rather than degenerated representations.
• Meta-cognitive resonance signals. Our MecReC method quantifies label reliability via meta-cognitive resonance signals derived from inter-model consensus, intra-model uncertainty, external validation, and temporal consistency. This holistic state representation enables the meta-weight network to capture fine-grained noise patterns beyond any single criterion.
• Differentiable meta correction policy. We design a lightweight meta-weight network to learn a differentiable label rectification policy. Unlike heuristic reweighting methods, the meta-weight network maps resonance signals to adaptive soft labels and is optimized using unbiased gradient feedback from a small clean validation set, explicitly maximizing generalization.
• A training-coupled thresholding strategy with generalization guarantees. The confidence threshold for deciding whether to correct a label is adaptively updated throughout training, rather than being manually specified. Under this strategy, we prove that the model trained on the corrected dataset yields a tighter error bound.

2 RelatedWork

Learning with Noisy Labels and Sample Selection. Learning with noisy labels is typically addressed via robust losses or sample selection. Robust loss functions, such as GCE [Zhang and Sabuncu, 2018] and MAE [Ghosh et al., 2017], mitigate the impact of noisy labels by bounding gradients or balancing convergence speed with robustness. Sample selection methods, starting from Co-teaching [Han et al., 2018], have evolved into hybrid frameworks. SOTA methods, including DivideMix [Li et al., 2020], utilize a Gaussian mixture model to partition data for semi-supervised learning, while confidence-aware approaches like CC [Zhao et al., 2022] and DISC [Li et al., 2023] further refine reliability assessments using prediction probabilities. To enhance feature robustness against noise-induced corruption, recent studies such as SV-Learner [Liang et al., 2024], UniCon [Karim et al., 2022], and PSSCL [Zhang et al., 2025] incorporate contrastive learning into the selection process. However, under high noise rates or IDN, the loss distributions of clean and noisy samples overlap significantly, rendering the smallloss assumption invalid, which inevitably discards potentially useful semantic information within the noisy set. In contrast, MecReC utilizes a soft label correction mechanism that progressively salvages effective supervisory signals from all samples via multi-dimensional consistency checks.

Decoupled Representation Learning. Traditional LNL methods often suffer from the coupling inherent in E2E frameworks, where overfitting to noisy labels corrupts feature representations. Decoupling strategies aim to break this by separating representation learning from noisy supervision. Early efforts include confusion matrix estimation [Tanno et al., 2019], which jointly estimates annotation bias to decouple noise modeling from feature learning. DES [Zhang et al., 2020] adopts a two-stage framework that leverages selfsupervised learning for temporal decoupling of feature acquisition. More recently, optimization-level decoupling has emerged: DMLP [Tu et al., 2023] prevents noisy gradient backpropagation through a decoupled meta-objective, while prototype-guided calibration [Yuan et al., 2025a] employs noise-agnostic prototypes to separate representation learning from classification. However, restricted to visual expression, these approaches fail to separate visually confusing patterns under extreme noise, thereby yielding a less discriminative feature space.

Label Correction and Meta Learning. Meta-learning frameworks leverage a small clean validation set to guide the training process. MW-Net [Shu et al., 2019] first introduces a meta-weight network to re-weight samples and suppress noise. Beyond re-weighting, MSLC [Wu et al., 2021] treats labels as learnable parameters and purifies them via validation gradients. Recent methods further refine the correction process: DCD [Mu et al., 2025] employs dynamic center distances to distinguish hard samples from noise, while DMLP [Tu et al., 2023] and EMLC [Taraday and Baskin, 2023] improve efficiency and reduce confirmation bias. Yet, existing meta-correctors often rely on single-dimension feedback or static thresholds, lacking dynamic control over the correction trajectory. In contrast, our MecReC bridges these gaps by synthesizing multi-dimensional signals, i.e., consistency, entropy, and history, within a bi-level optimization framework, achieving a more stable and adaptive correction policy than prior single-view methods.

Algorithm 1 Meta-cognitive Resonance Label Correction

Input: Noisy training set Dtrain, clean validation set Dval, pretrained feature extractor Clip(·), main models fθ1 , fθ2 , metaweight network gϕ. Output: Optimized model parameters θ∗ 1 , θ∗ 2 , ϕ∗ , corrected label set {ˆ yi}Ni=1 . 1: For all x in Dtrain and Dval, compute zi = Clip(xi). 2: for epoch t = 1 to T do 3: for each mini-batch Btrain ⊆ Dtrain do 4: if t > twarm-up then 5: Get feature ztrain and current labels ˆ ytrain. 6: Calculate ˆ ymeta i for each sample by Eq. (3). 7: Update ϕ by solving the objective in Eq. (7). 8: Compute Si by Eqs. (1), (2), (4), (5), (6). 9: Generate λi = g(xi, ϕ) based on Si. 10: Update θk ← θk - αmain∇θk Lmain, k ∈ {1, 2}. 11: Compute confidence score Confi by Eq. (12). 12: If Confi > σt i , update label ˆ yi by Eq. (14). 13: end if 14: end for 15: end for

3 Methodology

The overall framework of MecReC is shown in Fig. 2 and the pseudo-code is presented in Algorithm 1. MecReC recovers high-quality supervisory signals from noisy data through a co-learning paradigm consisting of two peer networks fθ1 and fθ2 , guided by a meta-weight network gϕ.

Let Dtrain = {(xi, yi)}Ni=1 be the noisy training set and Dval = {(xval i , yval i )}Mi=1 be a small clean validation set.

3.1 CLIP Decoupled Feature Extraction

CLIP [Radford et al., 2021] demonstrates outstanding zeroshot generalization capability through training on massive image-text pairs. We denote the pre-trained and frozen CLIP image encoder as Clip(·). For any input image xi, its feature is extracted as zi = Clip(xi). These frozen features serve as inputs for both the dual main models and the meta-weight network, ensuring that the correction strategy is derived from stable and noise-invariant representations.

3.2 Meta-Cognitive Resonance Signals

MecReC performs label reliability assessment from multiple views through a mechanism termed meta-cognitive resonance. For each sample, we construct a state vector Si by aggregating four dynamic signals, which capture label correction confidence from complementary sources.

Prediction Consistency (PC). The semantic consensus between two peer networks is measured by the cosine similarity of their predicted probability distributions:

where pV1 i = Softmax(fθ1 (zi)), pV2 i = Softmax(fθ2 (zi)). Higher consistency indicates more reliable predictions.

Dynamic Entropy Consistency (DEC). This signal captures the synchronization of prediction uncertainty between the two peer networks:

where EntVk,i = - ∑Cc=1 pVk i,c log (pVk i,c + ϵ). Here, ϵ is a small constant for numerical stability, C is the number of classes, and logC represents the maximum entropy value. Closer alignment suggests higher reliability.

Meta-Guided Consistency (MGC). This signal injects external reliable knowledge using a small clean validation set. For each training sample feature zi, we retrieve its KNN in Dval, and construct a meta-guided pseudo-label by similarityweighted aggregation:

where wi,j denotes the normalized feature similarity between zi and its j-th neighbor, satisfying wi,j ≥ 0 and ∑zj∈KNN(Dval) wi,j = 1.

The quality of this guidance, denoted qi, is measured by the normalized average similarity over the first-K neighbors. Then, we compute the consistency between the average prediction pi = [Softmax(fθ1 (zi)) + Softmax(fθ2 (zi))]/2 and the meta-guided label ymeta i dynamically modulated by qi:

Historical Correction Consistency (HCC). To mitigate potential over-correction and prevent label oscillation, we introduce a historical consistency signal that tracks correction stability over time. This signal is defined based on the cumulative correction count hi up to the current epoch t.

A lower correction frequency implies high label stability. Finally, the input state vector Si is:

3.3 Meta-Guided Optimization Procedure

Bi-level Meta Loop. Formally, we cast the noise-robust training process as a bi-level optimization problem. Our objective is to learn the meta-weight network parameters ϕ, to achieve minimal loss on a clean validation set. Specifically, the outer objective minimizes the validation loss Lval, while the inner objective defines a temporary, differentiable update of the main model parameters θk to ˆ θk (k ∈ {1, 2}):

Computing the gradient of the outer objective with respect to ϕ, i.e., ∇ϕLval, is challenging as it requires differentiating through the inner optimization trajectory. To achieve this efficiently, we employ automatic-differentiation-based optimization techniques (implemented via the higher library). Fig. 3 shows the micro-circulation flow in detail.

1. Meta-weight generation. For each sample xi ∈ Btrain, the meta-weight network generates an instance-specific loss weight λi = g(xi, ϕ) based on its input state vector Si.
2. Virtual loss computation. We define a meta-weight virtual training loss for the current batch. To guide the model towards stability, this loss targets the current smoothed label estimate ˆ yi (Eq. (14)).

1. Differentiable virtual update. Following the bi-level optimization objective in Eq. (7), a differentiable virtual update is performed on the main model parameters, yielding the virtually updated parameters, ˆ θk = θk - αmain∇θk Lvirt. Crucially, the computational graph of this step is retained to enable higher-order meta-gradient computation.
2. Meta-gradient calculation. The virtually updated model fˆ θk is evaluated over the full clean validation Dval. The validation loss Lval(ϕ) is computed through the virtual update path via the chain rule to capture how changes in ϕ affect validation performance by influencing ˆ θ:

1. Meta-weight network update. Finally, the meta-weight network parameters are updated using the computed metagradient: ϕ ← ϕ - αmeta∇ϕLval. This update mechanism enables the meta-weight network to learn a label-correction strategy that explicitly maximizes the generalization ability of the main models.

Main Model Actual Training. After each meta-update (updating ϕ), the main models (fθ1 and fθ2 ) perform a real gradient descent step on the current training batch. This process utilizes the high-quality meta weights (λ) generated by the updated meta-weight network to dynamically reweight the loss function, thereby enabling effective noiserobust training. Specifically, for each main model fθk (with k ∈ {1, 2}), the real training loss LVk i for the i-th sample is defined as a weighted combination of the original label loss and the meta-guided loss:

The meta weight λi ∈ [0, 1] acts as an adaptive balancing factor: a larger λi encourages the model to learn more from the meta-guided label, while a smaller λi prioritizes the current smoothed label.

The total training loss for the main models is defined as the average loss of the two peer networks over the current Btrain:

The parameters of the main models fθ1 and fθ2 are updated by θk: θk ′ ← θk - αmain∇θk Lmain, with λi treated as a constant during this phase.

3.4 Adaptive Meta-label Correction

To refine training targets and combat noise, we employ a soft label evolution strategy based on Meta-cognitive Resonance. In this phase, the meta-weight network g(·; ϕ) is designed to output not only the loss reweighting factor λi, but also two critical parameters: a correction attention weight vector ai ∈ R4 (softmax-normalized) targeting the four resonance signals, and an instance-specific update strength µi ∈ [0, 1].

We first compute a composite confidence score by aggregating the signal vector Si via weighted summation:

It quantifies the necessity for correction of the current label.

To ensure high-precision correction, we introduce a personalized threshold σt i by Eq. (16), which is adaptively adjusted throughout training. For samples whose confidence score exceeds this threshold (Confi > σt i ), we generate a refined soft label target ˆ ytarget i . This target is a weighted combination of the model ensemble prediction pi and the metaguided label ymeta i :

Finally, the current smoothed label estimate is updated via an Exponential Moving Average mechanism, where µi is produced by the meta-weight network:

This dynamic and instance-specific update mechanism ensures that the correction process can rapidly adapt to severe label noise while maintaining training stability and robustness.

The confidence threshold (Confi > σt i ) is determined based on the following theoretical findings.

Theorem 1 (Generalization guarantee). Let D = {(xi, yi)}Ni=1 be a classification dataset with noisy labels, and let D+ denote the corrected dataset obtained by a confidence-based label correction procedure. Let B(h;D) denote the Rademacher-based generalization bound of a hypothesis h ∈ H trained on dataset D. Suppose that for each corrected sample, the updated confidence satisfies

where Conf ti and Acct denote the label confidence and model accuracy at iteration t, respectively, σp is a noise tolerance parameter (σp > 0), and the accuracy improvement in one correction step ∆Acc = Acct+1 - Acct . Then, under the above condition, the Rademacher generalization bound is strictly reduced, i.e., B(h;D+) < B(h;D).

Theorem 1 shows that correcting labels for highconfidence samples, characterized by consistent signals, leads to improved generalization. Accordingly, we set the confidence threshold as

such that Conf t+1 i > σt i satisfies the sufficient condition in Eq. (15) and guarantees a tighter error bound. The term ∆Acc · Conf t+1 i is omitted since accuracies across adjacent iterations are close, and this simplification introduces a conservative margin to prevent over-correction. Since the true model accuracy is unavailable during training, we approximate its evolution using a surrogate functionAcct = 1- c ˜ t+2c , where c is a positive constant, and ˜ t is the normalized epoch. In our experiments, we set the hyperparameters as σp=0.15, c=0.20.

4 Experiments

4.1 Results on Synthetic Noisy Datasets

We evaluate MecReC on the standard synthetic noisy datasets CIFAR-10 and CIFAR-100. To simulate realistic scenarios, we generate two types of instance-dependent noise: Part-dependent label noise (PDL) [Xia et al., 2020] and Classification-based label noise (CBL) [Chen et al., 2021], with noise rates set to 20%, 40%, and 60%. Unlike traditional end-to-end approaches, we employ a pre-trained CLIP ViTL/14 as a frozen feature extractor to leverage robust semantic representations. Following standard meta-learning-based protocols [Wu et al., 2021; Shu et al., 2019], we reserve 1000 clean images as the validation set, using the remaining samples for training.

Tables 1 and 2 present the results comparisons on CIFAR10 and CIFAR-100 under PDL and CBL noise, respectively. MecReC consistently outperforms all competing baselines across varying noise rates, with the performance gap becoming particularly pronounced in high-noise regimes. Specifically, in the challenging setting of CIFAR-100 with 60% PDL noise, the MecReC achieved 79.46% accuracy, more than 2.12% higher than the next best method. Generally, methods leveraging ViT backbones surpass conventional approaches.

MecReC, in particular, stands out by achieving the best results within this category. A key distinction is that while competing ViT-based methods suffer from performance deterioration under rising noise levels, MecReC demonstrates remarkable robustness.

We visualize the label correction quality in Fig. 4. MecReC demonstrates the highest label accuracy and superior stability across all noise ratios. Notably, it shows a much smoother convergence trajectory under EMA guidance. This mechanism effectively stabilizes training and mitigates the memorization of corrupted labels observed in baselines.

4.2 Results on Real-world Noisy Datasets

We evaluate MecReC on three widely used real-world noisy benchmarks, covering diverse noise patterns and dataset scales. As shown in Table 3 and 4, our method achieves SOTA performance across all evaluated scenarios. On the large-scale Clothing1M and WebVision, MecReC attains accuracies of 76.04% and 85.24%, respectively, achieving the best overall performance, with particularly notable gains on WebVision. Notably, on WebVision, our model demonstrates strong generalization, achieving 83.72% accuracy on the clean ILSVRC12 validation set. Furthermore, on Animal10N, which is characterized by challenging fine-grained label noise, our approach effectively distinguishes subtle visual differences, improving the accuracy to 90.82%. These consistent gains confirm the robustness and generalization capability of our framework in handling complex, real-world label noise.

4.3 Ablation Studies

To comprehensively validate the effectiveness and necessity of the core components within the MecReC framework, we conducted ablation studies on the CIFAR-10 dataset under a 40% PDL instance-dependent label noise setting.

Component Analysis. We compare the full MecReC model against variants where specific modules are either removed or replaced. As presented in Table 5, the baseline without label correction suffers from a low label accuracy of 60%. In contrast, the full MecReC framework achieves a remarkable label accuracy of 96.65% (+36.65%) while achieving the highest test accuracy of 97.13%. Ablating metaweight network or resonance signals reduces the label accuracy to 80.30% and 79.40%, which strongly validates the necessity of each component in our framework for effectively correcting noisy labels.

Meta-Data Size Analysis. We analyze the impact of metadata size on model performance in Table 6. Although a positive correlation exists between meta-set size and accuracy, the improvement is subtle. The model yields a high accuracy of 97.09% at the 1% ratio, which is comparable to the 97.16% accuracy obtained with 6% data. This suggests that MecReC can effectively extract guidance from very limited clean data.

Computational Efficiency. Fig. 5 illustrates the training efficiency on CIFAR-100 (40% CBL noise). MecReC

achieves the lowest computational cost, requiring only 31.4 seconds per epoch. The reported per-epoch runtime excludes the one-time CLIP feature extraction cost, which is performed only once before training. This efficiency stems from our decoupled training strategy where features are pre-extracted and cached. By bypassing expensive backbone back-propagation, the runtime is limited to label correction and classifier optimization.

5 Conclusion

In this paper, we propose MecReC to address the challenging problem of learning with IDN label noise. We fundamentally rethink the interaction between features and labels by leveraging frozen pre-trained features to break their vicious coupling, providing a stable foundation for noise correction. We further introduced the meta-cognitive resonance, which integrates multi-perspective complementary signals to comprehensively quantify label reliability. By these components, our meta-weight network is capable of learning dynamic correction strategies in a data-driven manner. Our principled confidence thresholding strategy guarantees a tighter error bound for label purification. Extensive empirical evaluations validate the robustness and versatility of MecReC across a wide spectrum of noise types and levels.

Limitations

MecReC relies on a small clean validation set for metaguided optimization, which may be costly in some real-world scenarios. Moreover, the surrogate accuracy function used in the adaptive thresholding strategy is an approximate estimation and may introduce deviations under different domains or data distributions.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (U21A20513, 62576201, 62476157, 62276161).