Label Distribution Imputation and Bias-Corrective Representation Learning for Incomplete and Imbalanced Label Distribution

1 Introduction

In supervised learning, a common classification paradigm is single-label learning (SLL) [Krizhevsky et al., 2012], in which each instance is assigned a single label. When multiple labels can describe an instance, multi-label learning

∗Corresponding Author.

(MLL) [Zhang and Zhou, 2013; Tsoumakas and Katakis, 2008] treats label relevance as binary indicators. However, in real-world scenarios, labels often describe an instance to different extents rather than being simply relevant or irrelevant. To capture this ambiguity, Label Distribution Learning (LDL) [Geng, 2016] assigns a label distribution to each instance. Each element in this distribution corresponds to a description degree that quantifies the strength with which a specific label characterizes the instance. Owing to its ability to capture fine-grained label ambiguity, LDL has been successfully applied to numerous real-world tasks, including age estimation [He et al., 2021; Yu et al., 2023; Lee et al., 2024], emotion analysis [Wang and Li, 2025; Zhao et al., 2025], and facial beauty perception [Li et al., 2024; Cheng et al., 2024; Liu et al., 2025], etc.

LDL [Geng, 2016] is widely adopted for modeling label ambiguity. Standard LDL models are typically trained by minimizing the KL divergence between predicted and observed label distributions. WInLDL [Li and Chen, 2024] shows that more accurate modeling of label distributions improves performance. This holds when label distributions are mostly complete and description degrees exhibit low variance. However, in practice, label distributions are often incomplete. Description degrees can also vary substantially across labels, resulting in imbalanced label distributions. Under such imbalance, optimization is dominated by labels with high description degrees, while labels with low description degrees receive insufficient supervision [Gao et al., 2025]. Missing entries in label distributions further exacerbate this issue, which weakens supervision for labels with low description degrees and distorts the already skewed training signal. As illustrated in Fig. 1, the impact of missing labels becomes more pronounced under imbalanced label distributions, further skewing optimization and degrading model performance. Nevertheless, existing methods typically address label missingness or degree imbalance in isolation. There is no relevant research work that addresses the challenges in this more complex scenario.

In real-world scenarios, acquiring complete label distributions is often impractical due to high annotation costs. Existing incomplete label distribution learning (InLDL) methods mainly adopt two approaches: (i) mitigating missing information through suppression, smoothing, or weighted modeling (e.g., WInLDL [Li and Chen, 2024] and GLDL [Jin et al., 2024]), and (ii) reconstructing distributions by exploiting label correlations via a low-rank structure or matrix factorization [Kou et al., 2024; Kou et al., 2025]. However, most approaches treat imputation as a standalone step and fail to address how imputation errors propagate to representation learning. In imbalanced label distribution learning (ILDL), a few dominant labels contribute most of the total description degree. As a result, optimization is driven mainly by these labels, while labels with low description degrees receive weak supervision and insufficient updates [Gao et al., 2025]. Even under randommissingness, imputation relies on recovered degrees whose reliability varies across entries, making explicit reliability modeling crucial. In this setting, imputation errors on entries with high description degrees may dominate optimization and bias the training signal. Imputation can also introduce reconstruction noise, which may distort optimization dynamics and amplify bias in learned representations. While recent works address imbalance through representation alignment [Zhao et al., 2023] or component decoupling [Gao et al., 2025], they generally apply uniform strategies and overlook the interaction between imputation errors and the degree disparity across labels.

To address these challenges, we propose Label Distribution Imputation and Bias-Corrective Representations (LDIBR), a unified framework for learning with random label missingness under imbalanced label distributions. As shown in Fig. 2, LDIBR follows a two-stage design with correlationpreserving imputation and asymmetric bias mitigation. Unlike conventional methods, it further leverages a low-rank structure for bias correction. The completed distributions are decomposed into a shared semantic core and an instancespecific residual, enabling asymmetric feature modulation that reinforces the reliable core while suppressing imputation noise. Additionally, we provide a theoretical analysis and derive a generalization error bound that links the expected risk to the imputation and representation errors, with explicit dependence on the imbalance factor and the low-rank basis rank. Extensive experiments on multiple benchmark datasets validate the effectiveness and robustness of the LDIBR.

Our main contributions are summarized as follows:

• We propose LDIBR, a unified framework for label distribution imputation and bias-corrective representation learning under random label missingness and imbalanced label distributions. To the best of our knowledge, this is the first work in this more complex scenario.
• We theoretically analyze LDIBR and derive a general-

ization error bound that explicitly characterizes the effects of imputation error, representation bias, and description degree imbalance.

• We conduct extensive experiments under varying missing ratios and imbalance levels, demonstrating the effectiveness and robustness of LDIBR.

2 RelatedWork

2.1 Label Distribution Learning

Label Distribution Learning (LDL) assigns each instance a label distribution, where each element denotes the degree to which a label characterizes the instance. LDL has been widely applied to tasks with inherent label ambiguity, such as age estimation [He et al., 2021; Yu et al., 2023; Lee et al., 2024], facial beauty perception [Li et al., 2024; Cheng et al., 2024; Liu et al., 2025], and emotion analysis [Wang and Li, 2025; Wu et al., 2025; Zhao et al., 2025], etc. Recent advances have improved LDL by exploiting richer structural information, including graph-based models that capture label correlations [Jin et al., 2024] and diffusion-based methods that recover structurally coherent label distributions under noisy supervision [Hou et al., 2025; Wang et al., 2024]. Nevertheless, most existing LDL approaches still assume complete and balanced label distributions, which limits the applicability in practical scenarios.

2.2 Imbalanced Label Distribution Learning

Imbalanced Label Distribution Learning (ILDL) considers scenarios where description degrees vary significantly across labels, resulting in highly skewed label distributions. To mitigate performance degradation caused by label imbalance, [Zhao et al., 2023] proposed Representation Distribution Alignment (RDA), which aligns feature and label distributions in a shared latent space via KL divergence. RDA alleviates the feature shift induced by imbalance at the distribution level. However, it relies on a global alignment objective and cannot adaptively adjust the contributions of labels with different description degrees during representation learning. More recently, [Gao et al., 2025] introduced Decoupled Imbalanced Label Distribution Learning (DILDL), which decomposes label distributions into dominant and non-dominant components and optimizes them using separate branches. While this decoupling strategy mitigates the suppression of non-dominant labels, DILDL still relies on static weighting schemes, shared optimization hyperparameters, and fully observed label distributions. Crucially, it does not exploit distribution components to provide fine-grained, instance-adaptive regulation for representation learning.

2.3 Incomplete Label Distribution Learning

Incomplete Label Distribution Learning (InLDL) focuses on learning label distributions when only partial annotation information is available for each instance. Early InLDL methods address incompleteness by exploiting structural assumptions of label distributions, such as low-rank structures within the label distribution matrix. These structures are enforced via trace norm regularization to capture global or local correlations [Wang et al., 2021; Wang and Geng, 2023]. More recent studies relax the low-rank assumption by modeling label distributions on structured manifolds or graphs and leveraging both global and local correlations to improve label distribution learning under incomplete supervision [Jin et al., 2024; Wang et al., 2024]. In addition, neighborhood-based inference [Zhang et al., 2025], matrix completion methods [Xu et al., 2025; Li et al., 2025a; Chang et al., 2025], and robustness-oriented learning strategies [Li and Chen, 2024] have been explored to enhance performance under incomplete supervision. While existing methods effectively mitigate annotation incompleteness through correlation mining or structural regularization, they typically treat missing entries implicitly via masking, normalization, or reconstruction objectives. This conflates truly low description degrees with missing data at the label distribution level, biasing the training signal toward labels with higher description degrees and further suppressing labels with low description degrees. Moreover, most existing approaches do not explicitly consider label missingness under imbalanced distributions, which is a more prevalent and complex problem in real-world scenarios.

3 Method

3.1 ProblemDefinition

Let X = [x1, . . . , xN]⊤ ∈ RN×d denotes the feature matrix of N instances, where xi ∈ Rd is the feature vector of the i-th instance. Let D = [d1, . . . , dN]⊤ ∈ RN×m denotes the complete label distribution matrix over m labels, where di = [dy1 xi , . . . , dym xi ]⊤, dyj xi ∈ [0, 1], and ∑mj=1 dyj xi = 1.

In the InLDL setting, we introduce a binary mask Ω ∈ {0, 1}N×m to indicate label availability. The observed label distribution matrix Y ∈ RN×m is defined as:

where Ωi,j = 1 if dyj xi is observed, and Ωi,j = 0 otherwise.

To characterize the label distribution imbalance, we define an imbalance factor γ based on the complete label distribution matrix D as:

where a larger γ indicates more severe dataset-level imbalance.

The objective is to learn a predictor pθ : Rd → ∆m that infers the complete label distribution for each instance under incomplete and imbalanced supervision.

3.2 Distribution-Aware Label Completion

Inspired by [Gao et al., 2025; Li and Chen, 2024], given the partially observed label distribution matrix Y, we infer a complete label distribution for each instance by explicitly modeling instance-specific imputation dynamics.

For each instance xi, let ¯di = Ωi,: ∈ {0, 1}m denote the binary observation mask, where ¯ di,j = 1 if the j-th label is observed and 0 otherwise. The corresponding missing ratio is computed as:

To infer missing description degrees, we employ a completion network H(·) implemented as a multilayer perceptron. Specifically, H takes the concatenation of the instance feature vector and the observation mask as input and outputs four mdimensional completion parameters:

where [xi; ¯di] denotes concatenation. Here, µi ∈ Rm outputs the logits transformed into the prior label distribution, si ∈ Rm encodes reliability information used to derive the variance vector σ2i , ∆i ∈ Rm provides a residual correction term, and wi ∈ Rm yields elementwise fusion weights for integrating observed and imputed signals.

The prior label distribution is obtained as dprior i = g(µi), where g(·) denotes the Softmax function. The corresponding variance vector σ2i is recovered from si via an exponential mapping:

To ensure robust imputation, we adaptively control the correction strength using a reliability-aware gating mechanism:

where a, b1, b2 are learnable scalars and sigmoid(·) denotes the sigmoid function. The imputed label distribution signal is then formulated as:

The refined label distribution is obtained by integrating the observed entries with the imputed signals via an adaptive weighting scheme:

Finally, we normalize the refined signal to obtain a valid label distribution:

where ϵ > 0 is a small constant for numerical stability. The resulting ˆd comp i serves as the completed label distribution for subsequent bias-corrective representation learning.

3.3 Bias-Corrective Representation Learning

Firstly, to enable bias-corrective representation learning, we decompose the completed label distribution ˆd comp i ∈ Rm into a low-rank core and a bias-indicative residual:

where B ∈ Rm×r is a learnable basis matrix with fixed rank r that is jointly optimized with θ, and αi ∈ Rr denotes the corresponding latent coefficients. The core component d(a) i captures shared description degree patterns encoded by the low-rank basis, while the residual component d(b) i represents instance-specific residual variations that often manifest as distribution bias, enabling bias-corrective representation learning under incomplete and imbalanced label distributions [Kou et al., 2024; Ma et al., 2024; Li et al., 2025b].

To ensure scale consistency, both components are normalized as:

where N(z) = max(z, ϵ)/(1⊤ max(z, ϵ)) denotes the normalization operator.

The instance feature and the decoupled label distribution components are then projected into a shared latent space:

where ϕ : Rd → RH and ψ : Rm → RH denote the feature encoder and the label distribution encoder, respectively.

Secondly, we employ a gating mechanism to generate component-specific modulation parameters:

where Fk(·) is an MLP that outputs dimension-wise scaling and shifting coefficients.

Thirdly, we learn a dimension-wise responsibility mask to determine the active dimensions for modulation:

where M(·) denotes the mask network, and bias-corrective modulation is performed via an asymmetric update rule:

The final modulated feature is obtained as:

where R(·) denotes a feature normalization operator.

Finally, the predicted label distribution is generated as:

where U(·) denotes a fusion network and f(·) is the prediction head.

For analysis, we summarize the instance-wise modulation preference as:

During training, the predictor is supervised only by the observed entries of the label distribution. We minimize the masked KL divergence:

This objective enforces consistency on observed entries. Under random missingness, the masked loss provides a consistent estimate of the full KL risk used in the theoretical analysis. The algorithm pseudocode is provided in the Appendix.

3.4 Theoretical Analysis

Motivated by recent Lipschitz-based generalization analyses for deep predictors [Zhang and Zhang, 2024], we analyze the generalization properties of LDIBR under incomplete and imbalanced supervision. Let P denotes the distribution over pairs (x, d), where d ∈ ∆m is the complete label distribution.

Definition 1. The expected risk is

and the empirical risk over N i.i.d. samples {(xi, di)}Ni=1 is

The KL divergence is

where d, p ∈ ∆m. Since complete label distributions are not observable, LDIBR minimizes a masked KL loss on observed entries. Under random label missingness, this masked loss consistently estimates the full KL risk.

Definition 2. Let ˆd comp (x) and h∗ (x) denote the completed label distribution and the bias-corrected representation. Define

where h† (x, d) is an oracle representation.

Assumption 1. pθ(x) = softmax(f(h∗ (x))) and

where ε0 > 0. Assume f is Lf -Lipschitz under ℓ2, and softmax is Ls-Lipschitz from ℓ2 to ℓ1 on ∆(ε0) m (cf. [Nguyen et al., 2024]).

Theorem 1. For any δ ∈ (0, 1), with probability at least 1-δ,

where γ is the imbalance factor in Eq. (2), r is the rank of B ∈ Rm×r , and C1, C2 are constants.

Proof. Let p = pθ(x) and p′ = ˆd comp (x), and define ∆(x) = ℓKL(d, p) - ℓKL(d, p′ ). By Lemma 1, since p, p′ ∈ ∆(ε0) m ,

Moreover, the triangle inequality gives ∥p - p′ ∥1 ≤ ∥p - d∥1 + ∥p′ - d∥1, so taking expectation yields the completion term 1 ε0 Ec. Lemma 2 together with Assumption 1 further bounds the prediction deviation induced by replacing the oracle representation h† (x, d) with h∗ (x), which leads to the term LsLf ε0 Er. The remaining generalization gap is controlled by standard concentration and complexity arguments, resulting in the C1γ/√N and C2r log(2/δ)/N terms. Details are in the Appendix.

Lemma 1. For any d ∈ ∆m and p, p′ ∈ ∆(ε0) m ,

Proof. ℓKL(d, p)-ℓKL(d, p′ ) = ∑j dj(log p′j -log pj), and | log p′j - log pj | ≤ ε-1 0 |p′j -pj| since pj, p′j ≥ ε0. Summing and using ∑j dj = 1 completes the proof.

Lemma 2. Let h1, h2 ∈ RH and pθ(h) = softmax(f(h)). Then

Proof. By Lipschitz composition, ∥pθ(h1) - pθ(h2)∥1 ≤ Ls∥f (h1) - f (h2)∥2 ≤ LsLf ∥h1 - h2∥2.

4 Experiments

4.1 Experimental Settings

Datasets and Incomplete Settings. To evaluate the effectiveness of our method, we conduct experiments on six benchmark datasets: SCUT-FBP [Xie et al., 2015], Flickr LDL [Yang et al., 2017], Movie [Geng and Hou, 2015], Emotion6 [Peng et al., 2015], Natural Scene [Geng, 2016], and RAF-ML [Li and Deng, 2019]. These datasets exhibit diverse characteristics and varying degrees of imbalanced label distributions. Their detailed statistics and imbalance factors γ (defined in Eq. (2)) are summarized in Table 1. For the incomplete setting, we adopt 10-fold cross-validation, randomly masking 50% of the training description degrees uniformly across instances.

Compared Methods. We evaluate LDIBR against seven state-of-the-art methods covering diverse challenges in label distribution learning. Specifically, WInLDL [Li and Chen, 2024] and I 2LDL [Kou et al., 2024] mitigate label incompleteness via weighted optimization and consistency constraints. DILDL [Gao et al., 2025] counters distribution imbalance through decoupling, whereas GLDL [Jin et al., 2024], LEIC [Lu et al., 2023], and UALDL [Le et al., 2023] emphasize structural correlations and reliability modeling. Furthermore, LCD [Xu et al., 2025] unifies label completion and learning via implicit correlation decomposition. All methods follow their original configurations.

EvaluationMetrics. We comprehensively evaluate the predictive performance using six standard metrics: Chebyshev, Clark, Canberra, and KL divergence as distance-based measures (↓), alongside Cosine and Intersection as similaritybased measures (↑). All results are reported as mean ± standard deviation over 10-fold cross-validation. Statistical significance is assessed via paired t-tests at the 0.05 level, where a solid circle (·) denotes that LDIBR significantly outperforms the compared baseline.

4.2 Results

As shown in Tables 2 and 3, we report quantitative comparisons under the 50% incomplete setting, where lower values indicate better performance for distance-based metrics and higher values indicate better performance for similaritybased metrics. Across distance-based metrics, LDIBR consistently outperforms competing methods on most datasets. For instance, on Natural Scene, LDIBR reduces the Chebyshev distance to 0.4857, corresponding to relative reductions of 17.9% over I 2LDL and 29.9% over DILDL, and further lowers the Clark distance to 2.6584. On SCUT-FBP, LDIBR achieves the best performance across all distance-based measures, reducing the KL divergence from 0.8815 for I 2LDL to 0.6255, a 29.0% relative reduction. Similar gains are observed on Movie and Emotion6, with improvements ranging from 15% to 30% over incomplete and imbalance-aware baselines.

Regarding similarity-based metrics, LDIBR attains the highest Cosine similarity on five datasets and improves the Intersection score on the highly imbalanced RAF-ML to 0.5212, yielding a 14.8% relative gain. Compared with correlationbased completion methods and imbalance-aware baselines, LDIBR consistently achieves higher similarity scores, suggesting that addressing missingness and imbalance in a unified manner is more effective. Notably, larger standard deviations are observed on Flickr-LDL and RAF-ML, with values of 0.125 and 0.197 for the Chebyshev distance, while all other datasets exhibit standard deviations below 0.03. This variability is likely due to severe label imbalance, which increases sensitivity to split-dependent missing patterns and leads to heterogeneous completion difficulty across folds. Nevertheless, by explicitly recovering unobserved description degrees, LDIBR mitigates such fluctuations and maintains strong mean performance even under pronounced distribution bias.

4.3 Ablation Study

We examine the contribution of each component in LDIBR by ablating the Distribution-Aware Label Completion module and the Bias-Corrective Representation Learning module. Tables 4 and 5 report results on Natural Scene and SCUTFBP. The full model achieves the best performance across all metrics, and removing either module consistently degrades performance. Removing Distribution-Aware Label Completion causes the largest drop. On Natural Scene, KL increases to 1.421 and Cosine similarity decreases to 0.599, with Che., Cla., Can., and Int. also worsening. A similar trend is observed on SCUT-FBP, where KL increases to 1.034 and Cosine similarity decreases to 0.512. Removing Bias-Corrective Representation Learning leads to milder yet consistent degradation. On Natural Scene, KL increases to 1.012 and Cosine similarity decreases to 0.721, and on SCUT-FBP, KL increases to 0.781 with Cosine similarity decreasing to 0.603. When both modules are removed, performance deteriorates most severely on both datasets, confirming that the two components are complementary.

4.4 Further Analyses

We further investigate model robustness under increasing label incompleteness by gradually raising the missing ratio from 0.4 to 0.8 and evaluating performance using KL divergence. Fig. 3 shows that KL divergence consistently increases for all methods as the label missing ratio grows, indicating that higher levels of label missingness make accurate label distribution recovery more challenging. On the Movie dataset, LDIBR consistently achieves lower KL divergence across all missing ratios and exhibits a smooth degradation trend as label missingness increases, demonstrating stable behavior under progressively reduced supervision. On RAFML, although LDIBR does not achieve the lowest KL divergence at lower missing ratios, its performance degrades more slowly as the missing ratio rises and it exhibits the smallest overall increase in KL divergence under severe label missingness. Compared with baseline methods, LDIBR maintains more stable relative performance as the label missing ratio increases, particularly in high missing-ratio regimes. Experimental results demonstrate that LDIBR maintains robust performance even under increasing levels of label incompleteness, particularly in challenging scenarios where concurrent label missingness and distribution skew jointly compromise model training.

5 Conclusion

In this paper, we propose LDIBR, a unified framework for more complex label distribution learning with missingness under imbalanced label distributions. We highlight the limitations of conventional methods that treat missing description degrees as low values, thereby introducing systematic bias in label distribution recovery and representation learning. LDIBR explicitly models missing entries as unobserved description degrees, applies distribution-aware label completion, and incorporates bias-corrective representation learning to mitigate imbalance-induced bias. Additionally, we provide a theoretical generalization analysis that characterizes how completion error, representation error, and description degree imbalance jointly affect the risk. Extensive experiments on benchmark datasets demonstrate that LDIBR achieves more accurate and stable predictions, particularly under high missing ratios and severe label imbalance. Future work will focus on enhancing model robustness and extending LDIBR to hierarchical and continuous label scenarios.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under No. 62406155, the project No. ZR2024QF115 supported by Shandong Provincial Natural Science Foundation, the Innovation Capability Enhancement Project for Technology-based Small and Medium-sized Enterprises of Shandong Province under Grant No. 2025TSGCCZZB0077, the National Natural Science Foundation of China under No. 62202235 and No. 52303025, the project No. ZR2025MS750 supported by Shandong Provincial Natural Science Foundation, the Jinan Science and Technology Plan (Post-subsidy) Project under No.202430039.