Geodesic Expert Routing for Unbiased Knowledge Distillation in Recommendation

1 Introduction

Recommender systems have become indispensable infrastructure for modern online platforms, serving as the primary mechanism through which users discover products, content, and services aligned with their preferences [Deldjoo et al., 2024]. By filtering the overwhelming volume of available options into personalized suggestions, these systems significantly enhance user experience across diverse domains such as e-commerce, streaming media, and social networks [Park et al., 2026; Gao et al., 2023]. As data volume and item diversity continue to scale exponentially, system designers face an increasingly critical trade-off between recommendation accuracy and computational efficiency [Zhang et al., 2019].

∗Corresponding author

Knowledge Distillation (KD) has emerged as a widely adopted paradigm to address this efficiency challenge [Kang et al., 2020]. By transferring knowledge from a computationally expensive teacher model to a lightweight student model, KD enables the deployment of compact recommenders that retain much of the predictive capability of their larger counterparts while satisfying stringent latency requirements [Lee et al., 2019; Kweon et al., 2021]. This approach has demonstrated remarkable success across various recommendation architectures, from collaborative filtering to sequential models [Sarwar et al., 2001; Kang et al., 2021]. However, we identify a critical yet underexplored issue: knowledge distillation systematically amplifies popularity bias in the student model. Popularity bias, where models disproportionately favor frequently-interacted items at the expense of longtail content, is already pervasive in recommender systems trained on implicit feedback [Oestreicher-Singer and Sundararajan, 2012; Abdollahpouri et al., 2019]. Our empirical investigation reveals that this bias is not merely preserved but substantially magnified during the distillation process. We argue that the root cause lies in the reliance on a single global teacher trained on complete interaction data that is inherently dominated by popularity patterns. Figure 1 shows KD improvements are driven by popular items at the expense of unpopular ones. By mimicking such a teacher, the student internalizes and amplifies these popularity-driven shortcuts, resulting in reduced exposure diversity and exacerbated unfairness toward niche items [Lee et al., 2019; Tang and Wang, 2018].

To address this limitation, we propose GUIDE (Geodesicaware Unbiased Instructive Distillation with Experts), a novel framework that augments conventional single-teacher distillation with collaborative guidance from domain-specific debiasing experts. Our key insight is that while the global teacher provides comprehensive knowledge essential for accuracy, targeted experts trained on strategically partitioned data subsets can serve as corrective signals that counteract popularity bias. Specifically, we partition the original interaction data into multiple subsets based on item popularity strata and train specialized expert models in parallel. During distillation, the student model learns not only from the global teacher but also from these experts, enabling balanced knowledge acquisition that promotes both accuracy and fairness. However, transitioning from single-teacher distillation to this collaborative teacher-expert paradigm introduces two fundamental challenges:

• Expert Routing: How to select the most appropriate expert for each prediction context? A natural approach measures expert-student similarity via cosine or inner product in Euclidean space [Koren et al., 2009]. However, embedding norms encode popularity-correlated intensity information, as popular items naturally exhibit larger magnitudes due to more frequent gradient updates [Zhu et al., 2021; Wang et al., 2017]. This causes the student to be attracted to high-norm experts rather than semantically aligned ones. While post-hoc ℓ2 normalization offers an implicit remedy, it remains geometrically imprecise and optimization-unfriendly.
• Context Fusion: How to dynamically balance knowledge from the teacher and experts based on varying useritem characteristics? Static fusion weights are fundamentally inadequate, since susceptibility to popularity bias varies substantially across user-item contexts depending on user engagement diversity, item popularity characteristics, and interaction patterns [Han et al., 2024; Abdollahpouri et al., 2021]. Determining appropriate debiasing intensity requires real-time contextual reasoning rather than a fixed arbitration strategy.

To tackle Expert Routing, we propose Spherical Expert Alignment, which conducts expert-student matching directly on the spherical manifold S d-1 , ensuring that all comparisons reflect pure directional similarity free from magnitude contamination. Crucially, we optimize geodesic distance rather than cosine angle: cosine-based gradients scale with sin(θ) and vanish as θ → 0, whereas geodesic gradients remain linear in θ, enabling stable optimization even when student and expert representations are closely aligned.

To address Context Fusion, we design a Meta-Debiasing Gate, a lightweight gating network that ingests contextual features such as user history diversity, item popularity percentile, and interaction density. The gate outputs a dynamic fusion weight α ∈ [0, 1] that arbitrates the balance between teacher and expert influence. It is trained end-to-end via amortized meta-learning to explicitly optimize for both accuracy and debiasing effectiveness.

Our main contributions are summarized as follows:

• We identify the phenomenon of popularity bias amplification in knowledge distillation and propose GUIDE, a collaborative distillation framework that integrates domainspecific debiasing experts alongside the global teacher to enable accuracy-fairness balanced knowledge transfer.
• We introduce Spherical Expert Alignment for geometryaware expert routing and Meta-Debiasing Gate for contextaware dynamic fusion, addressing the challenges of expert selection and adaptive knowledge arbitration respectively.
• Extensive experiments on real-world datasets demonstrate that GUIDE significantly mitigates popularity bias while maintaining competitive accuracy, achieving state-of-theart trade-offs among efficiency, accuracy, and fairness.

2 RelatedWork

2.1 Knowledge Distillation for Recommendation

Knowledge Distillation (KD) enhances recommender systems by transferring knowledge from intermediate layers [Romero et al., 2015], privileged features [Xu et al., 2020], or structured graphs [Zhang et al., 2020]. Beyond efficiency [Zhu et al., 2020] and collaborative learning [Gou et al., 2024], KD is widely applied to mitigate biases. For instance, [Liu et al., 2020] and UKDRec [Yang et al., 2025] leverage counterfactual data and multi-teacher frameworks for debiasing. UNKD [Chen et al., 2023a] and [Ding et al., 2022] address popularity bias and model fusion, respectively. Research also optimizes distillation losses, ranging from ranking-based top-N selection [Tang and Wang, 2018] and instance construction [Lee et al., 2019] to list-level losses [Kang et al., 2020], extending to heterogeneous graph integration [Tao et al., 2022; Wang et al., 2021].

2.2 Bias in Recommendation

Recommendation bias arises from user behavior and feedback loops [Chen et al., 2023b]. Mitigation strategies typically employ IPS [Joachims et al., 2018; Li et al., 2023; Schnabel et al., 2016], Doubly Robust methods [Song et al., 2023], or causal graphs [Liu et al., 2021; Liu et al., 2023; He et al., 2022] on biased data [Ning et al., 2024; Zhang and Shen, 2023], or leverage unbiased data via meta-learning [Chen et al., 2021] and distillation [Liu et al., 2020]. Beyond data, architectures [Lin et al., 2019] and optimizers [Tang et al., 2020] contribute to bias. Notably, KD itself can aggravate popularity bias [Chen et al., 2023a]: a teacher model trained on skewed data transfers inherent biases to the student, undermining debiasing efforts [Yang et al., 2025].

3 Preliminaries

3.1 ProblemFormulation

Let U = {u1, u2, . . . , uM} denote the set of M users and I = {i1, i2, . . . , iN} denote the set of N items. The useritem interaction matrix R ∈ {0, 1}M×N encodes implicit feedback, where rui = 1 indicates that user u has interacted with item i, and rui = 0 otherwise. For each user u, we denote the set of interacted items as I + u = {i ∈ I | rui = 1}. The primary objective of a recommender system is to learn a scoring function f : U×I → R that predicts user preferences over unobserved items.

3.2 Knowledge Distillation in Recommendation

Knowledge Distillation (KD) aims to transfer knowledge from a high-capacity teacher model T to a lightweight student model S. Given a user-item pair (u, i), the teacher produces a soft prediction ˆ yT ui that encodes rich relational knowledge beyond binary labels. The student is trained to mimic these soft targets while maintaining computational efficiency. The distillation objective typically minimizes the divergence between teacher and student outputs:

where ℓ(·, ·) denotes a divergence measure such as Mean Squared Error or KL divergence, and D is the training set.

3.3 Spherical Manifold

The (d-1)-dimensional unit spherical manifold S d-1 is defined as the set of all d-dimensional vectors with unit ℓ2 norm:

The spherical manifold provides an embedding space where vector magnitudes are normalized, enabling the model to focus purely on directional relationships. For any vector x ∈ Rd \ {0}, the projection onto S d-1 is given by:

The intrinsic distance on the spherical manifold is the geodesic distance, representing the length of the shortest arc connecting two points. For x, y ∈ S d-1 , the distance is:

4 Methodology

Figure 2 illustrates the overall framework of GUIDE. Given user-item interaction data, GUIDE first partitions the item set into K popularity-based subsets to construct domain-specific expert teachers. To address the Expert Routing challenge, we project both users and expert centroids onto a spherical manifold, where expert relevance is quantified via geodesic distance. To address the Context Fusion challenge, a MetaDebiasing Gate dynamically arbitrates between expert and global teacher knowledge based on real-time contextual features. The final distillation signal is derived as a contextaware ensemble tailored to each user-item interaction. We detail each component in the following subsections.

4.1 Domain-Specific Expert Construction

Relying on a single global teacher causes popularity bias amplification in knowledge distillation. A global teacher trained on the complete interaction data inevitably inherits the longtail distribution, leading to overfitting on popular items while under-representing niche content. To mitigate this limitation, we construct multiple domain-specific experts, each specializing in a distinct popularity stratum.

Popularity-Based Data Partitioning

We quantify the popularity of each item i by its interaction frequency di = |{u ∈ U | rui = 1}|. Based on these frequency values, we partition the item set I into K disjoint subsets {I (1) , I (2) , . . . , I (K) }, where each subset corresponds to a distinct popularity tier. Specifically, we sort all items by di in descending order and divide them into K groups of approximately equal size, yielding a spectrum from head (highpopularity) to tail (low-popularity) items. Each item subset I (k) induces a corresponding interaction subset:

This partitioning creates isolated training environments that shield tail item learning from head item interference.

Expert Teacher Training

For each partition D(k) , we train a dedicated expert teacher T (k) exclusively on interactions within I (k) . This constraint ensures that each expert captures the intrinsic collaborative patterns specific to its popularity domain without being dominated by global popularity shortcuts. Formally, the expert T (k) is optimized via:

where j denotes a negative sample drawn from I (k) . In parallel, we train a global teacher T (g) on the complete interaction data D to preserve comprehensive collaborative signals.

Remark. The expert construction procedure is modelagnostic: any recommendation architecture (e.g., matrix factorization, neural collaborative filtering, or sequential models) can serve as the backbone for both expert and student models. This flexibility ensures broad applicability across diverse recommendation scenarios.

4.2 Spherical Expert Alignment

With K expert teachers constructed, the next challenge lies in dynamically routing each user to the most semantically aligned expert. Conventional approaches measure expertstudent similarity via cosine or inner product in Euclidean space. However, as discussed in Section 1, embedding magnitudes encode popularity-correlated intensity information that confounds directional similarity. We address this challenge through Spherical Expert Alignment, which conducts matching directly on the spherical manifold S d-1 .

Expert Centroid Representation

To enable efficient expert routing, we represent each expert T (k) by a semantic centroid that summarizes its domain knowledge. Let E(k) = {ei | i ∈ I (k) } denote the set of item embeddings learned by expert T (k) . We compute the expert centroid ck by aggregating these embeddings and projecting onto the spherical manifold:

where ck ∈ S d-1 serves as the representative prototype for the k-th expert domain.

User Projection and Geodesic Matching

For any user u with embedding eu learned by the student model, we project it onto the same spherical manifold:

The affinity between user u and expert k is then quantified by the geodesic distance on S d-1 :

This geodesic formulation offers two key advantages over cosine similarity. First, constraining representations to the unit sphere strictly decouples embedding magnitude-often correlated with item popularity-from semantic direction, ensuring pure content-based routing. Second, unlike cosine gradients which scale with sin(θ) and vanish as θ → 0, the geodesic gradient ∇vLgeo ∝ ∂θ2 ∂θ ∂θ ∂ cos θ ∇v cos θ introduces a 1/ sin(θ) term that perfectly cancels the projection-induced sin(θ) decay. This yields stable gradients linear in θ (i.e., ∇L ∝ 2θ) even for closely aligned representations.

Expert Selection via Soft Routing

Based on geodesic distances, we compute a probability distribution over experts for each user u:

where τ > 0 is a temperature parameter controlling the sharpness of the distribution. To minimize noise from less relevant experts, we adopt a winner-takes-all strategy that selects the expert with the highest probability:

The selected expert T (k∗ ) provides the specialized soft label ˆ yTk∗ ui for the current user-item pair.

4.3 Meta-Debiasing Gate

While Spherical Expert Alignment identifies the optimal domain-specific expert for each user, relying solely on expert knowledge may be suboptimal. Expert teachers, trained on isolated popularity strata, may lack the broader collaborative signals captured by the global teacher. Conversely, the global teacher provides robust general knowledge but inherits popularity bias. To dynamically balance these complementary knowledge sources, we design a Meta-Debiasing Gate that adaptively arbitrates their influence based on real-time contextual features.

Context Feature Extraction

The optimal balance between expert and global teacher knowledge depends on the specific characteristics of each user-item interaction. We extract a context feature vector xui that captures two key factors:

Item Popularity. We quantify the global exposure of candidate item i using its log-normalized interaction frequency:

This feature informs the gate about the item's position in the popularity spectrum.

User Diversity. We measure the diversity of user u's historical interactions by computing the variance of popularity values across interacted items:

This feature indicates the user's tendency to explore across different popularity tiers. The context vector concatenates these features with learned user and item embeddings, where ⊕ denotes concatenation:

Adaptive Gating Mechanism

The Meta-Debiasing Gate M uses a lightweight multi-layer perceptron (MLP) with sigmoid activation, mapping the context vector to a scalar fusion coefficient αui ∈ (0, 1):

where W1,W2, b1, b2 are learnable parameters, and σ(·) denotes the sigmoid function.

The coefficient αui quantifies the necessity of debiasing for the current context: higher values indicate that the model should prioritize expert knowledge (e.g., for long-tail items or users with diverse histories), while lower values suggest relying on the global teacher for stability.

Dynamic Knowledge Fusion

The final distillation target is synthesized as a convex combination of soft labels from the selected expert T (k∗ ) and the global teacher T (g) :

This dynamic fusion ensures that the distillation signal is adaptively tailored to each user-item interaction, effectively resolving the trade-off between accuracy and fairness.

4.4 Optimization

We formulate a multi-objective loss function that jointly optimizes recommendation accuracy, knowledge transfer, and gate calibration:

where λKD, λAlign, and λReg are hyperparameters balancing each component.

Recommendation Loss. We employ the BPR loss to ensure the student captures fundamental collaborative signals:

where (u, i, j) denotes a triplet with positive item i ∈ I + u and negative item j /∈ I + u .

Ranking-Aware Distillation Loss. Standard pointwise distillation may fail to preserve the nuanced ranking relationships essential for recommendation. We propose a rankingaware objective that aligns the preference margins between student and teacher:

where ∆pSuij = σ(ˆ yS ui ) - σ(ˆ yS uj ) and ∆pTuij = σ(ˆ yT final (u, i)) - σ(ˆ yT final (u, j)) represent the preference margins from the student and fused teacher, respectively.

Gate Alignment Loss. To prevent the gate coefficient αui from collapsing to trivial solutions, we introduce an explicit supervisory signal. Let ti = I(i ∈ Itail) be an indicator for tail items. We employ binary cross-entropy to guide the gate:

This objective encourages the gate to prioritize expert knowledge (αui → 1) for tail items while retaining global knowledge (αui → 0) for popular items.

4.5 Complexity Analysis

Training Complexity. Given batch size B, embedding dimension d, and K experts, the primary computational costs include: (1) student forward pass: O(Bd); (2) geodesic distance computation: O(BKd); and (3) Meta-Debiasing Gate: O(Bd2 ). Since K is typically small (e.g., K ≤ 5), and the winner-takes-all strategy activates only the selected expert and global teacher, the overall training complexity remains linear in batch size.

Inference Complexity. During inference, all auxiliary components (expert teachers, spherical projection, MetaDebiasing Gate) are discarded. Only the lightweight student model is deployed, yielding inference complexity identical to the base architecture with no additional latency.

5 Experiments

We conduct comprehensive experiments to evaluate GUIDE, aiming to answer the following research questions:

• RQ1: How does GUIDE perform compared to state-of-theart knowledge distillation methods for recommendation?
• RQ2: What are the individual contributions of each proposed component?
• RQ3: How sensitive is GUIDE to key hyperparameters?
• RQ4: How effective is GUIDE in handling popularity bias and enhancing item fairness?

5.1 Experimental Setup

Datasets. We evaluate our model on three public benchmarks: CiteULike (scholarly articles), Amazon-Movie, and Amazon-Game (e-commerce). Following standard preprocessing, we retain users with at least 20 interactions and split the data into training, validation, and test sets using an 8:1:1 ratio. Detailed statistics are reported in Table 1.

Baselines. We compare GUIDE against five state-of-theart distillation methods: (1) Ranking-based: RD [Tang and Wang, 2018] and CD [Lee et al., 2019] transfer ranking knowledge via position-aware weighting and rank-aware sampling; (2) Structure-aware: DE-RRD [Kang et al., 2020] and HTD [Kang et al., 2021] distill latent knowledge and topological structures; and (3) Bias-aware: UNKD [Chen et al., 2023a] mitigates popularity bias via stratified distillation. Evaluation Metrics. We adopt two widely-used top-K metrics: Recall@K (R@K), which measures the proportion of relevant items retrieved, and NDCG@K (N@K), which accounts for ranking position via discounted cumulative gain. We report results for K ∈ {10, 20}.

Implementation Details. All hyperparameters are tuned based on NDCG@20 on the validation set. The teacher uses 3-layer LightGCN, for students, we evaluate 2-layer GCN and BPRMF. The learning rate is searched over {0.0002, 0.001, 0.005, 0.01}. The distillation coefficient λKD is selected from {0.01, 0.02, ..., 0.9, 0.99}, and the alignment coefficient λAlign from {0.0005, 0.001, ..., 0.9, 0.99}. The number of experts K is set to 5 by default. All experiments are repeated 5 times with different random seeds, and we report the mean and standard deviation.

5.2 Overall Performance (RQ1)

Table 2 presents the overall performance comparison. From these results, we draw the following observations:

GUIDE Consistently Outperforms All Baselines Across Datasets and Metrics. The improvements are particularly pronounced on the sparser Amazon-Game dataset, demonstrating GUIDE's robustness in challenging scenarios where popularity bias is most detrimental. Remarkably, GUIDE surpasses the teacher on several metrics, indicating that filtering popularity noise yields superior generalization.

Ranking-Based Methods Exhibit Limited Effectiveness. RD yields the poorest performance among all methods, as simple ranking alignment neglects the rich semantic relationships essential for recommendation. This highlights the need for superior knowledge transfer mechanisms.

Structure-Aware Methods Provide Substantial Improvements Over Ranking-Based Approaches. CD, HTD, and UNKD significantly outperform RD by incorporating structural knowledge. Notably, CD achieves competitive results by distilling collaborative similarity distributions, while UNKD leverages unobserved feedback for enhanced supervision.

The Advantages of GUIDE Stem From Its Two Core Innovations. Spherical Expert Alignment eliminates magnitudeinduced bias in expert routing, enabling semantically meaningful knowledge selection. The Meta-Debiasing Gate further adapts the fusion strategy to each context, ensuring debiasing preserves accuracy on well-represented items.

5.3 Ablation Study (RQ2)

To validate the contribution of each component, we compare GUIDE against four ablated variants on Amazon-Game:

• w/o Spherical: Replaces spherical manifold matching with Euclidean distance-based expert selection.
• w/o Meta-Gate: Substitutes the adaptive gate with a fixed fusion weight (α = 0.5).
• w/o Ranking KD: Replaces the ranking-aware distillation loss with standard pointwise MSE.
• w/o Alignment: Removes the gate alignment loss LAlign.

As shown in Table 3, the full GUIDE model consistently outperforms all variants, confirming the necessity of each component: (i) w/o Spherical shows that Euclidean embeddings are prone to magnitude-induced bias, emphasizing the need for directional alignment; (ii) w/o Meta-Gate confirms that static fusion is insufficient for handling user-item heterogeneity; (iii) w/o Ranking KD proves that pointwise distillation fails to capture nuanced preference margins; and (iv) w/o Alignment indicates that explicit gate supervision is vital to avoid trivial solutions.

5.4 Hyperparameter Sensitivity (RQ3)

We examine three key hyperparameters: (1) Number of experts K: As shown in Figure 3, performance peaks around K = 5; too few experts fail to capture popularity diversity, while excessive K introduces noise due to data sparsity. (2) Distillation coefficient λKD: Figure 4 (Left) shows optimal result at 0.8, balancing knowledge transfer and student adaptation. (3) Alignment coefficient λAlign: Figure 4 (Right) indicates that λAlign = 0.01 is optimal, while larger values may conflict with the primary ranking objective.

5.5 Debiasing Effectiveness (RQ4)

To evaluate the effectiveness of GUIDE in addressing popularity bias and enhancing fairness, we conduct three complementary analyses.

Addressing Popularity Bias and Fairness. We evaluate GUIDE on Amazon-Game (stratified into Head/Tail groups) and CiteULike (fairness metrics). Table 4 presents the results. GUIDE achieves substantial improvements on Tail items (+6.21%) while simultaneously improving Head performance (+2.75%). On CiteULike, GUIDE significantly lowers the Gini index and doubles Item Coverage. These results across datasets prove that GUIDE effectively handles popularity bias and improves fairness while maintaining superior overall accuracy.

Visualization of Adaptive Gating. To verify that the MetaDebiasing Gate learns meaningful context-aware strategies, we visualize the distribution of gating coefficients αui across item popularity on the Amazon-Game test set. As shown in Figure 5, a clear trend emerges: α ≈ 0 for popular (Head) items, indicating reliance on the global teacher, while α ≈ 1 for long-tail (Tail) items, shifting toward expert knowledge. This confirms that the gate learns a principled popularityaware arbitration strategy rather than random assignment, validating the effectiveness of the alignment loss LAlign.

Case Study: Cold-Start Item Recommendation. To evaluate performance under extreme sparsity, we analyze a coldstart interaction (User #8137, Item #10872) in Amazon- Game. While the LightGCN teacher fails to recommend the item (Rank > 200) due to absent collaborative signals, GUIDE achieves Rank 1 with a high gating coefficient (αui = 0.84). This confirms that GUIDE adaptively prioritizes specialized expert knowledge to compensate for data scarcity, effectively enabling zero-shot cold-start recommendations.

6 Conclusion

Existing KD approaches, using standard Euclidean teachers, often suffer from bias amplification. We propose GUIDE, projecting diverse expert knowledge onto a Spherical Manifold. This design eliminates magnitude-induced noise via distinct subspace experts, while our Meta-Debiasing Gate intelligently arbitrates between global and specialized supervision to adapt to dynamic user preferences. Experiments confirm that GUIDE consistently outperforms competing methods in both recommendation accuracy and bias mitigation.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No.62173199 and No.62506366).