Relation-Aware Graph Learning with Mixture-of-Experts Prediction for Cognitive Diagnosis

1 Introduction

Cognitive diagnosis in intelligent education assesses students' mastery of knowledge concepts from exercise response logs [Anderson et al., 2014; Burns et al., 2014]. As shown in Fig. 1, a cognitive diagnosis model estimates each student's concept-level mastery from students' responses and exerciseconcept associations, supporting applications such as learning path planning [Liu et al., 2019] and resource recommendation [Kuh et al., 2011]. As online education platforms and large-scale response data rapidly grow, accurately characterizing students' cognitive states from complex response patterns has become a fundamental challenge.

Early cognitive diagnosis studies are largely grounded in psychometric theories, including DINA [De La Torre, 2009], MIRT [Reckase, 2009], and IRT [Embretson and Reise,

∗Corresponding author.

2013]. These models are concise and interpretable, yet their limited expressiveness makes it difficult to accommodate diverse response patterns. As deep learning developed, some methods leverage neural networks to model student-exercise interactions [Wang et al., 2020]. Recently, graph-based cognitive diagnosis methods have gained attention by leveraging graph structure to represent students, exercises, concepts, and their relations [Gao et al., 2021; Wang et al., 2023b]. Despite this progress, these methods still face challenges in both relation modeling and prediction mechanisms.

First, holistic heterogeneous relations are still underexplored. Existing graph-based methods usually model only partial relations, such as student-exercise interactions or exercise-concept links, making it hard to distinguish different relation effects. Second, a single predictor has limited capacity to characterize students' response performance. Correctness depends on student mastery and exercise difficulty, which vary widely in real-world educational data. Most methods model these factors with one predictor for all students and exercises, making it difficult to capture such variations and limiting prediction accuracy.

To address these challenges, we propose RMCD (RelationAware Graph Learning with Mixture-of-Experts Prediction), which integrates relation-aware graph learning with a Mixture-of-Experts (MoE) prediction head for cognitive diagnosis. First, RMCD constructs a heterogeneous relational graph that jointly represents students, exercises, and concepts with student-exercise, exercise-concept, and student-concept relations. On this graph, a relation-aware graph encoder jointly learns node and edge representations. In particular, it learns dedicated representations for student-concept and exercise-concept edges and maps them to relation-strength vectors, which distinguish relation effects and are fed back to refine node representations. Second, RMCD introduces an MoE-based prediction head to overcome the limitation of a single predictor. Conditioned on the representations of students, exercises, and concepts, it adaptively combines multiple expert predictors to estimate students' response performance. The experts capture diverse concept-level masterydifficulty discrepancies, yielding a flexible prediction mechanism for varied student mastery and exercise difficulty.

Our main contributions include: 1) We build a relational graph over students, exercises, and concepts with studentexercise, exercise-concept, and student-concept relations. 2) We propose a relation-aware graph encoder that jointly learns node and edge representations and derives relation-strength vectors to differentiate relation effects and refine nodes. 3) We design an MoE-based prediction head that conditions on student, exercise, and concept representations and adaptively combines experts to model diverse mastery-difficulty discrepancies. To the best of our knowledge, this is the first work that introduces MoE into cognitive diagnosis.

2 RelatedWork

2.1 Cognitive Diagnosis

Cognitive diagnosis aims to infer students' mastery of knowledge concepts from response records. Early psychometric models include IRT [Embretson and Reise, 2013], MIRT [Reckase, 2009], and DINA [De La Torre, 2009], where DINA models student-concept mastery with guessing and slipping for interpretability. NCD [Wang et al., 2020] uses neural networks to capture nonlinear student-exercise interactions. Recent graph-based methods jointly model students, exercises, and concepts to exploit higher-order relations, e.g., RCD [Gao et al., 2021] and SCD [Wang et al., 2023b], with ISGCD [Shao et al., 2025] furtherconsidering edge heterogeneity and uncertainty. Beyond structural relations, ACD [Wang et al., 2024] incorporates students' affective states. Inspired by graph-based cognitive diagnosis, RMCD constructs the relational graph with student-exercise, exercise-concept, and student-concept relations, and learns node and edge representations simultaneously. Relationstrength vectors derived from student-concept and exerciseconcept edges further differentiate relation effects.

2.2 Mixture of Experts

MoE is a conditional computation framework that uses a gating function to route inputs to different experts [Jacobs et al., 1991; Masoudnia and Ebrahimpour, 2014]. MoE has dense or sparse variants. Dense MoE evaluates all experts and aggregates their outputs with mixture weights [Eigen et al., 2013; Masoudnia and Ebrahimpour, 2014], while sparse MoE activates only a subset of experts for controlled computation [Shazeer et al., 2017; Fedus et al., 2022]. Both variants have been explored in large models and graph learning [Lin et al., 2026; Wang et al., 2023a; Zeng et al., 2023; Zhou et al., 2025]. In RMCD, we adopt a dense MoE prediction head tailored to cognitive diagnosis. Experts operate on concept-level mastery-difficulty discrepancies, and a gating function conditioned on student, exercise, and concept representations adaptively combines expert outputs. This design provides a flexible prediction mechanism under substantial variations in mastery and difficulty.

2.3 Graph Neural Networks

GNNs learn graph representations by aggregating neighborhood information [Wu et al., 2020; Zhou et al., 2020]. GCN [Kipf, 2016] and GAT [Veliˇckovi´c et al., 2018] are representative homogeneous GNNs, while heterogeneous GNNs such as R-GCN [Schlichtkrull et al., 2018] and HAN [Wang et al., 2019] perform type-aware message passing over multityped nodes and relations. Recent studies further investigate meta-path modeling and analysis in heterogeneous graphs [Li et al., 2024; Li et al., 2025]. Different from these methods, RMCD parameterizes key educational relations with learnable edge representations and relation-strength vectors, which are fed back to refine node representations and capture relation effects critical for cognitive diagnosis.

3 ProblemFormulation

3.1 ProblemDefinition

Given a set of students, exercises, and knowledge concepts, we define the cognitive diagnosis problem as follows: Input: A student set {si}ns i=1 , an exercise set {qj}nq j=1 , and a knowledge concept set {ck}nc k=1 . In addition, a Q-matrix Q ∈ {0, 1}nq×nc that describes the exercise-concept association is provided, where each element Qjk = 1 indicates that exercise qj involves concept ck and Qjk = 0 otherwise. An observed historical response log is denoted by L = {(si, qj, rij)}, where rij denotes the response of student si on exercise qj , i.e., rij = 1 if correct and 0 otherwise. Output: A diagnosis model Dθ estimates the correctness probability of a student on an exercise, thereby implicitly characterizing students' concept mastery from L and Q. For any unobserved student-exercise pair (si, qj), the model outputs the correctness probability:

where θ denotes the model parameters.

3.2 Graph Formulation

We construct a heterogeneous relational graph G = (V, E) to model the heterogeneous relations among students, exercises, and concepts in cognitive diagnosis. Each entity (student, exercise, or concept) is associated with a node. The node set is defined as V = Vs ∪ Vq ∪ Vc, where Vs = {si}ns i=1 , Vq = {qj }nq j=1 , and Vc = {ck}nc k=1 denote the three types of nodes corresponding to students, exercises, and concepts, respectively. Each node is assigned a one-hot vector si ∈ {0, 1}ns , qj ∈ {0, 1}nq , or ck ∈ {0, 1}nc .

The edge set is defined as E = Esq ∪ Esc ∪ Eqc. Here, Esq ⊆ Vs × Vq represents student-exercise interactions recorded in L, where an edge (si, qj) ∈ Esq is included if (si, qj , rij ) ∈ L. Moreover, Eqc ⊆ Vq×Vc encodes exerciseconcept associations induced by Q: for each Qjk = 1, we include an edge (qj , ck) ∈ Eqc. To explicitly model studentconcept relations, we define Esc ⊆ Vs × Vc by linking a student to the concepts involved in the exercises the student has interacted with, i.e.,

4 Methodology

As illustrated in Fig. 2, RMCD contains three main stages:

• Feature Initialization. We project one-hot student, exercise, and concept nodes to dense representations, and initialize learnable features on student-concept and exercise-concept edges as inputs for graph encoding.
• Relation-Aware Graph Encoder. The encoder learns node and edge representations through three sub-layers: node updating, edge updating, and node refinement. It further map edge representations to relation-strength vectors, which support edge updating, node refinement, and concept-level mastery-difficulty modeling.
• MoE-Based Prediction Head. The prediction head uses multiple experts to model diverse mastery-difficulty discrepancies, with a gating function adaptively combines expert outputs for final prediction.

4.1 Feature Initialization

Given the one-hot node vectors defined in Sec. 3.2, we map students, exercises, and concepts to initial node representations vs i , vq j , vck ∈ Rdv via linear projections:

where Ws ∈ Rns×dv , Wq ∈ Rnq×dv and Wc ∈ Rnc×dv are learnable projection matrices, and dv denotes the hidden dimension of the node representations.

We further introduce the edge representations for studentconcept and exercise-concept relations. For each edge (si, ck) ∈ Esc and (qj, ck) ∈ Eqc, we associate a trainable vector esc ik ∈ Rde and eqc jk ∈ Rde , respectively, where de denotes the hidden dimension of edge representations. These representations are randomly initialized as:

where σ is a hyperparameter controlling the initialization standard deviation. Student-exercise edges in Esq only encode observed interactions from the response log L and do not carry additional learnable edge features in this stage.

4.2 Relation-Aware Graph Encoder

The relation-aware graph encoder learns representations on G by stacking nl GNN layers (Fig. 2(b)). In each layer, we update node and edge representations through three sub-layers: a node sub-layer, an edge sub-layer, and a node refinement sub-layer. In particular, node representations are jointly updated by the node sub-layer and the node refinement sublayer. For notation, we use unprimed and primed symbols to denote variables before and after an update (e.g., v, v′ ).

Node Sub-layer. For each node, we obtain its intermediate representation via an attention-based aggregation function with residual connections. Specifically, we adopt an off-theshelf heterogeneous GNN operator Φ(·) to mean-aggregate neighbor representations along the student-exercise edges Esq and the exercise-concept edges Eqc, thereby producing residual updates:

Then the intermediate node representations are computed with residual connections:

The updated node representations will be obtained in the subsequent node refinement sub-layer.

Edge Sub-layer. We update edge representations on Esc and Eqc using the current edge state and the intermediate representations of the two incident nodes (Fig. 2(d)). For each student-concept edge (si, ck) ∈ Esc, we first compute a relation-strength vectorpsc ik = Ψ(esc ik ) , (7)

where Ψ(·) denotes an element-wise squashing function that maps vectors to (0, 1). Given the relation-strength psc ik and the incident node representations (¯vs i , ¯vck ), we then compute (i) an edge update vector ∆esc ik , (ii) a similarity-based scaling factor αsc ik from the node inner product, and (iii) a balancing vector βsc ik to trade off the current edge representation and the scaled update.

where [·] denotes concatenation, ⊙ is the element-wise product, Fsc(·) and Hsc(·) are learnable functions.

Similarly, for each exercise-concept edge (qj, ck) ∈ Eqc, we update the representation eqc jk analogously following Eqs. (7)-(8), using its context [pqc jk ; ¯vq j ; ¯vck ] and the functions Fqc(·) and Hqc(·), producing eqc jk ′ .

Node Refinement Sub-layer. Given the updated edge representations esc ik ′ and eqc jk ′ , we first compute the updated relation-strength vectors psc ik ′ and pqc jk ′ by Eq. (7). We then refine node representations by aggregating neighbor information weighted by these strengths, followed by residual connections and layer normalization.

As shown in Fig. 2(e), for each student node, we aggregate neighbor concept representations weighted by the studentconcept strengths psc ik ′ and update the node representation as:

Similarly, for each exercise node, the representation is refined based on the exercise-concept strengths pqc jk ′ :

For each concept node, we fuse neighbor representations from both the student nodes and the exercise nodes:

where Fs(·), Fq(·) and Fc(·) are learnable functions.

Batch-wise Relation-Strength. After nl layers of the graph encoder, we obtain the final edge representations on Esc and Eqc. To support mini-batch training, we sample a batch of B student-exercise interactions from the response log L, yielding index pairs {(ib, jb)}Bb=1 such that (sib , qjb , ribjb ) ∈ L. For each sample index b and each concept index k, we collect the corresponding student-concept and exercise-concept relation-strength vectors to form two 3D tensors M ∈ RB×nc×de and D ∈ RB×nc×de . For a required concept without a corresponding student-concept edge, which may occur when predicting an unobserved student-exercise pair, we use a hybrid estimate ˜psc ibk = 1 2 (¯psib + ¯pck ), where ¯psib and ¯pck denote the average strengths over the existing student-concept edges incident to student sib and concept ck, respectively:

Here, psc ibk and pqc jbk are the final relation-strength vectors obtained by applying Eq. (7) to the final edge representations.

The relation-strength vectors on student-concept and exercise-concept edges provide concept-level diagnostic factors for prediction. The student-concept strength reflects the mastery state of a student over a concept, while the exerciseconcept strength captures the difficulty level of an exercise with respect to that concept. The prediction head jointly models these two factors to estimate the correctness probability.

4.3 MoE-based Prediction Head

The MoE-based prediction head estimates the correctness probability for each sampled student-exercise pair (Fig. 2(c)). It uses multiple experts to capture varied mastery-difficulty discrepancies and a gating function to adaptively combine expert outputs. Given the batch-wise tensors M and D, we construct the discrepancy tensor X ∈ RB×nc×de :

where Xb,k,: represents the discrepancy between the mastery of student sib on concept ck and the difficulty of exercise qjb with respect to ck. This discrepancy serves as the core input to the expert predictors.

Gating Function. We instantiate ne expert predictors {Em}ne m=1 and compute their mixture expert weights using a gating function G(·). For each interaction (sib , qjb ) and concept ck, G(·) outputs the mixture weights πb,k ∈ (0, 1)ne conditioned on the corresponding student, exercise, and concept representations:

Here, τ is a temperature parameter that controls the smoothness of expert selection.

Expert Mixture Prediction. Given the discrepancy vector Xb,k,:, each expert Em produces a scalar logit, and the concept-level prediction is obtained by aggregating expert outputs with the mixture weights:

Finally, predictions are aggregated over the concepts involved in the exercise qjb to obtain the predicted correctness probability ˆ ribjb for the student-exercise pair (sib , qjb ):

4.4 Training Objective

We train RMCD by minimizing a prediction loss with an MoE load-balancing regularizer. For the batch {(ib, jb)}Bb=1 sampled from the response log L, we optimize the correctness probability ˆ ribjb against the ground-truth response ribjb . The prediction loss is instantiated as the binary cross-entropy:

To prevent the gating function from collapsing to a few experts, we introduce a load-balancing regularizer to encourage balanced expert utilization during training. For each expert Em, we compute its average mixture weight pm over the concept positions with Qjbk = 1 in the batch, and penalize the deviation of pm from the uniform usage 1 ne :

Finally, the overall objective is:

where λ controls the strength of load balancing.

5 Experiments

We evaluate the performance of RMCD by comparing it with various methods for cognitive diagnosis. Additionally, we conduct ablation studies for further evaluation and analysis.

5.1 Experimental Setup

Evaluation Benchmarks. We conduct experiments on three real-world cognitive diagnosis datasets: ASSIST17 [Feng et al., 2009], ASSIST09 [Feng et al., 2009], and Junyi [Chang et al., 2015]. Following common practice, we remove students with fewer than 15 response logs to ensure sufficient observations for reliable modeling. Dataset statistics are summarized in Tab. 1, and each dataset is split into 80% training and 20% testing.

Implementation Details. RMCD is implemented in PyTorch [Paszke et al., 2019] and optimized with Adam [Kingma, 2014]. We set the hidden dimensions to dv = de = 256 and use a mini-batch size of B = 256. The graph encoder stacks nl = 1 relation-aware GNN layer, where Eq. (5) is instantiated with GAT [Veliˇckovi´c et al., 2018]. The function Ψ(·) is implemented as sigmoid, and all F(·) and H(·) in the encoder are fully-connected (FC) layers. The gate G(·) is a two-layer MLP with ReLU, and each expert is an FC layer. On ASSIST17/ASSIST09/Junyi, we set (ne, η, λ) = (2, 5×10-5 , 0.4), (16, 5×10-4 , 0.01), and (2, 5×10-5 , 0.3), respectively, where η denotes the learning rate. All experiments are conducted on a single NVIDIA GeForce RTX 3090 GPU with an Intel Core i9-12900K CPU.

5.2 Evaluation Results

We compare RMCD with eight cognitive diagnosis models: IRT [Embretson and Reise, 2013], MIRT [Reckase, 2009], DINA [De La Torre, 2009], NCD [Wang et al., 2020], RCD [Gao et al., 2021], SCD [Wang et al., 2023b], ACD [Wang et al., 2024], and ISGCD [Shao et al., 2025]. All models are evaluated using ACC, RMSE, and AUC. For baselines, we adopt the reported results on ASSIST17 and Junyi from ACD, and those on ASSIST09 from RCD, except for DINA, SCD, and ISGCD as described below. Since the available results of DINA and SCD on ASSIST09 and those of ISGCD on all three datasets are not fully aligned with our evaluation protocol, we re-train these models under the same data split and original hyperparameter settings. For RMCD and re-trained baselines, we report mean and standard deviation over five runs.

Table 2 shows that RMCD consistently outperforms all compared models on all three datasets across ACC, RMSE, and AUC, with the largest gains on ASSIST17. This is consistent with the denser relations in ASSIST17 given its smaller student set and richer exercise and concept structures (Tab. 1). Moreover, RMCD outperforms ACD on ASSIST17 across all metrics without using affect-related information, where we report the best ACD variant, i.e., SCD augmented with affective states. Since ASSIST17 provides affect labels while ASSIST09 and Junyi do not, ACD further relies on an unsupervised affect perception module on datasets without affect labels. These results suggest that the relation-aware graph learning and the MoE-based prediction can improve cognitive diagnosis even without external auxiliary features. In addition, RMCD is also more stable, yielding smaller standard deviations among models.

5.3 Ablation Study

We conduct ablation studies to shed light on the impact of each key design within RMCD. The findings from these ablations create possibilities for further optimization.

MoE Head & Load Balancing. To examine the effects of conditional prediction and training regularization in RMCD, we ablate the MoE-based prediction head and the loadbalancing regularizer Lr (Fig. 3). Both components are beneficial. MoE brings larger gains, while Lr mainly regularizes training and improves ranking quality (RMSE/AUC). Removing MoE (w/o MoE) consistently degrades performance on all datasets, with the largest drop on ASSIST17, implying that under highly heterogeneous behaviors and complex relations, a single predictor is insufficient, whereas the MoEbased prediction better adapts to diverse samples. Removing Lr (w/o Lr) has a smaller impact on overall performance but yields consistently mild degradations in RMSE and AUC, suggesting that it stabilizes predicted probabilities rather than directly improving accuracy. On ASSIST09, the w/o MoE variant slightly improves ACC but hurts RMSE/AUC, while remaining competitive with prior models (Tab. 2). This suggests that the MoE head refines response probability distributions, improving ranking stability.

We further analyze how the graph encoder depth affects performance under w/o MoE. Table 3 shows that without MoE, the model benefits from a deeper encoder, indicating that ASSIST17 involves more complex student-exerciseconcept relations and a single predictor requires richer representations to capture mastery-difficulty discrepancies. With increased depth, the w/o MoE variant remains competitive with the compared models (Tab. 2), indicating that deeper graph encoding can partially compensate for the lack of conditional prediction. When MoE is used, RMCD achieves strong performance with a shallow encoder by adapting to fine-grained mastery-difficulty discrepancies, supporting the value of MoE in complex cognitive diagnosis scenarios.

Sub-layer Roles. We conduct sub-layer ablations of the relation-aware GNN layer to analyze the contribution of each sub-layer (Tab. 4). For brevity, the three sub-layers are denoted as N, E, and R. The results show that the sub-layers are complementary and play distinct roles. Prediction relies on edge representations, while gating is driven by node representations. Using only N or R yields limited performance due to missing explicit relation modeling and edge information. Using only E enables prediction but remains limited without node feedback for effective gating. Combining sub-layers leads to clear improvements. E+R outperforms N+E, highlighting the importance of injecting relation-aware edge information into node representations. Building on this, RMCD progressively integrates N, E, and R, achieving the best performance across all three metrics.

Sub-layer Order. Building on the necessity of combining the three sublayers, we further study how their order affects performance. Table 5 shows that different orders lead to consistent changes in all metrics, indicating that relation learning is order-sensitive. Under our design, node representations are first learned, edge representations are then updated based on relational structures, and nodes are finally refined through node-edge interactions. This progressive scheme leads to smoother and more hierarchical message passing. In contrast, updating edges too early or skipping node refinement after edge updates weakens information propagation and degrades performance. Overall, the order N → E → R achieves the best and most stable results, supporting our design of aligning representation learning with MoE-based prediction.

Gating Input. We ablate the input of the gating function by feeding different combinations of student, exercise, and concept representations (Tab. 6). The results show that the gating input affects performance. Using a single representation yields limited results, indicating that one type alone is insufficient to combine experts effectively. Using two representations improves performance, with [vs ; vq ] outperforming combinations involving concepts, suggesting that student and exercise information provides a stronger gating signal. Adding vc brings only marginal but consistent gains, since concept information has been largely encoded by the relationaware graph encoder and helps stabilize the gating decision.

Hyperparameter. We study hyperparameter sensitivity by varying the graph encoder depth nl and the number of experts ne on the three datasets, reporting mean results over three runs. Since ACC and RMSE show similar trends to AUC, we focus on AUC. Figure 4 shows that a shallow encoder already performs strongly and stably, and increasing nl brings no consistent gains and may slightly hurt, likely due to redundant or noisy signals in the gating input. In contrast, under the w/o MoE setting (Tab. 3), deeper encoders consistently help, suggesting that MoE reduces the reliance on deeper graph encoding. RMCD also works well with few experts, and increasing ne is not monotonic. In particular, ASSIST17 tends to drop with larger ne, while ASSIST09 and Junyi are relatively flat, indicating that gains come from effective conditional prediction rather than many experts.

Efficiency. To analyze the effect of theMoE head onmodel complexity and efficiency, we vary the number of experts ne while keeping all other settings fixed, and report results together with RCD and SCD in Tab. 7. For a fair comparison, RMCD, RCD, and SCD use the same hidden dimension of 123. RMCD trains much faster per epoch, averaged over three consecutive epochs, than both baselines, although it has more parameters due to the use of edge features. Increasing ne from 2 to 32 only marginally increases parameters and slightly increases per-epoch time, indicating limited overhead. This is because MoE is only applied at the prediction stage. Overall, RMCD achieves a favorable balance between efficiency and performance.

6 Conclusion

In this paper, we propose RMCD, a cognitive diagnosis model that unifies relation-aware graph learning with Mixture-ofExperts prediction. RMCD builds a student-exercise-concept relational graph and jointly learns node and edge representations, where relation-strength vectors support edge updating, node refinement, and concept-level mastery-difficulty modeling. The MoE head models concept-level mastery-difficulty discrepancies with conditional expert combination. Experiments on three benchmarks show consistent gains over stateof-the-art methods. RMCD shows encouraging results but still has limitations. It builds student-concept edges with a heuristic rule and maps edge representations to relation strengths with a fixed squashing function, which may overlook richer dependency cues. In future work, we plan to improve relation construction and introduce stronger constraints for more interpretable relation strengths.

Acknowledgments

This work was supported by the Fundamental Research Funds for the Central Universities (Nos. SWU-KR22032, SWUXDJH202303, and SWU222001) and the Innovation and Entrepreneurship Education Teaching Reform Research Project of Southwest University (No. 26SCJY29).