CHoE: Cross-Domain Heterogeneous Graph Prompt Learning via Structure-Conditioned Experts

1 Introduction

Heterogeneous Graphs (HGs) are ubiquitous across nearly all application domains, including biomedical [Davis et al., 2017] networks in biological and medical applications and bibliographic [Hu et al., 2020] networks in scientific research. In recent years, Heterogeneous Graph Neural Networks (HGNNs) have attracted significant attention for modeling and analyzing HGs. Most HGNN methods rely heavily on large amounts of labeled data, which is costly to obtain. To mitigate label dependency, "the pre-training and then finetuning" paradigm provides an effective alternative by using

∗Corresponding author.

Code is avaliable at: https://github.com/hedongxiao-tju/CHoE

unlabeled data to learn and adapt general knowledge to specific downstream tasks. This paradigm reduces dependence on labeled data while maintaining transferability across diverse downstream applications [Hu et al., 2020].

In practice, adapting a pre-trained model to accommodate diverse downstream conditions requires efficient, robust mechanisms beyond task-specific full fine-tuning [Shan et al., 2026]. Inspired by the success of prompt learning in natural language [Zhou et al., 2022] and computer vision [Nie et al., 2023], recent studies have attempted to extend this paradigm to graphs. Graph Prompt Learning (GPL) [Sun et al., 2023] has emerged as a powerful and versatile approach. By constructing flexible, lightweight prompting modules for specific downstream tasks, GPL aligns pre-trained models with downstream tasks and alleviates negative transfer [Huang et al., 2025]. Most existing GPL methods fix the parameters of pretrained models and only fine-tune lightweight prompt modules, enabling efficient adaptation of pre-trained models to downstream tasks. Some studies have explored GPL methods in heterogeneous graphs. HGPrompt [Yu et al., 2024] decomposes a heterogeneous graph into multiple homogeneous subgraphs and introduces unified-feature and subgraph-specific prompts, enabling models to adapt to downstream tasks. HetGPT [Ma et al., 2024] further enriches prompt design by incorporating virtual-class prompts and introducing nodespecific feature prompts. HetGPT also employs a multi-view neighborhood aggregation mechanism to model semantic and structural information in heterogeneous graphs.

However, despite recent advancements, existing HGPL methods are typically designed for in-domain scenarios. In real-world applications, models are often required to operate across domains, where source and target domains follow distinct distributions [Liang et al., 2026; Wang et al., 2026]. For example, a model may be pre-trained for biomedical networks but is applied to downstream applications on chemicals. The applicability of these HGPL methods remains limited for cross-domain scenarios. Changes in application domains typically degrade the performance of HGPL methods. As shown in Figure 1, existing HGPL methods exhibit reduced performance in cross-domain settings, and carefully designed prompt modules are less competitive than fully finetuning the pre-trained model. Therefore, addressing the performance degradation problem in cross-domain scenarios is an urgent issue in heterogeneous graph learning.

Effective cross-domain learning on heterogeneous graphs faces two key challenges. The first is making a pre-trained model compatible with downstream graphs from different domains. In such settings, both the types and counts of metapaths can vary. Because existing methods are tightly coupled to pre-defined meta-paths, their pre-trained representations transfer poorly to new domains. The second challenge is enabling knowledge transfer under semantic and structural distribution shifts. Semantic and structural patterns often differ significantly across domains. Even simple adaptations that render current methods nominally cross-domain tend to fail under these shifts, resulting in noticeable performance degradation. Therefore, there is a clear need for a cross-domain framework that keeps pre-trained models compatible with diverse downstream graphs while explicitly bridging semantic and structural discrepancies between domains.

To address these challenges, we propose CHoE, a crossdomain HGPL method built upon an expert network perspective. In the pre-training stage, we design a generative self-supervised learning framework that reconstructs masked node features and edges, encouraging the model to learn universal semantic and structural knowledge from the source domain. In the prompt-tuning stage, we introduce structure-conditioned experts learned during pre-training, allowing each meta-path view in the downstream graph to adaptively select and combine expert representations. Specifically, we develop a structure-aware expert routing mechanism that evaluates experts according to their ability to model the downstream graph structure and selects those that are structurally compatible, together with a load balancing mechanism that uses accumulated routing statistics from previous epochs to promote balanced expert usage and prevent biased expert selection. At the semantic level, we propose a prompt-based semantic fusion module that introduces meta-path-specific prompts to enable adaptive fusion of representations from different semantic views, thereby achieving more robust and efficient knowledge transfer across domains. In summary, our main contributions are as follows:

• We identify an under-explored but practically impor-
• tant cross-domain scenario for heterogeneous graph and show that existing HGPL methods degrade under semantic and structural shifts.
• We propose CHoE, a cross-domain HGPL method based on structure-conditioned experts. By transferring a pretrained expert pool and enabling structure-aware routing and load balancing, CHoE mitigates cross-domain structural discrepancies. Moreover, a prompt-based semantic fusion module is introduced to achieve cross-domain semantic adaptation.
• We conduct extensive few-shot experiments on four heterogeneous graph datasets. The results consistently demonstrate the effectiveness and robustness of CHoE in both cross-domain and few-shot settings.

2 RelatedWork

Heterogeneous graph pre-training. Graph pre-training has attracted growing attention in recent years, and numerous methods for heterogeneous graphs have been proposed. Among them, DMGI [Park et al., 2020] and HDMI [Jing et al., 2021] extend DGI [Veliˇckovi´c et al., 2019] to heterogeneous settings by maximizing mutual information between node representations and graph-level summaries within each meta-path view and then aggregating multi-view representations into a unified embedding. Inspired by graph contrastive learning, HeCo [Wang et al., 2021] employs a cocontrastive framework that contrasts local and higher-order views, whereas HGCML [Wang et al., 2023] performs multiview contrastive learning by constructing several meta-pathbased views. HGMAE [Tian et al., 2023] adopts a generative approach that reconstructs masked node features and masked edges. In contrast to these meta-path-dependent approaches, HERO [Mo et al., 2024] dispenses with pre-defined metapaths by modeling homophily through a self-expression matrix and capturing heterogeneity via a type-aware attentionbased aggregation of node representations

Heterogeneous graph prompt learning. Heterogeneous graph pre-training methods have shown strong representation learning capabilities but often suffer from negative transfer when adapted to diverse downstream tasks. HGPL has thus emerged as a new paradigm. HGPrompt [Yu et al., 2024] decomposes a heterogeneous graph into multiple homogeneous subgraphs and introduces unified-feature and subgraph-specific prompts, while HetGPT [Ma et al., 2024] further enriches the prompt space by incorporating virtualclass prompts and node-specific feature prompts, together with a multi-view neighborhood aggregation mechanism to capture complex neighborhood. HiGPT [Tang et al., 2024] leverages context-parameterized heterogeneity projectors and large language models to generate node representations and applies instruction tuning to better align with downstream tasks. Distinct from prior work that primarily emphasizes prompt design, GPAWP [Wei et al., 2025] instead investigates how prompts function in practice, proposing a promptimportance evaluation mechanism to quantify their contributions and prune ineffective or negatively impactful prompts.

Despite their effectiveness, existing methods remain limited to in-domain settings and have not yet fully achieved cross-domain transfer, which this paper aims to address.

3 Preliminary

Definition 3.1 Heterogeneous Graph. We define a heterogeneous graph as G = (V, E, A, R, X, P), where V = {vi}Ni=1 denotes the set of N nodes and X = {xi}Ni=1 represents the corresponding node features. The edge set is defined as E = {(va, r, vb)}Mj=1 , where each edge connects node va to node vb via a relation type r ∈ R. A and R denote the sets of node types and relation types, respectively, with |A| + |R| > 2. P = {pi}|P| i=1 is a set of meta-paths, where each meta-path pi is defined as a sequence of relations r1, r2, . . . , rs, i.e., v1 r1 -→ v2 r2 -→ · · · rs -→ vs+1, and s is the length of the meta-path.

Definition 3.2 Heterogeneous Graph Prompt Learning. HGPL adapts a pre-trained heterogeneous graph model to downstream applications by introducing learnable prompt parameters while keeping the pre-trained model fixed. Given a heterogeneous graph G with a set of meta-paths P, HGPL injects prompts into node features or graph topology to guide task-specific learning. Formally, it can be defined as:

where Gi denotes the graphs corresponding to meta-path pi ∈ P, pi is a learnable prompt specific to pi, and f(·) is a prompt injection function, producing the prompted graph ˜ Gi for downstream tasks.

Definition 3.3 Mixture of Experts (MoE). MoE has achieved notable success in language [Cai et al., 2025] and vision [Riquelme et al., 2021] domains. It is a modular learning paradigm consisting of a set of experts and a routing function that selects and combines compatible experts. In graph representation learning, given an input representation hi of a node vi, an MoE model maintains an expert pool E = {E1, E2, . . . , EM}, where each expert Em(·) ∈ RN×d is a learnable function. A routing function g(·) generates the routing weights αi from hi via a softmax operation. The output of the MoE h′i is obtained as:

4 Methodology

To resolve the challenges described in Section 1, we propose CHoE, which consists of four components: a selfsupervised pre-training framework, a structure-conditioned experts prompt-tuning module, a structure-aware expert routing and load balancing mechanism, and a prompt-based semantic fusion module. The overall CHoE framework is illustrated in Figure 2.

4.1 Self-supervised Graph Pre-training

In the pre-training stage, we adopt a generative learning framework to learn general semantic and structural information in the source domain. Given a heterogeneous graph Gsrc = (Vsrc,Xsrc,P), which contains a set of pre-defined meta-paths P = {pi}|P| i=1 , all methods in this subsection are performed on a single meta-path view unless otherwise noted. We first construct a meta-path-based adjacency matrix A ∈ RN×N via meta-path sampling [Dong et al., 2017], and apply Singular Value Decomposition (SVD) to align feature dimensions across domains, obtaining N×F -dimensional node features Xsrc ∈ RN×F . Then, we design an autoencoder architecture, in which the encoder fE(·) ∈ RN×d encodes node features Xsrc and meta-path-based adjacency matrices A to node representations, and the decoder fD(·) ∈ RN×F reconstructs the graph structure and node features from the learned representations.

Masked Feature and Edge Reconstruction Tasks

To achieve self-supervised heterogeneous graph pre-training, inspired by the representative masked reconstruction paradigm GraphMAE [Hou et al., 2022], we design masked feature and edge reconstruction tasks that aim to capture semantic and structural information in heterogeneous graphs.

In the masked feature reconstruction task, for the adjacency matrix A, we construct a feature masking matrix following a Bernoulli distribution Mf ∼ Bernoulli(r1), where r1 is the feature masking ratio. The masked features are obtained as ˜X = Mf⊙Xsrc+(1-Mf)⊙X[M], where X[M] denotes learnable mask embeddings. We feed ˜X and A into the encoder fE(·) to obtain the node representations Hf. Subsequently, the decoder fD(·) reconstructs the node features as ˆX ∈ RN×F , which can be formulated as:

The feature reconstruction loss Lf is formulated to measure the discrepancy between the ˆX and ˜X with a scaling factor γ1, which is defined as:

where V m src denotes the set of masked nodes. By applying the above procedure to each meta-path view pi ∈ P, we obtain {Lif }|P| i=1 . To aggregate Lif across meta-path views, we introduce an attention vector q ∈ RF to compute the importance score for each meta-path, and then apply a softmax to obtain the attention weight wi f , which is defined as:

where W is a learnable weight matrix and b is a bias vector. The loss Lfeat is defined as the weighted aggregation of L i f across all meta-paths:

In the masked edge reconstruction task, for the adjacency matrix A, we construct an edge masking matrix Me ∼ Bernoulli(r2), where r2 is the edge masking rate. The masked adjacency matrix is obtained as ˜A = Me ⊙ A. Given the node features Xsrc ∈ RN×F and ˜A, the fE(·) encodes the node representations He, which are further decoded by fD(·) to obtain Z. This process can be formulated as:

We compute the discrepancy between ˆA = σ(ZZ⊤) and A with a scaling factor γ2:

After applying the edge reconstruction task to all meta-paths, we obtain {Lie }|P| i=1 . We then calculate the corresponding attention weight wie according to Eq. 5, and the loss Ledge is obtained via a weighted aggregation:

Finally, the pre-training loss Lpre is defined as:

where λedge is used to balance the Lfeat and Ledge.

4.2 Structure-Conditioned Experts Prompt-tuning

In the fine-tuning stage, meta-paths in downstream graphs often differ from those in the source domain in type and number, limiting the compatibility of pre-trained models with downstream graphs. To address this challenge, we introduce structure-conditioned experts. The encoder fE(·) has learned general knowledge in the pre-training stage. We fix its parameters and adopt it as an expert pool E = {El | l = 1, 2 . . .L}. For the l-th expert El , it contains structure-conditioned experts, which is expressed as El = {El i | i = 1, 2 . . . |P|}, where each expert corresponds to a source-domain meta-path. Each El i is implemented as a Graph Attention Network [Veliˇckovi´c et al., 2018] encoder.

Given a heterogeneous graph Gtar = (Vtar,Xtar, T ), which contains a set of meta-paths T = {tj}|T | j=1 , all methods in this subsection are performed on a single meta-path view unless otherwise noted. We construct a meta-path-based adjacency matrix A and align feature dimensions across domains, obtaining N × F -dimensional node features Xtar ∈ RN×F . We then define Hl as the node representations at the l-th expert layer, with H0 = Xtar. Each expert El i generates the expertspecific node representations Hli , which are as follows:

Structure-Aware Expert Routing and Load Balancing

After obtaining node representations Hli of all experts at layer l, we learn routing weights to aggregate them into Hl . Based on the structural distribution discrepancy across domains, we design a routing mechanism that evaluates the structural consistency between A and Hli . An expert that can reconstruct a meta-path view should be assigned a large routing weight for that view. Moreover, connected nodes in the meta-path view are expected to have similar representations. Formally, given positive samples va, vb ∈ Vtar and negative samples vc, vd ∈ Vtar, we construct positive and negative sample sets:

where positive samples are randomly selected from connected node pairs and negative samples are sampled from other node pairs. We compute the similarity for each sampled node pair and obtain a positive score s+l i and a negative score s-l i by averaging the similarities within each set:

where hlu,i denotes the representation of node vu produced by El i . Then, we compute the expert score rli :

where τ denotes the temperature parameter.

Then, we note that assigning expert weights solely based on expert scores can lead to severe expert imbalance, where a small subset of experts dominates the routing decisions across layers. To address this issue, we further introduce a load balancing mechanism that adaptively penalizes experts based on their cumulative routing weights in previous epochs, encouraging a more balanced allocation of experts. Specifically, we maintain an expert load counter mi that records the cumulative sum of routing weights assigned to expert Ei across all layers over previous epochs. At the current epoch, we compute the relative load of expert Ei as:

Based on ˜mi, we use an exponential decay regularization term γi to compute the final routing score ˜ rli :

where λbalance controls the strength of expert balancing. Subsequently, the routing weights are computed by applying normalization to the routing scores:

Finally, we aggregate these expert-specific representations Hli using the routing weights αli to obtain H l , and update the expert load counter mi with αli :

Prompt-Based Semantic Fusion Module

While the structure-aware expert routing mechanism selects the most compatible experts for each meta-path from a structural perspective, thereby producing node representations for each meta-path view, the downstream tasks further require an effective integration of these representations. Therefore, we introduce a prompt-based fusion module to adaptively aggregate representations across different semantic views.

After applying the above procedure layer by layer to each meta-path view tj ∈ T , we obtain a set of representations at the L-th expert layer for all meta-path views in the target domain: {HLj }|T | j=1 . Then, to generate a global representation for each meta-path view, we perform mean pooling over all nodes:

in which hj denotes the global representation of meta-path tj . We introduce a learnable prompt vector pj for each metapath to assess its semantic importance, computing a semantic importance score sj that is further normalized across all metapaths to obtain the corresponding semantic weight βj :

The final Hfinal is obtained by aggregating meta-path-specific node representations with their semantic weights and then fed into a linear classifier fC(·):

During the fine-tuning stage, we optimize the Lft over labeled nodes:

where V label tar is the set of labeled nodes, yv and ˆyv denote the true label and predicted label of node v, respectively. Notably, only {pj}|T | j=1 , and the parameters of fC(·), are updated during fine-tuning, enabling the pre-trained model to achieve efficient cross-domain transfer.

5 Experiments

5.1 Experimental Setup

Datasets and Baselines

In our experiments, we adopt four widely used heterogeneous graph datasets: ACM, DBLP, Aminer, and Freebase. Each dataset consists of multiple node types and meta-paths. The statistics of the datasets are shown in Table 1. We compare our method with five representative categories of existing methods, including: (1) homogeneous self-supervised methods: DGI [Veliˇckovi´c et al., 2019] and GRACE [Zhu et al., 2020]; (2) heterogeneous self-supervised methods: HeCo [Wang et al., 2021], HGMAE [Tian et al., 2023], and HERO [Mo et al., 2024]; (3) homogeneous graph prompt learning methods: GraphPrompt [Liu et al., 2023], GPFplus [Fang et al., 2023], and EdgePrompt [Fu et al., 2025]; (4) heterogeneous graph prompt learning methods: HGPrompt [Yu et al., 2024] and HetGPT [Ma et al., 2024]; (5) cross-domain homogeneous graph prompt learning methods: GraphLoRA [Yang et al., 2025].

Implementation Details

We follow [Dong et al., 2017] to select meta-paths for each dataset. For the proposed CHoE, we adopt HAN [Wang et al., 2019] as the encoder and decoder in the pre-training stage. For both the baselines and our method, models are pretrained for 10,000 epochs and fine-tuned for 500 epochs, with early stopping (patience=20). We independently searched for the learning rates for the pre-training and fine-tuning stages within the range of 1e-5 to 0.01. To obtain reliable results, we perform 20 repeated runs for each of 5 fixed random seeds, reporting average results over 100 trials.

5.2 Cross-Domain Node Classification

5-shot experiments on different downstream datasets. We report 5-shot node classification on four datasets using ACM for pre-training. As shown in Table 2, CHoE consistently outperforms all baselines, with the largest gains on Aminer and Freebase. On DBLP, the improvement is smaller because its dense graph structure enables baselines (e.g., HGPrompt and GraphPrompt) to capture sufficient structural information during fine-tuning, reducing the difficulty of crossdomain transfer. On ACM, CHoE achieves competitive performance against strong in-domain baselines (e.g., HERO and GRACE), demonstrating its effectiveness in both crossdomain and in-domain settings.

1/3-shot experiments on different downstream datasets. We conduct 1-shot and 3-shot node classification experiments using ACM for pre-training to evaluate the cross-domain adaptability of CHoE. As shown in Figure 3, CHoE consistently outperforms all baselines across different settings. The performance gains are more significant on Aminer and Freebase as the number of shots increases, while stable improvements are observed on DBLP. On ACM, CHoE remains competitive with strong in-domain baselines.

5-shot experiments on different pre-training datasets. To further evaluate the cross-domain capability of CHoE, we conduct 5-shot node classification on all combinations of ACM, DBLP, Aminer, and Freebase as upstream and downstream datasets, as shown in Table 4. CHoE achieves superior and more stable performance across most settings. Although HGPrompt shows higher performance in some upstream domains, its results often performance under domain shifts, indicating limited transferability. In contrast, CHoE generalizes better across domains, particularly on Aminer and Freebase, demonstrating stronger cross-domain knowledge transfer ability.

5.3 Ablation Study

To evaluate the effectiveness of each component in CHoE, we compare it with three variants: "w/o PR" (without the prompt module), "w/o LB" (without load balancing), and "w/o ER" (without structure-aware expert routing and load balancing). As shown in Table 3, we report 5-shot node classification on four datasets using ACM for pre-training. Removing expert routing leads to the largest performance drop, highlighting the importance of selecting structure-compatible experts for downstream graphs. Performance also decreases for w/o PR" and "w/o LB", demonstrating the effectiveness of the prompt module and load balancing mechanism.

5.4 Hyperparameter Analysis

To evaluate the sensitivity of CHoE to hyperparameters, we conduct analyses on the balancing coefficient λbalance, the temperature parameter τ, and the number of model layers L. As shown in Figure 4, the performance remains stable across a wide range of λbalance and τ, indicating that CHoE is not sensitive to these hyperparameters. We further investigate the effect of the model depth by varying the number of layers L. As shown in Table 5, CHoE achieves the best performance when L = 2 on all four datasets. 1-layer model lacks sufficient capacity to capture the structural and semantic knowledge from source domain. Increasing the number of layers to 3 leads to performance degradation, which may be caused by over-smoothing and the introduction of redundant information. This result suggests that a moderate model depth is more effective.

6 Conclusion

In this work, we find that existing HGPL methods rely on an in-domain assumption. Through a motivation experiment, we observe that HGPL methods suffer from severe performance degradation under domain shifts. Based on the observation, we identify two key challenges: model incompatibility and knowledge transfer difficulties. To address these challenges, we propose CHoE. CHoE introduces structureconditioned experts to transfer a pre-trained expert pool to downstream domains. It further incorporates a structureaware expert routing and load balancing mechanism to select structurally compatible experts, and a prompt-based semantic fusion module to integrate different semantic views. Extensive experiments validate the effectiveness of CHoE in crossdomain and in-domain scenarios. In summary, we propose a cross-domain HGPL method that accounts for model compatibility and cross-domain knowledge transfer, offering new insights for the heterogeneous graph foundation models.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 62422210, No. 62276187, No. 92370111, and No. 62272340), the Hong Kong RGCThemebased Strategic Target Grant Scheme (STG STG1/M-501/23N), the Hong Kong Global STEM Professor Scheme, and the Hong Kong Jockey Club Charities Trust.

Contribution Statement

Peiyuan Li and Yongqi Huang contributed equally to this work.