A Unified Prompt for Enhancing Heterogeneous Graph Pre-training via Edge-based Message Passing

1 Introduction

Real world complex data, such as recommendation systems, social networks, and academic citation networks [Cao et al., 2025; Hui et al., 2025; Zhao et al., 2020], often contain multirelational information. These scenarios can be naturally modeled as heterogeneous information networks. For representation learning on heterogeneous information networks, existing semi-supervised methods have been extensively studied [Wang et al., 2019; Hu et al., 2020; Fu et al., 2020a]. However, many real-world scenarios suffer from a lack of high-quality labeled data, which significantly limits the effectiveness of existing semi-supervised models. To address the challenge of limited labeled data, pre-training methods for heterogeneous information networks have emerged. These methods follow a "pre-train, fine-tuning" paradigm and involve carefully designed pre-training tasks [Jing et al., 2021; Wang et al., 2021; Tian et al., 2023], thereby mitigating the drawback of scarce labels.

∗Corresponding Authors.

However, a significant gap often exists between pretraining tasks and downstream tasks in heterogeneous graphs. To mitigate the side effects of such task gaps, existing work has drawn inspiration from the concept of prompts in NLP [Devlin et al., 2019; Radford et al., 2018; Brown et al., 2020], aiming to bridge the connection between upstream and downstream tasks, and has introduced the "prompt-tuning" paradigm during the fine-tuning stage. In homogeneous networks, this paradigm has been reasonably well studied, primarily by adding prompt information to existing node features or representations [Liu et al., 2023; Fang et al., 2023; Huang et al., 2026], or by inserting new nodes to simulate the form of prompts in NLP [Sun et al., 2023]. However, such approaches overlook the crucial relational information in heterogeneous information networks, leading to the loss of effective information in multi-relational scenarios.

To address the introduction of multi-relations, recent years have witnessed a number of prompt-based studies on heterogeneous information networks, which demonstrate the strong expressive power of this paradigm compared with pre-trained models. However, existing heterogeneous prompt methods primarily focus on holistic framework design and exhibit strong dependencies on specific pre-trained models, which limits their generalizability across different pre-training architectures. For example, the prompt design in HGPROMPT [Yu et al., 2024] is tightly coupled with its custom link-prediction-based pre-training scheme; the use of prompts in HGMP [Jiao et al., 2025] is bound to its specific, unified graph-level pre-training task; and the prompt transfer strategy in HetGPT [Ma et al., 2024] is constrained by the type of pre-training framework, which task transfer approach is only reusable within contrastive learning frameworks and difficult to adapt to other pre-training paradigms.

We note that various heterogeneous graph pre-trained models share a common encoder component, and message passing is a core stage in heterogeneous graph representation learning. This leads us to consider: can we design a general prompt mechanism based on the encoder to address the above generalization issue? While the existing method EdgePrompt [Fu et al., 2025], designed for homogeneous graphs, attempts to incorporate prompts during propagation, it overlooks the critical relational semantic information carried by multi-relation edges in heterogeneous settings. Moreover, employing multi-relational edge-level prompts in message-passing allows prompts to function independently across different semantic relations, thereby alleviating the optimization conflicts inherent in existing methods, such as the heterogeneous graph prompts above, which mainly focus on prompts at the node feature or representation level. For instance, in a recommendation system scenario illustrated in Figure 1, a single prompt attached to a user node propagates to both movie and director nodes during message-passing, yet fails to account for the potential conflicts arising from users' divergent preferences toward different movies and directors when optimizing the prompt.

To achieve the above objectives, we propose a novel method Heterogeneous Graph Multi-Relational Prompt (HGMRP). We leverage edge-based message passing to effectively achieve independent propagation. In heterogeneous graphs, relationships between different types of nodes carry distinct semantic information and play a crucial role during propagation. Therefore, edges of the same type share the same prompt vector, while prompt vectors differ across different edge types. These added multi-relation prompts are incorporated into the message passing process and are trained during downstream fine-tuning tasks. To enhance the multirelation prompts, we extend the single shared prompt vector per relation into a composite prompt structure, which includes both relation-specific and shared components across different relations. To validate the effectiveness of our method, we conduct extensive experiments on 4 heterogeneous graph datasets. Our approach consistently improves performance across different types of graph pre-trained models and outperforms existing heterogeneous prompt methods. The contributions of our work can be summarized as follows:

• Our proposed method breaks away from the traditional integrated framework of "pre-training, prompt-tuning". It adapts to a wider variety of heterogeneous graph pretrained models and unifies the approach to prompt-based fine-tuning for heterogeneous graph pre-training.
• We introduce a novel approach to prompt-based finetuning for heterogeneous graphs. To the best of our knowledge, it is the first to employ message-passingbased prompts in heterogeneous graph scenarios, highlighting the importance of relational information in such graphs. This opens up new directions for future research on heterogeneous graph prompts.
• We conduct extensive experiments on 4 commonly used heterogeneous graph datasets, comparing our method against 3 categories of heterogeneous graph representation learning approaches. The results demonstrate and validate the effectiveness of our proposed method.

2 RelatedWork

2.1 Heterogeneous Graph Pre-training

To address the issue of insufficient model expressivity caused by the scarcity of high-quality labels, the pre-training paradigm has been proposed and widely applied to graph data. For handling multi-relational scenarios, numerous studies in recent years have focused on pre-training for heterogeneous graphs. For instance, DMGI [Park et al., 2020] and HDMI [Jing et al., 2021] employ the maximization of mutual information between local node representations and global graph summaries as their pre-training objective. HeCo [Wang et al., 2021] utilizes contrastive learning between positive and negative samples from dual views, while HGMAE [Tian et al., 2023] adopts reconstruction of both structural and attribute information across multiple views. HERO [Mo et al., 2024] aligns homogeneous and heterogeneous perspectives within heterogeneous graphs. All the aforementioned methods solely employ corresponding task heads, such as classifiers, when adapting to specific downstream tasks.

2.2 Prompt-based Heterogeneous Graph Tuning

In natural language processing, prompt tuning helps bridge the gap between pre-training tasks and downstream tasks. Inspired by this, numerous studies in recent years have explored prompt-based methods on graphs, such as ProG [Sun et al., 2023] and GPF [Fang et al., 2023]. These works primarily focus on homogeneous graph scenarios. To address prompt learning in heterogeneous settings, a series of approaches have also been proposed. For instance, HGPROMPT [Yu et al., 2024] introduces a framework from heterogeneous graph pre-training to prompting, HGMP [Jiao et al., 2025] unifies both pre-training and downstream tasks as graph-level tasks, and HetGPT [Ma et al., 2024] designs adaptation methods using positive and negative sample pairs for upstream and downstream tasks. However, existing methods design prompts mainly by modifying features or representations, overlooking the positive and negative effects during the message propagation in the fine-tuning process.

3 Preliminaries

3.1 Heterogeneous Graph

A heterogeneous graph is formally defined as a tuple G = (V, E, ϕ, ψ, X). Here, V = {v1, v2, . . . , vN} denotes the set of nodes, and E = {e1, e2, . . . , eM} ⊆ V × V denotes the set of edges connecting them. The graph's heterogeneity is captured by two mapping functions: the node type mapping ϕ : V → TV assigns each node to a type from a predefined set TV , and the relation type mapping ψ : E → RE assigns each edge to a type from a predefined set RE, where the condition |TV | + |RE| > 2 must hold, and XTi ∈ RN×dTi represents the node feature matrix, Ti ∈ TV , with xi ∈ Rd being the feature vector for node vi.

3.2 Heterogeneous Graph Pre-training

The pre-training framework for heterogeneous graphs typically consists of two sequential stages. In the first stage, given a heterogeneous graph G, a heterogeneous graph encoder fθ and a pre-training head gϕ. The model parameters θ and ϕ are jointly optimized by minimizing a pre-training loss Lpre constructed from self-supervised signals Vself derived from the graph itself depends on the pre-training strategy:

For downstream tasks, the pre-trained encoder fθ and pretraining head gϕ are frozen, and only a task-specific classifier clsψ is trained on a limited set of labeled data Ddown = {(vi, yi)}. Its parameters ψ are optimized by minimizing a downstream supervised loss, such as cross-entropy, the finetuning process can be described as:

3.3 Heterogeneous Graph Prompt Tuning

In this stage, the pre-trained graph encoder fθ and the pretraining head gϕ∗ are kept frozen to preserve the knowledge acquired during self-supervised pre-training. Only the prompt parameters p and a task-specific classifier clsψ are trained using a labeled dataset Ddown. The heterogeneous graph prompt tuning can be described as:

4 Method

In this section, we will briefly introduce the heterogeneous graph pre-training module, followed by the design and application of prompts within the pre-trained model.

4.1 Overview Framework

Our model primarily consists of two parts. As illustrated in Figure 2, the first part is the heterogeneous graph pre-training module, where unified features are fed into a designed pretraining task for computation. For the second part, we introduce a guided propagation-based multi-relational prompt, termed HGMRP, into the pre-trained model. It incorporates a single learnable parameter in each layers of a heterogeneous graph neural network. To further enhance the expressive capacity for multi-relational scenarios, we propose an advanced form, HGMRP+, which designs multiple sets of learnable parameters for different relations across various network layers. Finally, trained with limited labeled data, we obtain controllable propagating fine-tuning prompts.

4.2 Feature Transformation

In heterogeneous scenarios, different types of nodes possess features with varying semantics and dimensions. Existing encoders are primarily divided into meta-path-based methods and attention-based methods. The former constructs homogeneous graphs based on predefined meta-paths, thus requiring no feature transformation, while the latter computes attention scores among different node types and relies on features with consistent semantics and dimensions. Our prompt-based finetuning method is applicable to both approaches. The process of unifying features hTV of type TV is described as follows:

where TV represents the type of the nodes, wTV is the typespecific transformation matrixs, σ is the activation function.

4.3 PretrainedModel

To better illustrate how our prompts are integrated, we briefly outline the two major types of heterogeneous graph pretraining methods that our framework is designed to accommodate. As shown in the overall framework (Figure 2), the transformed node features are fed into a heterogeneous graph encoder to obtain node representations:

which are then passed to task-specific pre-training heads for optimization. Specifically, for contrastive pre-training, the objective typically involves learning representations by distinguishing between positive and negative sample pairs. For reconstructive pre-training, a graph decoder is used to recover the original graph data by minimizing the loss between the original and reconstructed features and structures.

All these pre-training tasks employ an encoder-based architecture. Given that our approach incorporates guided propagation-based prompts into the heterogeneous graph encoder, it can be seamlessly adapted to various types of pretrained models.

4.4 Multiple Relational Prompt Design

To enable the independent propagation of prompts in heterogeneous graphs, inspired by EdgePrompt for homogeneous graphs and taking into account the unique structural specificity of heterogeneous graphs, we first design the HGMRP, which innovatively introduces prompts into the propagation mechanism. To enhance the expressiveness of HGMRP, an advanced version termed HGMRP+ is further proposed. Both are detailed in the following sections.

HGMRP. In the heterogeneous graph encoding stage, to avoid interfering with feature propagation while incorporating prompts, we design a novel edge fusion propagation mechanism. We assign a shared learnable prompt vector for each relation type at each layer. In two widely-used heterogeneous graph encoders both propagate node representations during message passing but do not explicitly consider how multi-relation edges can be fused in this process. Therefore, we integrate relation-aware prompts by combining edge messages during the aggregation phase. Specifically, in each metapath view, the propagation with multi-relation prompts is formulated as follows:

where p(l) r denotes the prompt vector for relation r at layer l, and Nr(vi) represents the set of neighbors connected to vi via relation r. After obtaining the prompt-enhanced representations for each view, we aggregate them across different relations to derive the final node representation:

where R denotes the set of relation types, and AGG denotes an aggregation function such as attention. This entire propagation framework is referred to as HGMRP. In attentionbased methods, we likewise design a prompt for each relation type at every layer. During the heterogeneous message passing steps, when computing attention, the prompt vector interacts with the query and key to adjust the attention weights.

HGMRP+. While the aforementioned HGMRP achieves independent aggregation of different semantics during message passing in the fine-tuning stage through the design of relation-specific prompts, it faces limitations in complex pretrained models. A single shared prompt vector per relation type per layer is often insufficient to fully represent the intricate interactions between relation edges during fine-tuning. To endow the prompt for each relation edge with stronger expressiveness, we propose an enhanced version HGMRP+.

If a unique prompt is assigned to each individual relation edge rather than sharing one across edges of the same relation type within a layer, issues can arise in settings with limited downstream task labels. In such cases, many randomly initialized prompts may remain underfitted due to insufficient training signals, while overfitting could simultaneously occur on edges with more frequent labels. Moreover, such a design is only applicable in semi-supervised settings and fails when new nodes or new relation edges are introduced. Considering these challenges, inspired by GPF [Fang et al., 2023], we introduce an anchor prompt set for each relation type r:

where l denotes the layer index of the pre-trained model decoder, and K is the hyperparameter for the number of anchor prompts. The prompt for a relation edge in layer l is composed from its corresponding relational anchor prompt set, which is shared among all edges of the same relation type within that layer. Within the anchor prompt set, we design two categories of prompts: one is relation-specific prompts, exclusive to a single relation type within the layer; the other is relation-shared prompts, shared across multiple relation types within the same layer, aiming to capture common information between relations. The prompt for a relation edge in layer l is dynamically composed via an attention mechanism:

where α(l) r,k denotes the contribution weight of the k-th anchor prompt to the final prompt for relation r at layer l.

In order to compute α(l) r,k , the information of an edge is derived from its two connected nodes. Inspired by the attention mechanism in GAT [Velickovic et al., 2018], we calculate it through a combination of the two nodes' representations.

Given an edge connecting node vi and vj, we design a function of their representations:

where W(l) r ∈ RK×2dT is a learnable projection matrix for relation r at layer l, and K denotes the number of anchor prompts. Using the computed q, we obtain attention weights through softmax normalization, which are then used to aggregate the anchor prompt set and derive the actual prompt for the relation edge, which can be expressed as:

Thus, we obtain the prompt composition for each relation edge in the graph, and the message passing step in the decoder remains identical to that in HGMRP.

The above describes the construction and fusion process of prompts. The overall heterogeneous prompt-based finetuning stage can be summarized as:

where P and W correspond to the set of prompt anchors and the parameters in the attention function for prompt composition, respectively.

5 Experiments

To validate the practical effectiveness of our prompt-tuning approach, we conducted comprehensive experiments. In the following sections, we first introduce the experimental setup, including the datasets, various hyperparameters in our method, as well as the pre-trained models and existing fine-tuning methods used for comparison. We then present the experimental results, which address the following research questions (RQs):

• RQ1: Does our designed model enhance pre-trained models more effectively than existing heterogeneous prompt methods?
• RQ2: How does the number of anchor prompts, as a key hyperparameter in HGMRP+, influence model performance?
• RQ3: What is the impact of limited labeled data on the performance of our method?
• RQ4: Can our relation-edge-based method be successfully combined with existing heterogeneous prompt techniques to yield further performance improvements?

5.1 Experimental Setup

Datasets. We evaluate our approach on four heterogeneous graph benchmarks. The ACM dataset is constructed from ACM [Zhao et al., 2020] conference publications, comprising four node types: paper, author, term, and subject. Relations such as authorship and topic association connect these nodes, and the task is to predict the research area of each paper. DBLP [Fu et al., 2020b] is a scholarly collaboration network consisting of authors, papers, and venues, with relations capturing publication and affiliation patterns; here we perform author classification based on their publication profiles. Freebase [Lv et al., 2021] is a large-scale knowledge graph containing real-world entities with semantic relations, and we focus on classifying book entities. AMiner [Hu et al., 2019] is an academic network involving papers, authors, and venues, where links reflect writing and collaboration relations; the objective is to assign papers to corresponding research topics. The statistical information of the datasets is presented in Table 1.

Baselines. To demonstrate our model's fine-tuning capability in few-shot scenarios, we compare it against the following three types of approaches: 1) traditional semi-supervised heterogeneous representation learning methods, such as HAN [Wang et al., 2019] and HGT [Hu et al., 2020], to illustrate the effectiveness of heterogeneous graph pre-training; 2) widely-used heterogeneous graph pre-training methods, such as HeCo [Wang et al., 2021], HGMAE [Tian et al., 2023], and HDMI [Jing et al., 2021], to highlight the advantage of our fine-tuning strategy; and 3) existing heterogeneous graph prompt-tuning methods, such as HGPROMPT [Yu et al., 2024] and HetGPT [Ma et al., 2024], to verify the effectiveness of our relation-edge-based fine-tuning approach.

Parameter Settings. In this section, we provide a comprehensive description of the parameter settings used in the experiments. To investigate the effectiveness under low-label scenarios, the main experiments adopt 10-shot and 20-shot settings, which refer to randomly selecting 10 or 20 labeled samples per class, respectively. For baseline models, we strictly follow the experimental setups reported in their original papers without modifications, and pre-trained models are also used as originally released without further adjustment. Three evaluation metrics are employed, including, MicroF1 (measures global classification accuracy by counting total true predictions across all classes), Macro-F1 (calculates the unweighted average of per-class F1 scores to treat each class equally), and AUC (evaluates the model's ranking capability, typically used in binary or multi-class classification tasks to measure separability between classes). Our prompt-tuning method runs five times with random node selections, and the mean and variance are reported.

Our main hyperparameters include the number of anchor prompts per relation in each layer. In the primary experiments, we set the total number of anchor prompts to 10, consisting of 8 relation-specific anchor prompts exclusive to each relation type and 2 relation-shared anchor prompts that are common across different relation types within the same layer. This key parameter will be further analyzed in subsequent ablation studies. During training, we adopt the same learning rate as in the original pre-training stage for fine-tuning the classifier, and increase the number of training epochs to 200 with early stopping enabled. The Adam optimizer (PyTorch implementation) is adopted for parameter optimization. All experiments are conducted on an RTX 5090 GPU.

5.2 Node Classification (RQ1)

To investigate the capability of our prompt-tuning approach in few-shot settings, we conduct 10-shot and 20-shot experiments on four datasets. The results are summarized in Table 2. Based on the results, the following conclusions can be drawn: 1) Under both 10-shot and 20-shot settings, mainstream pre-training methods (HDMI, HeCo, HGMAE) consistently outperform traditional semi-supervised heterogeneous graph methods, demonstrating the effectiveness of pre-training. More importantly, our proposed relation-aware tuning approach achieves noticeable gains over existing prompt-tuning methods. This indicates that controlling message propagation through different relation edges is more effective in this experimental setup than simply adding prompts on nodes for propagation. 2) Our designed prompt mechanism can be seamlessly integrated into various mainstream pre-trained models and consistently yields strong performance. In contrast, HGPROMPT only reports results in a single setting due to its own proprietary framework, and HetGPT is restricted to the HeCo format. This demonstrates that our approach not only preserves performance but also significantly enhances the generalization ability of heterogeneous prompts across different pre-trained architectures.

5.3 Impact of Anchor Prompt Number (RQ2)

In our method, employing a single prompt per relation type per layer results in limited expressive capacity, while assigning a distinct prompt to every edge can lead to underfitting due to scarce labeled data. To balance these trade-offs, we introduce HGMRP+, in which the size of the anchor prompt set acts as a key hyperparameter. Setting this size to 1 reverts the model to HGMRP, and letting it approach the total number of edges mimics the case of a dedicated prompt per edge. Therefore, we design the following experiment to examine the impact of the number of anchor prompts. All experiments are conducted under a 20-shot setting. The results are shown in Figure 3. It can be observed that with limited labeled data, a configuration of 10 anchor prompts consistently outperforms the use of 20 prompts across different datasets and pre-trained models. This is because an excessively large set of anchor prompts leads to insufficient training. Thus, an appropriate number of prompts is essential to fully exploit the expressive power of the pre-trained model.

5.4 Performance with Different Shots (RQ3)

The previous experiments fixed the amount of labeled data during fine-tuning. To analyze the impact of varying sample sizes on model performance, we conduct experiments with different numbers of shots. We also examine the scenario in HGMRP where anchor prompts are not shared across different relations within the same layer, denoted in the figure as HGMRP w/o shared. The experimental results are presented in Figure 4. Based on the above results, it is observed that across varying label settings, our method consistently outperforms existing prompt techniques. Furthermore, the designed shared anchor prompts demonstrate superior effectiveness compared to the non-shared version, confirming the value of sharing anchor prompts across different semantic relations. Even in the extreme low-data scenario of 1-shot, where performance is understandably lower due to insufficient prompt training, our approach still achieves better results than the original pre-training method.

5.5 Integration with Other HG Prompt (RQ4)

Our method is grounded in multi-relational propagation on heterogeneous graphs, which does not conflict in application with existing prompts designed for node representation

#shot

#shot

Figure 4: Impact of shot number

in such graphs. To investigate the practical effect of using both types jointly, we integrate our approach with HetGPT and HGPROMPT, the former employs HeCo as the pretraining task, while the latter uses its own pre-training framework. The results are shown in Table 3, demonstrating that in heterogeneous graphs, our message-passing-based method is compatible with node-level representation and featureoriented prompts, and further enhances the performance of the original methods.

6 Conclusion

In this paper, we propose a relation-edge prompt termed HGMRP. It addresses the lack of a unified design in existing heterogeneous prompt methods, which predominantly operate on node features or representations and may lead to inconsistent optimization objectives when propagating across neighboring relations. An enhanced version, HGMRP+, is further introduced. Our approach is compatible with mainstream pre-trained heterogeneous graph models. Extensive experimental results demonstrate that our method outperforms existing heterogeneous graph prompting techniques, confirming the strong fine-tuning capability of relation-edge-level prompts. While the proposed method achieves strong results in single-domain settings, future work will explore cross-domain scenarios with diverse relational semantics and investigate serializing such relation-edge prompts for integration with large language models.

Acknowledgments

This work was supported by the National Key Research and Development Program of China (2023YFC3304503), the National Natural Science Foundation of China (No. 62422210, 92370111, 62302333, 62272340, 62276187).