PACIA: Parameter-Efficient Adapter for Few-Shot Molecular Property Prediction

1 Introduction

Molecular property prediction (MPP) which predicts whether desired properties will be active on given molecules, can be naturally modeled as a few-shot learning problem [Waring et al., 2015; Altae-Tran et al., 2017]. As wet-lab experiments to evaluate the actual properties of molecules are expensive and risky, usually only a few labeled molecules are available for a specific property. While recently, Graph Neural Network (GNN) is popularly used to learn molecular representations [Xu et al., 2019; Yang et al., 2019; Xiong et al., 2019]. Modeling molecules as graphs, GNN can capture inherent structural information. Hence, GNN-based methods obtain better performance than classical ones [Unterthiner et al., 2014; Ma et al., 2015], especially when they are pretrained on self-learning tasks constructed from additional large scale corpus. As for tasks with only a few labeled molecules, the performance of existing GNN-based methods is still far from desired.

∗Corresponding author.

Various few-shot learning (FSL) methods have been developed to handle few-shot MPP problem. The earlier work IterRefLSTM [Altae-Tran et al., 2017] builds a metric-based model upon matching network [Vinyals et al., 2016]. Subsequent works mainly adopt gradient-based meta-learning strategy [Finn et al., 2017], which learns parameter initialization with good generalizability across different properties and adapts parameters by gradient descent for target property. Specifically, Meta-MGNN [Guo et al., 2021] brings chemical prior knowledge in the form of molecular reconstruction loss, and optimizes all parameters by gradient descents. PAR [Wang et al., 2021] introduces attention and relation graph module to better utilize the labeled samples for property-adaptation with the awareness of target chemical property, and conducts a selective gradient-based metalearning strategy. ADKF-IFT [Chen et al., 2022] takes a gradient-based meta-learning strategy with implicit function theorem to avoid computing expensive hypergradients, and builds a Gaussian Process for each task as classifier. There are also works that bring auxiliary information such as additional reference molecules from large molecule database [Schimunek et al., 2023] and auxiliary properties [Zhuang et al., 2023] to improve the performance of few-shot MPP [Schimunek et al., 2023; Zhuang et al., 2023].

Two primary issues persist in existing studies. First, gradient-based meta-learning strategy requires updating a large number of parameters in order to adapt to each task. This results in poor learning efficiency and is prone to overfit given insufficient labeled samples [Rajeswaran et al., 2019; Yin et al., 2020], as demonstrated in Figure 1 (a). While with more task-specific parameters, the model gets easily overfit, which would be more severe in extreme few-shot cases. Effective adaptation should be made in a parameter-efficient way, i.e, without modulating a large amount of parameters. Second, query-level adaptation is absent but it is important specially for few-shot MPP. The chemical space is enormous and the representations of molecules vary in a wide range. The query-level difference should be addressed when classifying the encoded molecules. When query molecules are more similar to the labeled molecules in one class, they can be easily classified. While others exhibit comparable similarity to both categories, they will be harder to be accurately classified. Thus, a fixed predictor can not fit all molecules even in a single task.

In this paper, we first summarize existing works into an encoder-predictor framework, where GNN performs well acting as both encoder and predictor. Upon the framework, we propose PACIA, a PArameter-effiCIent Adapter for few-shot MPP problem. To sum up, our contributions are as follows:

• We propose query-level adaptation for few-shot MPP problem and design a hierarchical adaptation mechanism for the encoder-predictor framework generally adopted by existing approaches.
• We design a hypernetwork-based GNN adapter to achieve parameter-efficient adaptation. This unified GNN adapter can generate a few adaptive parameters to modulate the message passing process of GNN in two aspects: node embedding and propagation depth.
• We conduct extensive results, and show PACIA obtains the state-of-the-art performance on benchmark few-shot MPP datasets MoleculeNet [Wu et al., 2018] and FS-Mol [Stanley et al., 2021]. We also closely examine and validate the effectiveness of our hierarchical adaptation mechanism.

2 RelatedWorks

2.1 Few-Shot Learning

Few-shot learning (FSL) aims to generalize to a task with a few labeled samples [Wang et al., 2020]. In terms of adaptation mechanism, existing FSL methods can be classified into three main categories: (i) gradient-based approaches [Finn et al., 2017; Grant et al., 2018] learn a model which can be generalized to new task by gradient descents, (ii) metric-based approaches [Vinyals et al., 2016; Snell et al., 2017] learn to embed samples into a space where similar and dissimilar samples can be easily discriminated by a distance function, and (iii) amortization-based approaches [Requeima et al., 2019; Lin et al., 2021; Przewiezlikowski et al., 2022] use hypernetworks to map the labeled samples in the task to a few parameters to adjust the main networks to be task-specific. Recent works [Requeima et al., 2019] found that amortization-based approaches can reduce the risk of overfitting compared with gradient-based approaches. They also have faster inference speed as the adapted parameters are generated by a single forward pass without taking optimization steps. Besides, the main networks can approximate various functions in addition to distance-based ones.

2.2 Hypernetworks

Hypernetworks [Ha et al., 2017] refer to neural networks which learn to generate parameters of the main network which handles the target tasks. Hypernetworks have been successfully used in various applications like cold-start recommendation [Lin et al., 2021] and image classification [Przewiezlikowski et al., 2022]. Designing appropriate hypernetworks is challenging, requiring domain knowledge to decide what information to be fed into hypernetworks, how to adapt the main network, and what is the appropriate architecture of hypernetworks. For general GNNs, hypernetworks are developed to modulate weight matrix in aggregation function in message passing [Brockschmidt, 2020], or to facilitate node-specific message passing [Nachmani and Wolf, 2020]. In contrast to them, we particularly consider designing parameter-efficient modulators for GNNs used in encoder-predictor framework for few-shot MPP.

3 Preliminaries: Few-Shot MPP

3.1 Problem Setup

In a few-shot MPP task Tτ with respect to a specific property, each sample Xτ,i is a molecular graph and its label yτ,i ∈ {0, 1} records whether the molecule is active or inactive on the target property. Only a few labeled samples are available in Tτ . Following earlier works [Altae-Tran et al., 2017; Stanley et al., 2021; Chen et al., 2022; Schimunek et al., 2023], we model a Tτ as a 2-way classification task Tτ , associating with a support set Sτ = {(Xτ,s, yτ,s)}Nτ s=1 containing labeled samples from active/inactive class, and a query set Qτ = {(Xτ,q, yτ,q)}Mτ q=1 containing Mτ samples whose labels are only used for evaluation. We consider both (i) balanced support sets, i.e., Sτ contains Nτ 2 samples per class which is consistent with the standard N-way K-shot FSL setting [Altae-Tran et al., 2017], and (ii) imbalanced support sets which exist in real-world applications [Stanley et al., 2021]. Our target is to learn a model from a set of tasks {Tτ}Nj=1 that can generalize to new task given the few-shot support set. Specifically, the target properties are different across tasks.

3.2 Encoder-Predictor Framework

In the past, molecules are encoded with certain properties (fingerprint vectors [Rogers and Hahn, 2010]) and fed to deep networks for prediction [Unterthiner et al., 2014; Ma et al., 2015]. While recently, GNNs are popularly taken as molecular encoders [Li et al., 2018; Yang et al., 2019; Xiong et al., 2019; Hu et al., 2019] due to their superior performance on learning from of topological data. In either case, existing works can be summarized within an encoderpredictor framework.

Consider a molecular graph X = {V, E} with node feature hv for each atom v ∈ V and edge feature bvu for each chemical bond evu ∈ E between atoms v, u. A GNN encoder maps X to molecular representation r which is a fixed-length vector. At the lth layer, GNN updates atom embedding hlv of v as

where H(v) contains neighbors of v. AGG(·) and UPD(·) are aggregation and updating functions respectively. After L layers, the query-level representation r for X is obtained as

where READOUT(·) function aggregates atom embeddings.

Then, a predictor f(·) assigns label for a query molecule Xτ,q given support molecules in Sτ :

The specific choice of f(·) is diverse, e.g., pair-wise similarity [Altae-Tran et al., 2017], multi-layer perceptron (MLP) [Guo et al., 2021; Wang et al., 2021] and Mahalanobis distance [Stanley et al., 2021]. Recently, GNN-based predictor which operates on relation graphs of molecules is found to effectively compensate for the lack of supervised information [Wang et al., 2021]. In particular, molecular representations are refined on relation graphs such that the similar molecules cluster closer. Initialize molecular representations as the output of the encoder, i.e., h0τ,i = rτ,i. Denote the set of Nτ +1 molecules as Rτ,q = (Xτ,q, yτ,q) ∪ Sτ , which contains all information to make prediction for the query molecule Xτ,q. The relation graph works by recurrently estimating the adjacency matrix and updating the molecular representations. At the lth layer, each element alij in the adjacent matrix Alτ,q of the relation graph is learned to represent pair-wise similarities between any two molecules in Rτ,q:

Then, each molecular representation is refined as

After L layers of refinement, hLτ,q and hLτ,s (in place of rτ,q and rτ,s) are fed to (3) to obtain final prediction ˆ yτ,q for Xτ .

4 Hierarchical Adaptation of Encoder-Predictor Framework

To generalize across different tasks with a few labeled molecules, existing works [Wang et al., 2021; Chen et al., 2022] usually conduct task-level adaptation by gradientbased meta-learning. (see Appendix). However, as discussed in Section 2, gradient-based meta-learning optimizes most parameters using a few labeled molecules, which is slow to optimize and easy to overfit. As for query-level adaptation, gradient is not accessible for each query molecule in testing since the label is unknown during training. Therefore, we turn to hypernetworks to achieve parameter-efficient adaptation. Task-level adaptation is achieved in encoder since the structural features in molecular graphs needs to be captured in a property-adaptive manner, while query-level adaptation is achieved in predictor based on the property-adaptive representations.

As discussed in Section 3.2, GNNs can effectively act as both encoder and predictor. Therefore, we propose PACIA (Figure 1), a method using a unified GNN adapter to generate a few adaptive parameters to hierarchically adapt the encoder at task-level and the predictor at query-level in a parameterefficient manner. In the sequel, we provide the details of the unified GNN adapter (Section 4.1), then describe how to learn the main networks (including encoder and predictor) with the unified GNN adapter by episodic training (Section 4.2). Finally, we present a comparative discussion of PACIA in relation to existing works (Section 4.3).

4.1 AUnified GNNAdapter

To adapt GNN's parameter-efficiently, we design a GNN adapter to modulate the node embedding and propagation depth, which are essential in message passing process.

Modulating Node Embedding. Denote the node embedding at the lth layer as hl , which can be atom embedding in encoder or molecular embedding in relation graph predictor. We obtain adapted embedding ˆhl as

where e(·) is an element-wise function, and γ l is adaptive parameter generated by the hypernetworks. This adapted embedding ˆhl is then fed to next layer of GNN.

Modulating Propagation Depth. Further, we manage to modulate the propagation depth, i.e., layer number l of a GNN. Controlling l is challenging since it is discrete. We achieve this by training a differentiable controller, which is a hypernetwork to generate a scalar pl corresponding to the lth layer. The value of pl estimates how likely the message passing should stop after the lth layer. Finally, assume there are L layers in total, the vector

represents the plausibility of choosing each layer. The hypernetwork is shared across all L layers. During meta-training, p is used as differentiable weight to modulate GNN layers. Specifically, after propagation through all L layers, the final node embedding ˜h is obtained as

where [p]l is the lth element of p.

Generating Adaptive Parameters. We generate adaptive parameters {γ l , pl }Ll=1 by hypernetworks. In particular, note that the generated adaptive parameter should be permutationinvariant to the order of input samples in Sτ . Therefore, we first calculate class prototypes rlτ,+ and rlτ,- of active class (+) and inactive class (-) for samples in Sτ by

where [·|·] means concatenating, S + τ and S - τ are the sets of active and inactive samples in Sτ , and yτ,s is the onehot encoding of label. Using rlτ,+ and rlτ,- allows subsequent steps to keep supervised information while being permutation-invariant.

Recall that task-level adaptation and query-level adaptation is achieved in encoder and predictor respectively. For tasklevel adaptation, we then map rlτ,+ and rlτ,- to γ lτ , plτ as

As for query-level adaptation, information comes from both Sτ and the specific query molecule Xτ,q. Likewise, we use class prototypes rlτ,+ and rlτ,- to keep permutation-invariant. We then generate γ lτ,q , plτ,q as

combining information in Xτ,q and Sτ . Note that parameters of these MLPs in hypernetworks are jointly meta-learned with the encoder and predictor.

4.2 Learning and Inference

Denote the collection of all model parameters in main network((1)-(5)) and hypernetwork ((9)-(11)) as Θ, excluding adaptive parameters. Our objective takes the form:

Lτ is the loss in task Tτ , yτ,q is one-hot ground-truth label vector and ˆ yτ,q is prediction obtained by (3).

Note that Θ is shared across all tasks. While the adaptive parameter {γ l }Ll=1 in (6) and {pl }Ll=1 in (7) are generated by hypernetworks. The size of adaptive parameter is far smaller than the main network. This realizes parameterefficient adaptation and mitigates the risk of overfitting.

Algorithm 1 Meta-training procedure of PACIA.

Input: meta-training task set T train;

• 1: initialize Θ randomly or use a pretrained one;
• 2: while not done do
• 3: for each task Tτ ∈ T train do
• 4: for l ∈ {1, 2, · · · , Lenc} do
• 5: + generate [γ lτ , plτ ] by (10);
• 6: modulate atom embedding hlv ← e(hlv , γ lτ );
• 7: * update atom embedding hlv by (1);
• 8: end for
• 9: obtain atom embedding after message passing hLenc v ← ∑Lenc l=1 [pτ ]lhlv and obtain molecular embeddings by (2);
• 10: end for
• 11: for each query (Xτ,q, yτ,q) ∈ Qτ do
• 12: for l ∈ {1, 2, · · · , Lrel} do
• 13: + generate [γ lτ,q , plτ,q ]s by (11);
• modulate molecular embedding hlτ,i ←
• 14: e(hlτ,iγ lτ,q );
• 15: * update molecular embedding by (4)-(5);
• 16: end for
• 17: obtain molecular embedding after message passing hLrel τ,i ← ∑Lrel l=1 [pτ,q]lhlτ,i ;
• 18: obtain prediction ˆyτ,q by (3);
• 19: end for
• 20: calculate loss by (12);
• 21: update Θ by gradient descent;
• 22: end while
• 23: return learned Θ∗ .

Algorithm 1 1 summarizes the training procedure of PACIA. As mentioned above, our unified GNN adapter can simultaneously modulate the node embedding and propagation depth, and be cascaded to adapt both encoder and predictor. During training, molecular graph Xτ,i is first processed by encoder (line 4-9). At each layer, adaptive parameters [γ lτ , plτ ] are obtained by (10) (line 5). Then, (6) modulates all atom embeddings hlv (line 6). After Lenc layers of message passing (1), (8) is applied before (2), to get property-adaptive molecular representations and initialize node embeddings h0τ,i = rτ,i in relation graph (line 9). Then in predictor, at each layer of GNN on relation graph, adaptive parameters [γ lτ,q , plτ,q ] are obtained with (11) (line 13) and (6) modulates all node embeddings hlτ,i (line 14). After Lrel layers of message passing by (4)(5), (8) is applied (line 17). The final prediction ˆyτ,q is obtained by (3).

Testing procedure is provided in Appendix. The process is similar. A noteworthy difference is the propagation depth is adapted by selecting the layer with maximal plausibility:

Only hl′ , rather than (8), is fed forward to the next module.

1 In Algorithm 1,"*" (resp. "+") indicates the step is executed by the main network (resp. hypernetwork).

4.3 Comparison with ExistingWorks

From the perspective of hypernetworks, the usage of hypernetworks for encoder is related to GNNFiLM [Brockschmidt, 2020], which considers a GNN as main network. It builds hypernetworks with target node as input to generate parameters of FiLM layers, to equip different nodes with different aggregation functions in the GNN. What and how to adapt are similar to ours, but it is different that the input of our hypernetworks for encoder is Sτ and how we encode a set of labeled graphs.

As for few-shot learning, some recent studies [Requeima et al., 2019; Lin et al., 2021] use hypernetworks to transform the support set into parameters that modulate the main network. The functionality of their hypernetworks is akin to that of our hypernetwork for the encoder. However, there is a distinct difference in the architecture of main networks: while their approaches employ convolutional neural networks and MLPs, our method uniquely modulates the message passing process of GNN. Furthermore, the application of hypernetworks for the predictor, aimed at adjusting the model architecture based on a query sample (without label information) and a set of labeled samples, has not yet been explored in the literature. A detailed comparison of PACIA w.r.t. existing few-shot MPP works is in Appendix.

5 Experiments

In this section, we evaluate the proposed PACIA 2 on fewshot MPP problems. We run all experiments with 10 random seeds, and report the mean and standard deviations (in the subscript bracket). Appendix provides more information of datasets, baselines, and implementation details.

5.1 Performance Comparison onMoleculeNet

We use Tox21 [National Center for Advancing Translational Sciences, 2017], SIDER [Kuhn et al., 2016], MUV [Rohrer and Baumann, 2009] and ToxCast [Richard et al., 2016] from MoleculeNet [Wu et al., 2018], which are commonly used to evaluate the performance on few-shot MPP [Altae-Tran et al.,

2 Code is available at https://github.com/LARS-research/PACIA.

2017; Wang et al., 2021]. We adopt the public data split provided by [Wang et al., 2021]. The support sets are balanced, each of them contains K labeled molecules per class, where K = 1 and K = 10 are considered. The performance is evaluated by ROC-AUC calculated on the query set of each meta-testing task and averaged across all meta-testing tasks.

We compare PACIA with the following baselines: 1) single-task method GNN-ST [Gilmer et al., 2017]; 2) multitask pretraining method GNN-MT [Corso et al., 2020; Gilmer et al., 2017]; 3) self-supervised pretraining method MAT [Maziarka et al., 2020]; 4) meta-learning methods, including Siamese [Koch et al., 2015], ProtoNet [Snell et al., 2017], MAML [Finn et al., 2017], EGNN [Kim et al., 2019]; and 5) methods proposed for few-shot MPP, including IterRefLSTM [Altae-Tran et al., 2017], PAR [Wang et al., 2021] and ADKF-IFT [Chen et al., 2022]. Note that MHNfs [Schimunek et al., 2023] is not included as it uses additional reference molecules from external datasets, which leads to unfair comparison. GS-META [Zhuang et al., 2023] has not been compared since that approach requires multiple properties of each molecule, which is not applicable when a molecule is only evaluated w.r.t. one property. Following earlier works [Guo et al., 2021; Wang et al., 2021], we use GIN [Xu et al., 2019] as encoder, which is trained from scratch.

Table 1 shows the results. Results of Siamese and IterRefLSTM are copied from [Altae-Tran et al., 2017] as their codes are unavailable, and their results on ToxCast are unknown. GNN-FiLM is a general GNN whose target is not few-shot MPP, which explains its bad performance. PACIA obtains the highest ROC-AUC scores on all cases except the 10-shot case on MUV, where ADKF-IFT outperforms the others by a large margin. This can be a special case where ADKF-IFT works well but may not be generalizable. Moreover, depending on the number of local-update steps of ADKF-IFT, PACIA is about 5 times faster than ADKF-IFT (both meta-training and inference time is about 1/5). In terms of average performance, PACIA significantly outperforms the second-best method ADKF-IFT by 3.25%. We also provide results obtained with a pretrained encoder in Appendix. Similar observations can be made: our PACIA with pretrained encoder (Pre-PACIA) performs the best, and its performance gain is more significant when fewer labeled samples are provided. Further, results in Appendix shows the performance comparison between PACIA and a fine-tuned GNN with varying support set size. This shows that PACIA nicely achieves its goal: handling few-shot MPP problem in a parameterefficient way.

5.2 Performance Comparison on FS-Mol

We also use FS-Mol [Stanley et al., 2021], a new benchmark consisting of a large number of diverse tasks for model pretraining and a set of few-shot tasks with imbalanced classes. We adopt the public data split [Stanley et al., 2021]. Each support set contains 64 labeled molecules, and can be imbalanced where the number of labeled molecules from active and inactive is not equal. All remaining molecules in the task form the query set. Testing tasks are divided into categories with support size 16 [Schimunek et al., 2023], which is close to real-world scenario. The performance is evaluated by ∆AUPRC (change in area under the precision-recall curve) w.r.t. a random classifier [Stanley et al., 2021], averaged across meta-testing tasks.

We use the same baselines that were applied to MoleculeNet. Table 2 shows the results. We find that PACIA performs the best. Besides, the time-efficiency of PACIA is much higher since its adaptation only needs a single forward pass. While IterRefLSTM and ADKF-IFT take multiple local-update steps, they are much slower to generalize.

5.3 Ablation Study

We consider various variants of PACIA, including (i) finetuning: using the same model structure and fine-tuning all parameters to adapt to each property without hypernetworks; (ii) w/o T: removing task-level adaptation, thus the GNN encoder will not be adapted by hypernetworks w.r.t. each property; and (iii) w/o Q: removing query-level adaptation, such that all molecules are processed by the same predictor.

Figure 2(a) provides performance comparison on Tox21. As shown, the performance gain of PACIA over "w/o Q" shows the necessity of query-level adaptation. The gap between PACIA and "w/o T" indicates the effect of adapting the model to be task-specific. One can also notice that without query-level adaptation, "w/o Q" still obtains better performance than gradient-based baselines like PAR, which indicates the advantage of designing the amortization-based hypernetwork. The poor performance of "fine-tuning" is possibly because of the overfitting caused by updating all parameters with only a few samples. In sum, every component of PACIA is important for achieving good performance.

Now that the effectiveness of task-level and query-level adaptation are validated, we further investigate modulation functions, i.e, modulating node embedding (N), modulating propagation depth (D), modulating both (ND), for encoder and predictor. There are 3 × 3 combinations, whose performance is reported in Figure 2(b). We find that only modulating node embedding in encoder while only modulating propagation depth in predictor obtains the best performance. As the GIN encoder has highly non-linearity across layers, truncation would lead to non-explainability and somehow perturb the black-box. While the operation of relation graph in predictor updates node embedding in a linear way (5), adapting the propagation depth is harmonious with message passing process.

Figure 3 shows the t-SNE visualization [Van der Maaten and Hinton, 2008] of molecular representations learned on a 10-shot support set and a query molecule with groundtruth label "active" in task SR-p53 from Tox21. As shown, molecular representations obtained without adaptation (Figure 3(a)) are mixed up, since the encoder has not been adapted to the target property of the task. Molecular representations being processed by our property-adaptive encoder (Figure 3(b)) becomes more distinguishable, indicating that adapting molecular representation in task-level takes effect. Molecular representations in Figure 3(c) and Figure 3(d) form clear clusters as we encourage similar molecules to be connected during relation graph refinement by (5). The difference is that molecular representations in Figure 3(c) are refined by the best propagation depth number for all tasks in 10-shot case, while molecular representations in Figure 3(d) are refined by 4 depths of propagation which are selected for the specific query molecule. As shown, we can conclude that our molecular-adaptive refinement steps help better separate molecules of different classes. Ablation study of configurations of hypernetworks is in Appendix.

5.4 Study of Hierarchical AdaptationMechanism

Task-Level Adaptation. In PACIA, parameter-efficient task-level adaptation is achieved by using hypernetworks to modulate the node embeddings during message passing. We compared this amortization-based adaptation with gradientbased adaptation in PAR which has similar main network with PACIA. We record their adaptation process, i.e., time required to process the support set and the test performance. Table 3 shows the results. PAR uses molecules in support set to take gradient steps, and updates all parameters in GNN. We record each of a maximum five steps, where we can find that it easily overfits as the testing ROC-AUC keeps dropping with more steps. The time consumption also grows. In contrast, PACIA processes molecules in support set by hypernetworks, which is much more efficient as only one single forward pass is needed. PACIA can obtain better performance due to the reduction of adaptive parameters, which also leads to better generalization and alleviates the risk of overfitting to a few shots. Table 3 and Figure 1(a) both indicate that the underlying overfitting problem can be mitigated by PACIA.

Query-Level Adaptation. Finally, we present a case study on query-level adaptation. More experimental results on validating the design of query-level adaptation are in Appendix. We use a 1-shot support set and 3 query molecules in task SRp53 of Tox21. In Figure 4(a), x1 and x0 are support molecules with different labels, q1, q2 and q3 are query molecules. As shown, classifying q1 and q3 is relatively easy and the propagation depth will be 1, while classifying q2 is hard and requires 4 depth of propagation. Considering the shared substructures (function groups), q1 and x1 are visually similar, q3 and x0 are visually similar. While both x1 and x0 share substructures with q2, it is hard to tell which of them is more similar to q2. Figure 4(a) provides the cosine similarity based on the molecule representations generated by Pre-GNN , which confirms our observation: q1 is much more similar to x1, q3 is much more similar to x0, and q2 shows large similarities with both samples. Intuitively, classifying q1 and q3 will be easier while q2 will be hard. In the dynamic propagation of PACIA, we find different depths are taken: 1 for both q1 and q3 while 4 for q2. PACIA achieves effective query-level adaptation by assigning more complex models for query molecules that are difficult to classify.

6 Conclusion

We propose PACIA to handle few-shot MPP in a parameterefficient manner. We investigate two key factors in few-shot molecular property prediction under the common encoderpredictor framework: adaptation-efficiency and query-level adaptation. Evidence shows that too much adaptive parameter would lead to overfitting, thus we design a parameterefficient GNN adapter, which can modulate node embedding and propagation depth of message passing of GNN in a unified way. We also notice the importance of capturing querylevel difference and therefore propose hierarchical adaptation mechanism, which is achieved by using a unified GNN adapter in both encoder and predictor. Empirical results show that PACIA achieves the best performance on both MoleculeNet and FS-Mol.

Acknowledgments

1. Yao is supported by research fund of National Natural Science Foundation of China (No. 92270106), and Independent Research Plan of the Department of Electronic Engineering Department at Tsinghua University.