{"type":"ScholarlyArticle","title":"PACIA: Parameter-Efficient Adapter for Few-Shot Molecular Property Prediction","authors":[{"name":"Shiguang Wu","slug":"shiguang-wu","given":"Shiguang","family":"Wu","affiliation":"Department of Electronic Engineering , Tsinghua University"},{"name":"Yaqing Wang","slug":"yaqing-wang","given":"Yaqing","family":"Wang","affiliation":"Baidu Research , Baidu Inc."},{"name":"Quanming Yao","slug":"quanming-yao","given":"Quanming","family":"Yao","affiliation":"Department of Electronic Engineering , Tsinghua University"}],"abstract":"Molecular property prediction (MPP) plays a crucial role in biomedical applications, but it often encounters challenges due to a scarcity of labeled data. Existing works commonly adopt gradient-based strategy to update a large amount of parameters for task-level adaptation.  However, the increase of adaptive parameters can lead to overfitting and poor performance. Observing that graph neural network (GNN) performs well as both encoder and predictor, we propose PACIA, a parameter-efficient GNN adapter for few-shot MPP. We design a unified adapter to generate a few adaptive parameters to modulate the message passing process of GNN. We then adopt a hierarchical adaptation mechanism to adapt the encoder at task-level and the predictor at query-level by the unified GNN adapter. Extensive results show that PACIA obtains the state-of-the-art performance in few-shot MPP problems, and our proposed hierarchical adaptation mechanism is rational and effective.","doi":"10.24963/ijcai.2024/576","url":"https://ijcai.org/proceedings/2024/576","pdf":"https://ijcai.org/proceedings/2024/0576.pdf","markdown":"https://ijcai.org/proceedings/2024/576.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/576","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":576,"pages":{"first":5208,"last":5216},"keywords":["Machine Learning: ML: Few-shot learning","Machine Learning: ML: Applications"],"videos":[],"references":[{"position":1,"raw":"[Altae-Tran et al., 2017] Han Altae-Tran, Bharath Ramsun- dar, Aneesh S Pappu, and Vijay Pande. Low data drug discovery with one-shot learning. ACS Central Science, 3(4):283-293, 2017.","title":"Han Altae-Tran, Bharath Ramsundar, Aneesh S Pappu, and Vijay Pande. Low data drug discovery with one-shot learning","authors":"Altae-Tran","year":"2017","venue":"ACS Central Science","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brockschmidt, 2020] Marc Brockschmidt. GNN-FiLM: Graph neural networks with feature-wise linear modula- tion. In International Conference on Machine Learning, pages 1144-1152, 2020.","title":"GNN-FiLM: Graph neural networks with feature-wise linear modulation","authors":"Marc Brockschmidt","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2022] Wenlin Chen, Austin Tripp, and José Miguel Hernández-Lobato. Meta-learning adaptive deep kernel gaussian processes for molecular property prediction. In International Conference on Learning Representations, 2022.","title":"Meta-learning adaptive deep kernel gaussian processes for molecular property prediction","authors":"Chen","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Corso et al., 2020] Gabriele Corso, Luca Cavalleri, Do- minique Beaini, Pietro Liò, and Petar Veličković. Prin- cipal neighbourhood aggregation for graph nets. In Ad- vances in Neural Information Processing Systems, pages 13260-13271, 2020.","title":"Principal neighbourhood aggregation for graph nets","authors":"Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, Petar Veličković","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pages 1126-1135, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neu- ral message passing for quantum chemistry. In Inter- national Conference on Machine Learning, pages 1263- 1272, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, S Samuel, Patrick F Schoenholz, Oriol Riley, George E Vinyals, Dahl","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Grant et al., 2018] Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient-based meta-learning as hierarchical Bayes. In International Conference on Learning Representations, 2018.","title":"Recasting gradient-based meta-learning as hierarchical Bayes","authors":"Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, Thomas Griffiths","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2021] Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, and Nitesh V Chawla. Few-shot graph learning for molecular property predic- tion. In The Web Conference, pages 2559-2567, 2021.","title":"Few-shot graph learning for molecular property prediction","authors":"Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, Nitesh V Chawla","year":"2021","venue":"The Web Conference","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ha et al., 2017] David Ha, Andrew Dai, and Quoc V Le. Hypernetworks. In International Conference on Learning Representations, 2017.","title":"Hypernetworks","authors":"David Ha, Andrew Dai, Quoc V Le","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2019] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural net- works. In International Conference on Learning Repre- sentations, 2019.","title":"Strategies for pre-training graph neural networks","authors":"Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, Jure Leskovec","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kim et al., 2019] Jongmin Kim, Taesup Kim, Sungwoong Kim, and Chang D Yoo. Edge-labeling graph neural net- work for few-shot learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11-20, 2019.","title":"Edge-labeling graph neural network for few-shot learning","authors":"Jongmin Kim, Taesup Kim, Sungwoong Kim, Chang D Yoo","year":"2019","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Koch et al., 2015] Gregory Koch, Richard Zemel, and Rus- lan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, 2015.","title":"Siamese neural networks for one-shot image recognition","authors":"Gregory Koch, Richard Zemel, Ruslan Salakhutdinov","year":"2015","venue":"ICML Deep Learning Workshop","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kuhn et al., 2016] Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. The SIDER database of drugs and side effects. Nucleic Acids Research, 44(D1):D1075- D1079, 2016.","title":"The SIDER database of drugs and side effects","authors":"Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, Peer Bork","year":"2016","venue":"Nucleic Acids Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2018] Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. Adaptive graph convolutional neural net- works. In AAAI Conference on Artificial Intelligence, pages 3546-3553, 2018.","title":"Adaptive graph convolutional neural networks","authors":"Ruoyu Li, Sheng Wang, Feiyun Zhu, Junzhou Huang","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2021] Xixun Lin, Jia Wu, Chuan Zhou, Shirui Pan, Yanan Cao, and Bin Wang. Task-adaptive neural pro- cess for user cold-start recommendation. In The Web Con- ference, pages 1306-1316, 2021.","title":"Task-adaptive neural process for user cold-start recommendation","authors":"Xixun Lin, Jia Wu, Chuan Zhou, Shirui Pan, Yanan Cao, Bin Wang","year":"2021","venue":"The Web Conference","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma et al., 2015] Junshui Ma, Robert P Sheridan, Andy Liaw, George E Dahl, and Vladimir Svetnik. Deep neu- ral nets as a method for quantitative structure-activity rela- tionships. Journal of Chemical Information and Modeling, 55(2):263-274, 2015.","title":"Deep neural nets as a method for quantitative structure-activity relationships","authors":"Ma","year":"2015","venue":"Journal of Chemical Information and Modeling","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Maziarka et al., 2020] Lukasz Maziarka, Tomasz Danel, Slawomir Mucha, Krzysztof Rataj, Jacek Tabor, and Stanislaw Jastrzebski. Molecule attention transformer. arXiv preprint arXiv:2002.08264, 2020.","title":"Molecule attention transformer","authors":"Lukasz Maziarka, Tomasz Danel, Slawomir Mucha, Krzysztof Rataj, Jacek Tabor, Stanislaw Jastrzebski","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nachmani and Wolf, 2020] Eliya Nachmani and Lior Wolf. Molecule property prediction and classification with graph hypernetworks. arXiv preprint arXiv:2002.00240, 2020. [National Center for Advancing Translational Sciences, 2017] National Center for Advancing Translational Sciences. Tox21 challenge. http://tripod.nih.gov/tox21/challenge/, 2017. Accessed: 2016-11-06.","title":"Molecule property prediction and classification with graph hypernetworks","authors":"Wolf ; Nachmani, Eliya Nachmani, Lior Wolf","year":"2016","venue":"Tox21 challenge","doi":null,"url":"http://tripod.nih.gov/tox21/challenge/,2017","internal_url":null},{"position":19,"raw":"[Przewiezlikowski et al., 2022] Marcin Przewiezlikowski, P Przybysz, Jacek Tabor, Maciej Zieba, and Przemyslaw Spurek. HyperMAML: Few-shot adaptation of deep mod- els with hypernetworks. arXiv preprint arXiv:2205.15745, 2022.","title":"HyperMAML: Few-shot adaptation of deep models with hypernetworks","authors":"Marcin Przewiezlikowski, Jacek Przybysz, Maciej Tabor, Przemyslaw Zieba, Spurek","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rajeswaran et al., 2019] Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta- learning with implicit gradients. In Advances in Neural Information Processing Systems, pages 113-124, 2019.","title":"Metalearning with implicit gradients","authors":"Aravind Rajeswaran, Chelsea Finn, Sergey Sham M Kakade, Levine","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Requeima et al., 2019] James Requeima, Jonathan Gordon, John Bronskill, Sebastian Nowozin, and Richard E Turner. Fast and flexible multi-task classification using conditional neural adaptive processes. In Advances in Neural Informa- tion Processing Systems, pages 7957-7968, 2019.","title":"Fast and flexible multi-task classification using conditional neural adaptive processes","authors":"James Requeima, Jonathan Gordon, John Bronskill, Sebastian Nowozin, Richard E Turner","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Richard et al., 2016] Ann M Richard, Richard S Judson, Keith A Houck, Christopher M Grulke, Patra Volarath, Inthirany Thillainadarajah, Chihae Yang, James Rathman, Matthew T Martin, John F Wambaugh, et al. ToxCast chemical landscape: Paving the road to 21st century tox- icology. Chemical Research in Toxicology, 29(8):1225- 1251, 2016.","title":"ToxCast chemical landscape: Paving the road to 21st century toxicology","authors":"Ann M Richard, Richard S Judson, Keith A Houck, Christopher M Grulke, Patra Volarath, Inthirany Thillainadarajah, Chihae Yang, James Rathman, John F Matthew T Martin, Wambaugh","year":"2016","venue":"Chemical Research in Toxicology","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rogers and Hahn, 2010] David Rogers and Mathew Hahn. Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 50(5):742-754, 2010.","title":"Extended-connectivity fingerprints","authors":"Rogers, Hahn, David Rogers, Mathew Hahn","year":"2010","venue":"Journal of Chemical Information and Modeling","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rohrer and Baumann, 2009] Sebastian G Rohrer and Knut Baumann. Maximum unbiased validation (MUV) data sets for virtual screening based on PubChem bioactivity data. Journal of Chemical Information and Modeling, 49(2):169-184, 2009.","title":"Maximum unbiased validation (MUV) data sets for virtual screening based on PubChem bioactivity data","authors":"G Sebastian, Knut Rohrer, Baumann","year":"2009","venue":"Journal of Chemical Information and Modeling","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Schimunek et al., 2023] Johannes Schimunek, Philipp Seidl, Lukas Friedrich, Daniel Kuhn, Friedrich Rippmann, Sepp Hochreiter, and Günter Klambauer. Context- enriched molecule representations improve few-shot drug discovery. arXiv preprint arXiv:2305.09481, 2023.","title":"Contextenriched molecule representations improve few-shot drug discovery","authors":"Johannes Schimunek, Philipp Seidl, Lukas Friedrich, Daniel Kuhn, Friedrich Rippmann, Sepp Hochreiter, Günter Klambauer","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Ad- vances in Neural Information Processing Systems, pages 4080-4090, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Stanley et al., 2021] Megan Stanley, John F Bronskill, Krzysztof Maziarz, Hubert Misztela, Jessica Lanini, Mar- win Segler, Nadine Schneider, and Marc Brockschmidt. FS-Mol: A few-shot learning dataset of molecules. In Neu- ral Information Processing Systems Track on Datasets and Benchmarks, 2021.","title":"FS-Mol: A few-shot learning dataset of molecules","authors":"Megan Stanley, John F Bronskill, Krzysztof Maziarz, Hubert Misztela, Jessica Lanini, Marwin Segler, Nadine Schneider, Marc Brockschmidt","year":"2021","venue":"Neural Information Processing Systems Track on Datasets and Benchmarks","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Unterthiner et al., 2014] Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Marvin Steijaert, Jörg K Wegner, Hugo Ceulemans, and Sepp Hochreiter. Deep learning as an opportunity in virtual screening. In NIPS Deep Learning Workshop, volume 27, pages 1-9, 2014.","title":"Deep learning as an opportunity in virtual screening","authors":"Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Marvin Steijaert, K Jörg, Hugo Wegner, Sepp Ceulemans, Hochreiter","year":"2014","venue":"NIPS Deep Learning Workshop","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Jour- nal of Machine Learning Research, 9(11), 2008.","title":"Visualizing data using t-SNE","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim- othy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, pages 3630-3638, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2020] Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. Generalizing from a few ex- amples: A survey on few-shot learning. ACM Computing Surveys, 53(3):1-34, 2020.","title":"Generalizing from a few examples: A survey on few-shot learning","authors":"Wang","year":"2020","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2021] Yaqing Wang, Abulikemu Abuduweili, Quanming Yao, and Dejing Dou. Property-aware rela- tion networks for few-shot molecular property prediction. In Advances in Neural Information Processing Systems, pages 17441-17454, 2021.","title":"Property-aware relation networks for few-shot molecular property prediction","authors":"Wang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Waring et al., 2015] Michael J Waring, John Arrowsmith, Andrew R Leach, Paul D Leeson, Sam Mandrell, Robert M Owen, Garry Pairaudeau, William D Pennie, Stephen D Pickett, Jibo Wang, et al. An analysis of the at- trition of drug candidates from four major pharmaceutical companies. Nature Reviews Drug discovery, 14(7):475- 486, 2015.","title":"An analysis of the attrition of drug candidates from four major pharmaceutical companies","authors":"Waring Michael, John Arrowsmith, Andrew R Leach, Paul D Leeson, Sam Mandrell, Garry Robert M Owen, Pairaudeau, Stephen D William D Pennie, Jibo Pickett, Wang","year":"2015","venue":"Nature Reviews Drug discovery","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wu et al., 2018] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: A benchmark for molecular machine learning. Chemical Sci- ence, 9(2):513-530, 2018.","title":"MoleculeNet: A benchmark for molecular machine learning","authors":"Wu","year":"2018","venue":"Chemical Science","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xiong et al., 2019] Zhaoping Xiong, Dingyan Wang, Xi- aohong Liu, Feisheng Zhong, Xiaozhe Wan, Xutong Li, Zhaojun Li, Xiaomin Luo, Kaixian Chen, Hualiang Jiang, et al. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of Medicinal Chemistry, 63(16):8749-8760, 2019.","title":"Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism","authors":"Xiong","year":"2019","venue":"Journal of Medicinal Chemistry","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xu et al., 2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural net- works? In International Conference on Learning Rep- resentations, 2019.","title":"How powerful are graph neural networks","authors":"Xu","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2019] Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, et al. Analyzing learned molecular representa- tions for property prediction. Journal of Chemical Infor- mation and Modeling, 59(8):3370-3388, 2019.","title":"Analyzing learned molecular representations for property prediction","authors":"Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea","year":"2019","venue":"Journal of Chemical Information and Modeling","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yin et al., 2020] Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, and Chelsea Finn. Meta-learning without memorization. In International Conference on Learning Representations, 2020.","title":"Meta-learning without memorization","authors":"Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, Chelsea Finn","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhuang et al., 2023] Xiang Zhuang, Qiang Zhang, Bin Wu, Keyan Ding, Yin Fang, and Huajun Chen. Graph sampling-based meta-learning for molecular property pre- diction. arXiv preprint arXiv:2306.16780, 2023.","title":"Graph sampling-based meta-learning for molecular property prediction","authors":"Xiang Zhuang, Qiang Zhang, Bin Wu, Keyan Ding, Yin Fang, Huajun Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2023/526"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}