{"type":"ScholarlyArticle","title":"A Systematic Survey of Chemical Pre-trained Models","authors":[{"name":"Jun Xia","slug":"jun-xia","given":"Jun","family":"Xia","affiliation":"Research Center for Industries of the Future , Westlake University"},{"name":"Yanqiao Zhu","slug":"yanqiao-zhu","given":"Yanqiao","family":"Zhu","affiliation":"University of California , Los Angeles"},{"name":"Yuanqi Du","slug":"yuanqi-du","given":"Yuanqi","family":"Du","affiliation":"Cornell University"},{"name":"Stan Z. Li","slug":"stan-z-li","given":"Stan Z.","family":"Li","affiliation":"Research Center for Industries of the Future , Westlake University"}],"abstract":"Deep learning has achieved remarkable success in learning representations for molecules, which is crucial for various biochemical applications, ranging from property prediction to drug design. However, training Deep Neural Networks (DNNs) from scratch often requires abundant labeled molecules, which are expensive to acquire in the real world. To alleviate this issue, tremendous efforts have been devoted to Chemical Pre-trained Models (CPMs), where DNNs are pre-trained using large-scale unlabeled molecular databases and then fine-tuned over specific downstream tasks. Despite the prosperity, there lacks a systematic review of this fast-growing field. In this paper, we present the first survey that summarizes the current progress of CPMs. We first highlight the limitations of training molecular representation models from scratch to motivate CPM studies. Next, we systematically review recent advances on this topic from several key perspectives, including molecular descriptors, encoder architectures, pre-training strategies, and applications. We also highlight the challenges and promising avenues for future research, providing a useful resource for both machine learning and scientific communities.","doi":"10.24963/ijcai.2023/760","url":"https://ijcai.org/proceedings/2023/760","pdf":"https://ijcai.org/proceedings/2023/0760.pdf","markdown":"https://ijcai.org/proceedings/2023/760.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/760","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Survey Track","track":"Survey Track","number":760,"pages":{"first":6787,"last":6795},"keywords":["Survey: Machine Learning","Survey: Multidisciplinary Topics and Applications","Survey: Data Mining","Survey: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Axelrod and Gómez-Bombarelli, 2022] S. Axelrod and R. Gómez- Bombarelli. GEOM, Energy-Annotated Molecular Conforma- tions for Property Prediction and Molecular Generation. Sci. Data, 2022.","title":"Energy-Annotated Molecular Conformations for Property Prediction and Molecular Generation","authors":"S Axelrod, R Gómez-Bombarelli, Geom","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bagal et al., 2021] V. Bagal, R. Aggarwal, et al. MolGPT: Molec- ular Generation Using a Transformer-Decoder Model. J. Chem. Inf. Model., 2021.","title":"MolGPT: Molecular Generation Using a Transformer-Decoder Model","authors":"V Bagal, R Aggarwal","year":"2021","venue":"J. Chem. Inf. Model","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berman et al., 2000] H. M. Berman, J. Westbrook, et al. The Pro- tein Data Bank. Nucleic Acids Res., 2000.","title":"The Protein Data Bank","authors":"H M Berman, J Westbrook","year":"2000","venue":"Nucleic Acids Res","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown et al., 2020] T. B. Brown, B. Mann, et al. Language Models are Few-Shot Learners. In NeurIPS, 2020.","title":"Language Models are Few-Shot Learners","authors":"T B Brown, B Mann","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chithrananda et al., 2020] S. Chithrananda, G. Grand, et al. Chem- BERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction. arXiv.org, October 2020.","title":"Chem-BERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction","authors":"S Chithrananda, G Grand","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Consonni and Todeschini, 2009] V. Consonni and R. Todeschini. Molecular Descriptors for Chemoinformatics. 2009.","title":"Molecular Descriptors for Chemoinformatics","authors":"V Consonni, R Todeschini","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Corso et al., 2020] G. Corso, L. Cavalleri, et al. Principal Neigh- bourhood Aggregation for Graph Nets. In NeurIPS, 2020.","title":"Principal Neighbourhood Aggregation for Graph Nets","authors":"G Corso, L Cavalleri","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Degen et al., 2008] J. Degen, C. Wegscheid-Gerlach, et al. On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces. ChemMedChem, 2008.","title":"On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces","authors":"J Degen, C Wegscheid-Gerlach","year":"2008","venue":"ChemMedChem","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Devlin et al., 2019] J. Devlin, M. Chang, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL, 2019.","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","authors":"J Devlin, M Chang","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Du et al., 2022a] W. Du, H. Zhang, et al. SE(3) Equivariant Graph Neural Networks with Complete Local Frames. In ICML, 2022.","title":"3) Equivariant Graph Neural Networks with Complete Local Frames","authors":"Du","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Du et al., 2022b] Y. Du, T. Fu, et al. MolGenSurvey: A System- atic Survey in Machine Learning Models for Molecule Design. arXiv.org, March 2022.","title":"MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design","authors":"Du","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Edwards et al., 2022] C. Edwards, T. Lai, et al. Translation be- tween Molecules and Natural Language. In EMNLP, 2022.","title":"Translation between Molecules and Natural Language","authors":"C Edwards, T Lai","year":"2022","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Fang et al., 2022a] X. Fang, L. Liu, et al. Geometry-Enhanced Molecular Representation Learning for Property Prediction. Nat. Mach. Intell., 2022.","title":"Geometry-Enhanced Molecular Representation Learning for Property Prediction","authors":"Fang","year":"2022","venue":"Nat. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Fang et al., 2022b] Y. Fang, Q. Zhang, et al. Molecular Contrastive Learning with Chemical Element Knowledge Graph. In AAAI, 2022.","title":"Molecular Contrastive Learning with Chemical Element Knowledge Graph","authors":"Fang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Feng et al., 2022] J. Feng, Z. Wang, et al. MGMAE: Molecu- lar Representation Learning by Reconstructing Heterogeneous Graphs with A High Mask Ratio. In CIKM, 2022.","title":"MGMAE: Molecular Representation Learning by Reconstructing Heterogeneous Graphs with A High Mask Ratio","authors":"J Feng, Z Wang","year":"2022","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gaulton et al., 2012] A. Gaulton, L. J. Bellis, et al. ChEMBL: A Large-Scale Bioactivity Database for Drug Discovery. Nucleic Acids Res., 2012.","title":"ChEMBL: A Large-Scale Bioactivity Database for Drug Discovery","authors":"A Gaulton, L J Bellis","year":"2012","venue":"Nucleic Acids Res","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Guo et al., 2022] Z. Guo, P. K. Sharma, et al. Multilingual Molec- ular Representation Learning via Contrastive Pre-training. In ACL, 2022.","title":"Multilingual Molecular Representation Learning via Contrastive Pre-training","authors":"Z Guo, P K Sharma","year":"2022","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hassani and Ahmadi, 2020] K. Hassani and A. H. K. Ahmadi. Contrastive Multi-View Representation Learning on Graphs. In ICML, 2020.","title":"Contrastive Multi-View Representation Learning on Graphs","authors":"K Hassani, A H K Ahmadi","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hawkins, 2017] P. C. D. Hawkins. Conformation Generation: The State of the Art. J. Chem. Inf. Model., 2017.","title":"Conformation Generation: The State of the Art","authors":"P C D Hawkins","year":"2017","venue":"J. Chem. Inf. Model","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[He et al., 2016] K. He, X. Zhang, et al. Deep Residual Learning for Image Recognition. In CVPR, 2016.","title":"Deep Residual Learning for Image Recognition","authors":"K He, X Zhang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Hjelm et al., 2019] R. D. Hjelm, A. Fedorov, et al. Learning Deep Representations by Mutual Information Estimation and Maxi- mization. In ICLR, 2019.","title":"Learning Deep Representations by Mutual Information Estimation and Maximization","authors":"R D Hjelm, A Fedorov","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ho et al., 2020] J. Ho, A. Jain, et al. Denoising Diffusion Proba- bilistic Models. In NeurIPS, 2020.","title":"Denoising Diffusion Probabilistic Models","authors":"J Ho, A Jain","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Honda et al., 2019] S. Honda, S. Shi, et al. SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery. arXiv.org, November 2019.","title":"SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery","authors":"S Honda, S Shi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Hou et al., 2022] Z. Hou, X. Liu, et al. GraphMAE: Self- Supervised Masked Graph Autoencoders. In KDD, 2022.","title":"GraphMAE: Self-Supervised Masked Graph Autoencoders","authors":"Z Hou, X Liu","year":"2022","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Hu et al., 2020a] W. Hu, B. Liu, et al. Strategies for Pre-training Graph Neural Networks. In ICLR, 2020.","title":"Strategies for Pre-training Graph Neural Networks","authors":"Hu","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Hu et al., 2020b] Z. Hu, Y. Dong, et al. GPT-GNN: Generative Pre-Training of Graph Neural Networks. In KDD, 2020.","title":"GPT-GNN: Generative Pre-Training of Graph Neural Networks","authors":"Hu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Hu et al., 2021] W. Hu, M. Fey, et al. OGB-LSC: A large-scale challenge for machine learning on graphs. In NeurIPS Datasets and Benchmarks, 2021.","title":"OGB-LSC: A large-scale challenge for machine learning on graphs","authors":"W Hu, M Fey","year":"2021","venue":"NeurIPS Datasets and Benchmarks","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Hu et al., 2022] B. Hu, J. Xia, J. Zheng, C. Tan, Y. Huang, Y. Xu, and S. Z. Li. Protein language models and structure prediction: Connection and progression. arXiv preprint arXiv:2211.16742, 2022.","title":"Protein language models and structure prediction: Connection and progression","authors":"B Hu, J Xia, J Zheng, C Tan, Y Huang, Y Xu, S Z Li","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Huang et al., 2021] K. Huang, T. Fu, et al. Therapeutics Data Com- mons: Machine Learning Datasets and Tasks for Drug Discovery and Development. In NeurIPS Datasets and Benchmarks, 2021.","title":"Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development","authors":"K Huang, T Fu","year":"2021","venue":"NeurIPS Datasets and Benchmarks","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Jiao et al., 2023] R. Jiao, J. Han, et al. Energy-Motivated Equiv- ariant Pretraining for 3D Molecular Graphs. In AAAI, 2023.","title":"Energy-Motivated Equivariant Pretraining for 3D Molecular Graphs","authors":"R Jiao, J Han","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Kearnes et al., 2016] S. Kearnes, K. McCloskey, et al. Molecular Graph Convolutions: Moving Beyond Fingerprints. J. Comput. Aided Mol. Des., 2016.","title":"Molecular Graph Convolutions: Moving Beyond Fingerprints","authors":"S Kearnes, K Mccloskey","year":"2016","venue":"J. Comput. Aided Mol. Des","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Kim et al., 2022] D. Kim, J. Baek, et al. Graph Self-supervised Learning with Accurate Discrepancy Learning. In NeurIPS, 2022.","title":"Graph Self-supervised Learning with Accurate Discrepancy Learning","authors":"D Kim, J Baek","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Kipf and Welling, 2017] N. T. Kipf and M. Welling. Semi- Supervised Classification with Graph Convolutional Networks. In ICLR, 2017.","title":"Semi-Supervised Classification with Graph Convolutional Networks","authors":"N T Kipf, M Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Krenn et al., 2020] M. Krenn, F. Häse, et al. Self-Referencing Embedded Strings (SELFIES): A 100% Robust Molecular String Representation. Mach. Learn. Sci. Technol., 2020.","title":"Self-Referencing Embedded Strings (SELFIES): A 100% Robust Molecular String Representation","authors":"M Krenn, F Häse","year":"2020","venue":"Mach. Learn. Sci. Technol","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Li et al., 2021] P. Li, J. Wang, et al. An Effective Self-Supervised Framework for Learning Expressive Molecular Global Represen- tations to Drug Discovery. Briefings Bioinform., 2021.","title":"An Effective Self-Supervised Framework for Learning Expressive Molecular Global Representations to Drug Discovery","authors":"P Li, J Wang","year":"2021","venue":"Briefings Bioinform","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Li et al., 2022a] H. Li, D. Zhao, et al. KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Pre- diction. In KDD, 2022.","title":"KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction","authors":"Li","year":"2022","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Li et al., 2022b] S. Li, J. Zhou, et al. GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction. In AAAI, 2022.","title":"GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction","authors":"Li","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Lin et al., 2022] X. Lin, C. Xu, et al. PanGu Drug Model: Learn a Molecule Like a Human. bioRxiv.org, April 2022.","title":"PanGu Drug Model: Learn a Molecule Like a Human","authors":"Lin","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Liu et al., 2022] S. Liu, H. Wang, et al. Pre-training Molecular Graph Representation with 3D Geometry. In ICLR, 2022.","title":"Pre-training Molecular Graph Representation with 3D Geometry","authors":"S Liu, H Wang","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Liu et al., 2023] S. Liu, H. Guo, et al. Molecular Geometry Pre- training with SE(3)-Invariant Denoising Distance Matching. In ICLR, 2023.","title":"Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching","authors":"S Liu, H Guo","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Nguyen et al., 2021] T. Nguyen, H. Le, et al. GraphDTA: Predict- ing Drug-Target Binding Affinity with Graph Neural Networks. Bioinform., 2021.","title":"GraphDTA: Predicting Drug-Target Binding Affinity with Graph Neural Networks","authors":"T Nguyen, H Le","year":"2021","venue":"Bioinform","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Pinheiro et al., 2022] G. A. Pinheiro, J. L. Da Silva, et al. SMI- CLR: Contrastive Learning on Multiple Molecular Representa- tions for Semisupervised and Unsupervised Representation Learn- ing. J. Chem. Inf. Model., 2022.","title":"SMI-CLR: Contrastive Learning on Multiple Molecular Representations for Semisupervised and Unsupervised Representation Learning","authors":"G A Pinheiro, J L Da Silva","year":"2022","venue":"J. Chem. Inf. Model","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Raffel et al., 2020] C. Raffel, N. Shazeer, et al. Exploring the Lim- its of Transfer Learning with a Unified Text-to-Text Transformer.","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","authors":"C Raffel, N Shazeer","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"J. Mach. Learn. Res., 2020.","title":"Learn. Res","authors":"J Mach","year":"2020","venue":"Learn. Res","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Rong et al., 2020] Y. Rong, Y. Bian, et al. Self-Supervised Graph Transformer on Large-Scale Molecular Data. In NeurIPS, 2020.","title":"Self-Supervised Graph Transformer on Large-Scale Molecular Data","authors":"Y Rong, Y Bian","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Ross et al., 2022] J. Ross, B. Belgodere, et al. Molformer: Large Scale Chemical Language Representations Capture Molecular Structure and Properties. Nat. Mach. Intell., 2022.","title":"Molformer: Large Scale Chemical Language Representations Capture Molecular Structure and Properties","authors":"J Ross, B Belgodere","year":"2022","venue":"Nat. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Sanchez-Gonzalez et al., 2020] A. Sanchez-Gonzalez, J. Godwin, et al. Learning to Simulate Complex Physics with Graph Net- works. In ICML, 2020.","title":"Learning to Simulate Complex Physics with Graph Networks","authors":"A Sanchez-Gonzalez, J Godwin","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Satorras et al., 2021] V. G. Satorras, E. Hoogeboom, et al. E(n) Equivariant Graph Neural Networks. In ICML, 2021.","title":"Equivariant Graph Neural Networks","authors":"V G Satorras, E Hoogeboom","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Schütt et al., 2017] K. Schütt, P.-J. Kindermans, et al. SchNet: A Continuous-Filter Convolutional Neural Network for Modeling Quantum Interactions. In NIPS, 2017.","title":"SchNet: A Continuous-Filter Convolutional Neural Network for Modeling Quantum Interactions","authors":"K Schütt, P.-J Kindermans","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Schütt et al., 2021] K. Schütt, O. T. Unke, et al. Equivariant mes- sage passing for the prediction of tensorial properties and molec- ular spectra. In ICML, 2021.","title":"Equivariant message passing for the prediction of tensorial properties and molecular spectra","authors":"K Schütt, O T Unke","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Stärk et al., 2022] H. Stärk, D. Beaini, et al. 3D Infomax Improves GNNs for Molecular Property Prediction. In ICML, 2022.","title":"3D Infomax Improves GNNs for Molecular Property Prediction","authors":"H Stärk, D Beaini","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Sterling and Irwin, 2015] T. Sterling and J. J. Irwin. ZINC 15- Ligand Discovery for Everyone. J. Chem. Inf. Model., 2015.","title":"ZINC 15-Ligand Discovery for Everyone","authors":"T Sterling, J J Irwin","year":"2015","venue":"J. Chem. Inf. Model","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Stumpfe et al., 2019] D. Stumpfe, H. Hu, et al. Evolving Concept of Activity Cliffs. ACS Omega, 2019.","title":"Evolving Concept of Activity Cliffs","authors":"D Stumpfe, H Hu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Sun et al., 2020] F. Sun, J. Hoffmann, et al. InfoGraph: Unsuper- vised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In ICLR, 2020.","title":"InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization","authors":"F Sun, J Hoffmann","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Sun et al., 2021] M. Sun, J. Xing, et al. MoCL: Contrastive Learn- ing on Molecular Graphs with Multi-level Domain Knowledge. In KDD, 2021.","title":"MoCL: Contrastive Learning on Molecular Graphs with Multi-level Domain Knowledge","authors":"M Sun, J Xing","year":"2021","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Sun et al., 2022] R. Sun, H. Dai, et al. Does GNN Pretraining Help Molecular Representation? In NeurIPS, 2022.","title":"Does GNN Pretraining Help Molecular Representation?","authors":"R Sun, H Dai","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Suresh et al., 2021] S. Suresh, P. Li, et al. Adversarial Graph Aug- mentation to Improve Graph Contrastive Learning. In NeurIPS, 2021.","title":"Adversarial Graph Augmentation to Improve Graph Contrastive Learning","authors":"S Suresh, P Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Vaswani et al., 2017] A. Vaswani, N. Shazeer, et al. Attention is All You Need. In NIPS, 2017.","title":"Attention is All You Need","authors":"A Vaswani, N Shazeer","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Velickovic et al., 2018] P. Velickovic, G. Cucurull, et al. Graph Attention Networks. In ICLR, 2018.","title":"Graph Attention Networks","authors":"P Velickovic, G Cucurull","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Velickovic et al., 2019] P. Velickovic, W. Fedus, et al. Deep Graph Infomax. In ICLR, 2019.","title":"Deep Graph Infomax","authors":"P Velickovic, W Fedus","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Wang et al., 2017] Y. Wang, S. H. Bryant, et al. Pubchem Bioas- say: 2017 Update. Nucleic Acids Res., 2017.","title":"Pubchem Bioassay: 2017 Update","authors":"Y Wang, S H Bryant","year":"2017","venue":"Nucleic Acids Res","doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Wang et al., 2019] S. Wang, Y. Guo, et al. SMILES-BERT: Large Scale Unsupervised Pre-Training for Molecular Property Predic- tion. In BCB, 2019.","title":"SMILES-BERT: Large Scale Unsupervised Pre-Training for Molecular Property Prediction","authors":"S Wang, Y Guo","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Wang et al., 2021] Y. Wang, Y. Min, et al. Molecular Graph Con- trastive Learning with Parameterized Explainable Augmentations. In BIBM, 2021.","title":"Molecular Graph Contrastive Learning with Parameterized Explainable Augmentations","authors":"Y Wang, Y Min","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Wang et al., 2022a] Y. Wang, R. Magar, et al. Improving Molec- ular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast. J. Chem. Inf. Model., 2022.","title":"Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast","authors":"Wang","year":"2022","venue":"J. Chem. Inf. Model","doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Wang et al., 2022b] Y. Wang, J. Wang, et al. MolCLR: Molecu- lar Contrastive Learning of Representations via Graph Neural Networks. Nat. Mach. Intell., 2022.","title":"MolCLR: Molecular Contrastive Learning of Representations via Graph Neural Networks","authors":"Wang","year":"2022","venue":"Nat. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":66,"raw":"[Weininger, 1988] D. Weininger. SMILES, A Chemical Language and Information System. 1. Introduction to Methodology and Encoding Rules. J. Chem. Inf. Comput. Sci., 1988.","title":"SMILES, A Chemical Language and Information System. 1. Introduction to Methodology and Encoding Rules","authors":"D Weininger","year":"1988","venue":"J. Chem. Inf. Comput. Sci","doi":null,"url":null,"internal_url":null},{"position":67,"raw":"[Wu et al., 2018] Z. Wu, B. Ramsundar, et al. MoleculeNet: A Benchmark for Molecular Machine Learning. Chem. Sci., 2018.","title":"MoleculeNet: A Benchmark for Molecular Machine Learning","authors":"Z Wu, B Ramsundar","year":"2018","venue":"Chem. Sci","doi":null,"url":null,"internal_url":null},{"position":68,"raw":"[Xia et al., 2022a] J. Xia, L. Wu, et al. ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning. In ICML, 2022.","title":"ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning","authors":"Xia","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":69,"raw":"[Xia et al., 2022b] J. Xia, L. Wu, et al. SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Aug- mentation. In WWW, 2022.","title":"SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation","authors":"Xia","year":"2022","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":70,"raw":"[Xia et al., 2023] J. Xia, C. Zhao, et al. Mole-BERT: Rethinking Pre-training Graph Neural Networks for Molecules. In ICLR, 2023.","title":"Mole-BERT: Rethinking Pre-training Graph Neural Networks for Molecules","authors":"J Xia, C Zhao","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":71,"raw":"[Xu et al., 2019] K. Xu, W. Hu, et al. How Powerful are Graph Neural Networks? In ICLR, 2019.","title":"How Powerful are Graph Neural Networks?","authors":"K Xu, W Hu","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":72,"raw":"[Xu et al., 2021a] D. Xu, W. Cheng, et al. InfoGCL: Information- Aware Graph Contrastive Learning. In NeurIPS, 2021.","title":"InfoGCL: Information-Aware Graph Contrastive Learning","authors":"Xu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":73,"raw":"[Xu et al., 2021b] M. Xu, H. Wang, et al. Self-supervised Graph- level Representation Learning with Local and Global Structure. In ICML, 2021.","title":"Self-supervised Graphlevel Representation Learning with Local and Global Structure","authors":"Xu","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":74,"raw":"[Xu et al., 2021c] Z. Xu, Y. Luo, et al. Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphs. arXiv.org, September 2021.","title":"Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphs","authors":"Xu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":75,"raw":"[Yi et al., 2022] J. Yi, C. Wu, et al. MICER: A Pre-Trained Encoder-Decoder Architecture for Molecular Image Caption- ing. Bioinform., 2022.","title":"MICER: A Pre-Trained Encoder-Decoder Architecture for Molecular Image Captioning","authors":"J Yi, C Wu","year":"2022","venue":"Bioinform","doi":null,"url":null,"internal_url":null},{"position":76,"raw":"[Ying et al., 2021] C. Ying, T. Cai, et al. Do Transformers Really Perform Badly for Graph Representation? In NeurIPS, 2021.","title":"Do Transformers Really Perform Badly for Graph Representation?","authors":"C Ying, T Cai","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":77,"raw":"[You et al., 2020] Y. You, T. Chen, et al. Graph Contrastive Learn- ing with Augmentations. In NeurIPS, 2020.","title":"Graph Contrastive Learning with Augmentations","authors":"Y You, T Chen","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":78,"raw":"[You et al., 2021] Y. You, T. Chen, et al. Graph Contrastive Learn- ing Automated. In ICML, 2021.","title":"Graph Contrastive Learning Automated","authors":"Y You, T Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":79,"raw":"[You et al., 2022] Y. You, T. Chen, et al. Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmen- tations. In WSDM, 2022.","title":"Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations","authors":"Y You, T Chen","year":"2022","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":80,"raw":"[Zaidi et al., 2023] S. Zaidi, M. Schaarschmidt, et al. Pre-training via Denoising for Molecular Property Prediction. In ICLR, 2023.","title":"Pre-training via Denoising for Molecular Property Prediction","authors":"S Zaidi, M Schaarschmidt","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":81,"raw":"[Zeng et al., 2022a] X. Zeng, H. Xiang, et al. Accurate Predic- tion of Molecular Properties and Drug Targets Using a Self- Supervised Image Representation Learning Framework. Nat. Mach. Intell., 2022.","title":"Accurate Prediction of Molecular Properties and Drug Targets Using a Self-Supervised Image Representation Learning Framework","authors":"Zeng","year":"2022","venue":"Nat. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":82,"raw":"[Zeng et al., 2022b] Z. Zeng, Y. Yao, et al. A Deep-Learning Sys- tem Bridging Molecule Structure and Biomedical Text with Com- prehension Comparable to Human Professionals. Nat. Commun., 2022.","title":"A Deep-Learning System Bridging Molecule Structure and Biomedical Text with Comprehension Comparable to Human Professionals","authors":"Zeng","year":"2022","venue":"Nat. Commun","doi":null,"url":null,"internal_url":null},{"position":83,"raw":"[Zhang et al., 2021] Z. Zhang, Q. Liu, et al. Motif-based Graph Self-Supervised Learning for Molecular Property Prediction. In NeurIPS, 2021.","title":"Motif-based Graph Self-Supervised Learning for Molecular Property Prediction","authors":"Z Zhang, Q Liu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":84,"raw":"[Zheng et al., 2022] J. Zheng, Y. Wang, G. Wang, J. Xia, Y. Huang, G. Zhao, Y. Zhang, and S. Z. Li. Using context-to-vector with graph retrofitting to improve word embeddings. ACL, 2022.","title":"Using context-to-vector with graph retrofitting to improve word embeddings","authors":"J Zheng, Y Wang, G Wang, J Xia, Y Huang, G Zhao, Y Zhang, S Z Li","year":"2022","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":85,"raw":"[Zhou et al., 2023] G. Zhou, Z. Gao, et al. Uni-Mol: A Universal 3D Molecular Representation Learning Framework. In ICLR, 2023.","title":"Uni-Mol: A Universal 3D Molecular Representation Learning Framework","authors":"G Zhou, Z Gao","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":86,"raw":"[Zhu et al., 2021] J. Zhu, Y. Xia, et al. Dual-view Molecule Pre- training. arXiv.org, June 2021.","title":"Dual-view Molecule Pretraining","authors":"J Zhu, Y Xia","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":87,"raw":"[Zhu et al., 2022a] J. Zhu, Y. Xia, et al. Unified 2D and 3D Pre- Training of Molecular Representations. In KDD, 2022.","title":"Unified 2D and 3D Pre-Training of Molecular Representations","authors":"Zhu","year":"2022","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":88,"raw":"[Zhu et al., 2022b] Y. Zhu, D. Chen, et al. Featurizations Matter: A Multiview Contrastive Learning Approach to Molecular Pre- training. In AI4Science@ICML, 2022.","title":"Featurizations Matter: A Multiview Contrastive Learning Approach to Molecular Pretraining","authors":"Zhu","year":"2022","venue":"AI4Science@ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}