{"type":"ScholarlyArticle","title":"Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders","authors":[{"name":"Chuang Liu","slug":"chuang-liu","given":"Chuang","family":"Liu","affiliation":"School of Computer Science , Wuhan University , Wuhan , China"},{"name":"Yuyao Wang","slug":"yuyao-wang","given":"Yuyao","family":"Wang","affiliation":"School of Computer Science , Wuhan University , Wuhan , China"},{"name":"Yibing Zhan","slug":"yibing-zhan","given":"Yibing","family":"Zhan","affiliation":"JD Explore Academy , JD .com , China"},{"name":"Xueqi Ma","slug":"xueqi-ma","given":"Xueqi","family":"Ma","affiliation":"School of Computing and Information Systems , The University of Melbourne , Melbourne , Australia"},{"name":"Dapeng Tao","slug":"dapeng-tao","given":"Dapeng","family":"Tao","affiliation":"School of Computer Science , Yunnan University , Kunming , China"},{"name":"Jia Wu","slug":"jia-wu","given":"Jia","family":"Wu","affiliation":"School of Computing , Macquarie University , Sydney , Australia"},{"name":"Wenbin Hu","slug":"wenbin-hu","given":"Wenbin","family":"Hu","affiliation":"School of Computer Science , Wuhan University , Wuhan , China"}],"abstract":"Graph masked autoencoders (GMAE) have emerged as a significant advancement in self-supervised pre-training for graph-structured data. Previous GMAE models primarily utilize a straightforward random masking strategy for nodes or edges during training. However, this strategy fails to consider the varying significance of different nodes within the graph structure. In this paper, we investigate the potential of leveraging the graph's structural composition as a fundamental and unique prior in the masked pre-training process.  To this end, we introduce a novel structure-guided masking strategy (i.e., StructMAE), designed to refine the existing GMAE models. StructMAE involves two steps: 1) Structure-based Scoring: Each node is evaluated and assigned a score reflecting its structural significance.  Two distinct types of scoring manners are proposed: predefined and learnable scoring. 2) Structure-guided Masking: With the obtained assessment scores, we develop an easy-to-hard masking strategy that gradually increases the structural awareness of the self-supervised reconstruction task. Specifically, the strategy begins with random masking and progresses to masking structure-informative nodes based on the assessment scores. This design gradually and effectively guides the model in learning graph structural information. Furthermore, extensive experiments consistently demonstrate that our StructMAE method outperforms existing state-of-the-art GMAE models in both unsupervised and transfer learning tasks. Codes are available at https: //github.com/LiuChuang0059/StructMAE.","doi":"10.24963/ijcai.2024/241","url":"https://ijcai.org/proceedings/2024/241","pdf":"https://ijcai.org/proceedings/2024/0241.pdf","markdown":"https://ijcai.org/proceedings/2024/241.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/241","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":241,"pages":{"first":2180,"last":2188},"keywords":["Data Mining: DM: Mining graphs","Machine Learning: ML: Self-supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Brown et al., 2020] Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, et al. Language models are few-shot learners. In NeurIPS, 2020.","title":"Language models are few-shot learners","authors":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang and Lin, 2011] Chih-Chung Chang and Chih-Jen Lin. Libsvm: a library for support vector machines. TIST, 2011.","title":"Libsvm: a library for support vector machines","authors":"Chang, Lin ; Chih-Chung Chang, Chih-Jen Lin","year":"2011","venue":"TIST","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In ACL, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hasanzadeh et al., 2019] Arman Hasanzadeh, Ehsan Haji- ramezanali, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. Semi-implicit graph variational auto-encoders. In NeurIPS, 2019.","title":"Semi-implicit graph variational auto-encoders","authors":"Arman Hasanzadeh, Ehsan Hajiramezanali, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, Xiaoning Qian","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2022] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In CVPR, 2022.","title":"Masked autoencoders are scalable vision learners","authors":"Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hou et al., 2022] Zhenyu Hou, Xiao Liu, Yukuo Cen, Yux- iao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In SIGKDD, 2022.","title":"Graphmae: Self-supervised masked graph autoencoders","authors":"Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, Jie Tang","year":"2022","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hou et al., 2023] Zhenyu Hou, Yufei He, Yukuo Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, and Jie Tang. Graphmae2: A decoding-enhanced masked self- supervised graph learner. In WWW, 2023.","title":"Graphmae2: A decoding-enhanced masked selfsupervised graph learner","authors":"Zhenyu Hou, Yufei He, Yukuo Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, Jie Tang","year":"2023","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2020a] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. arXiv:2005.00687, 2020.","title":"Open graph benchmark: Datasets for machine learning on graphs","authors":"Hu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu* et al., 2020b] Weihua Hu*, Bowen Liu*, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In ICLR, 2020.","title":"Strategies for pre-training graph neural networks","authors":"* Hu","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Inae et al., 2023] Eric Inae, Gang Liu, and Meng Jiang. Motif-aware attribute masking for molecular graph pre- training. arXiv:2309.04589, 2023.","title":"Motif-aware attribute masking for molecular graph pretraining","authors":"Eric Inae, Gang Liu, Meng Jiang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv:1611.07308, 2016.","title":"Variational graph auto-encoders","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2022] Sihang Li, Xiang Wang, An Zhang, Yingxin Wu, Xiangnan He, and Tat-Seng Chua. Let invariant ratio- nale discovery inspire graph contrastive learning. In ICLR, 2022.","title":"Let invariant rationale discovery inspire graph contrastive learning","authors":"Sihang Li, Xiang Wang, An Zhang, Yingxin Wu, Xiangnan He, Tat-Seng Chua","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2023] Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, and Weiqiang Wang. What's behind the mask: Understanding masked graph modeling for graph autoen- coders. In SIGKDD, 2023.","title":"What's behind the mask: Understanding masked graph modeling for graph autoencoders","authors":"Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, Weiqiang Wang","year":"2023","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2017] Weiwei Liu, Ivor W Tsang, Müller Klaus- Robert, et al. An easy-to-hard learning paradigm for mul- tiple classes and multiple labels. JMLR, 2017.","title":"An easy-to-hard learning paradigm for multiple classes and multiple labels","authors":"Weiwei Liu, Ivor W Tsang, Müller Klaus-Robert","year":"2017","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2018] Weiwei Liu, Donna Xu, Ivor W Tsang, and Wenjie Zhang. Metric learning for multi-output tasks. IEEE TPAMI, 2018.","title":"Metric learning for multi-output tasks","authors":"Weiwei Liu, Donna Xu, Ivor W Tsang, Wenjie Zhang","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2023a] Chuang Liu, Yibing Zhan, Xueqi Ma, Liang Ding, Dapeng Tao, Jia Wu, and Wenbin Hu. Gap- former: Graph transformer with graph pooling for node classification. In IJCAI-23, 2023.","title":"Gapformer: Graph transformer with graph pooling for node classification","authors":"Liu","year":"2023","venue":"IJCAI-23","doi":null,"url":null,"internal_url":"/proceedings/2023/244"},{"position":17,"raw":"[Liu et al., 2023b] Chuang Liu, Yibing Zhan, Xueqi Ma, Dapeng Tao, Bo Du, and Wenbin Hu. Masked graph auto- encoder constrained graph pooling. In ECML PKDD, 2023.","title":"Masked graph autoencoder constrained graph pooling","authors":"Liu","year":"2023","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2023c] Zhiyuan Liu, Yaorui Shi, An Zhang, En- zhi Zhang, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Rethinking tokenizer and decoder in masked graph modeling for molecules. NeurIPS, 2023.","title":"Rethinking tokenizer and decoder in masked graph modeling for molecules","authors":"Liu","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Page et al., 1999] Lawrence Page, Sergey Brin, Rajeev Mot- wani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford In- foLab, 1999.","title":"The pagerank citation ranking: Bringing order to the web","authors":"Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pan et al., 2018] Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. Adversarially regu- larized graph autoencoder for graph embedding. In IJCAI, 2018.","title":"Adversarially regularized graph autoencoder for graph embedding","authors":"Pan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/362"},{"position":21,"raw":"[Qiu et al., 2020] Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. Gcc: Graph contrastive coding for graph neural network pre-training. In SIGKDD, 2020.","title":"Gcc: Graph contrastive coding for graph neural network pre-training","authors":"Qiu","year":"2020","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shi et al., 2023] Yucheng Shi, Yushun Dong, Qiaoyu Tan, Jundong Li, and Ninghao Liu. Gigamae: Generalizable graph masked autoencoder via collaborative latent space reconstruction. In CIKM, 2023.","title":"Gigamae: Generalizable graph masked autoencoder via collaborative latent space reconstruction","authors":"Yucheng Shi, Yushun Dong, Qiaoyu Tan, Jundong Li, Ninghao Liu","year":"2023","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sterling and Irwin, 2015] Teague Sterling and John J Irwin. Zinc 15-ligand discovery for everyone. Journal of chemical information and modeling, 2015.","title":"Zinc 15-ligand discovery for everyone","authors":"Teague Sterling, John J Irwin","year":"2015","venue":"Journal of chemical information and modeling","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun et al., 2020] Fan-Yun Sun, Jordan Hoffman, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In ICLR, 2020.","title":"Infograph: Unsupervised and semisupervised graph-level representation learning via mutual information maximization","authors":"Fan-Yun Sun, Jordan Hoffman, Vikas Verma, Jian Tang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tan et al., 2023] Qiaoyu Tan, Ninghao Liu, Xiao Huang, Soo-Hyun Choi, Li Li, Rui Chen, and Xia Hu. S2gae: Self- supervised graph autoencoders are generalizable learners with graph masking. In WSDM, 2023.","title":"S2gae: Selfsupervised graph autoencoders are generalizable learners with graph masking","authors":"Qiaoyu Tan, Ninghao Liu, Xiao Huang, Soo-Hyun Choi, Li Li, Rui Chen, Xia Hu","year":"2023","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tian et al., 2023] Yijun Tian, Kaiwen Dong, Chunhui Zhang, Chuxu Zhang, and Nitesh V Chawla. Heteroge- neous graph masked autoencoders. In AAAI, 2023.","title":"Heterogeneous graph masked autoencoders","authors":"Yijun Tian, Kaiwen Dong, Chunhui Zhang, Chuxu Zhang, Nitesh V Chawla","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tu et al., 2024] Wenxuan Tu, Qing Liao, Sihang Zhou, Xin Peng, Chuan Ma, Zhe Liu, Xinwang Liu, and Zhiping Cai. Rare: Robust masked graph autoencoder. IEEE TKDE, 2024.","title":"Rare: Robust masked graph autoencoder","authors":"Wenxuan Tu, Qing Liao, Sihang Zhou, Xin Peng, Chuan Ma, Zhe Liu, Xinwang Liu, Zhiping Cai","year":"2024","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Veličković et al., 2019] Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In ICLR, 2019.","title":"Deep graph infomax","authors":"Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, Devon Hjelm","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2024] Yuxiang Wang, Xiao Yan, Chuang Hu, Fangcheng Fu, Wentao Zhang, Hao Wang, Shuo Shang, and Jiawei Jiang. Generative and contrastive paradigms are complementary for graph self-supervised learning. In ICDE, 2024.","title":"Generative and contrastive paradigms are complementary for graph self-supervised learning","authors":"Yuxiang Wang, Xiao Yan, Chuang Hu, Fangcheng Fu, Wentao Zhang, Hao Wang, Shuo Shang, Jiawei Jiang","year":"2024","venue":"ICDE","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2018] Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay Pande. Moleculenet: a benchmark for molecular machine learning. Chem. Sci., 2018.","title":"Moleculenet: a benchmark for molecular machine learning","authors":"Wu","year":"2018","venue":"Chem. Sci","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xia et al., 2022a] Jun Xia, Lirong Wu, Jintao Chen, Bozhen Hu, and Stan Z. Li. Simgrace: A simple framework for graph contrastive learning without data augmentation. In WWW, 2022.","title":"Simgrace: A simple framework for graph contrastive learning without data augmentation","authors":"Xia","year":"2022","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xia et al., 2022b] Jun Xia, Yanqiao Zhu, Yuanqi Du, and Stan Z Li. A survey of pretraining on graphs: Taxonomy, methods, and applications. arXiv:2202.07893, 2022.","title":"A survey of pretraining on graphs: Taxonomy, methods, and applications","authors":"Xia","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xia et al., 2023] Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z. Li. Mole-BERT: Rethinking pre-training graph neural networks for molecules. In ICLR, 2023.","title":"Mole-BERT: Rethinking pre-training graph neural networks for molecules","authors":"Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, Stan Z Li","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xu et al., 2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.","title":"How powerful are graph neural networks?","authors":"Xu","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xu et al., 2021a] Dongkuan Xu, Wei Cheng, Dongsheng Luo, Haifeng Chen, and Xiang Zhang. InfoGCL: Information-aware graph contrastive learning. In NeurIPS, 2021.","title":"InfoGCL: Information-aware graph contrastive learning","authors":"Xu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2021b] Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. Self-supervised graph-level representation learning with local and global structure. In ICML, 2021.","title":"Self-supervised graph-level representation learning with local and global structure","authors":"Xu","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yan et al., 2023] Hong Yan, Yang Liu, Yushen Wei, Zhen Li, Guanbin Li, and Liang Lin. Skeletonmae: graph-based masked autoencoder for skeleton sequence pre-training. In ICCV, 2023.","title":"Skeletonmae: graph-based masked autoencoder for skeleton sequence pre-training","authors":"Yan","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Ying et al., 2018] Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differen- tiable pooling. In NeurIPS, 2018.","title":"Hierarchical graph representation learning with differentiable pooling","authors":"Ying","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Ying et al., 2021] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? In NeurIPS, 2021.","title":"Do transformers really perform badly for graph representation?","authors":"Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, Tie-Yan Liu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph con- trastive learning with augmentations. In NeurIPS, 2020.","title":"Graph contrastive learning with augmentations","authors":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[You et al., 2021] Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning auto- mated. In ICML, 2021.","title":"Graph contrastive learning automated","authors":"You","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yu et al., 2024] Yue Yu, Xiao Wang, Mengmei Zhang, Nian Liu, and Chuan Shi. Provable training for graph contrastive learning. NeurIPS, 36, 2024.","title":"Provable training for graph contrastive learning","authors":"Yue Yu, Xiao Wang, Mengmei Zhang, Nian Liu, Chuan Shi","year":"2024","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Yuan et al., 2023] Mingzhi Yuan, Ao Shen, Kexue Fu, Ji- aming Guan, Yingfan Ma, Qin Qiao1, and Manning Wang. ProteinMAE: Masked Autoencoder for Protein Surface Self-supervised Learning. Bioinformatics, 2023.","title":"ProteinMAE: Masked Autoencoder for Protein Surface Self-supervised Learning","authors":"Yuan","year":"2023","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhang et al., 2022a] Sixiao Zhang, Hongxu Chen, Hao- ran Yang, Xiangguo Sun, Philip S Yu, and Guandong Xu. Graph masked autoencoders with transformers. arXiv:2202.08391, 2022.","title":"Graph masked autoencoders with transformers","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhang et al., 2022b] Yifei Zhang, Hao Zhu, Zixing Song, Piotr Koniusz, and Irwin King. Costa: covariance- preserving feature augmentation for graph contrastive learning. In SIGKDD, pages 2524-2534, 2022.","title":"Costa: covariancepreserving feature augmentation for graph contrastive learning","authors":"Zhang","year":"2022","venue":"SIGKDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}