{"type":"ScholarlyArticle","title":"Rethinking the Promotion Brought by Contrastive Learning to Semi-Supervised Node Classification","authors":[{"name":"Deli Chen","slug":"deli-chen","given":"Deli","family":"Chen","affiliation":"Pattern Recognition Center , WeChat AI , Tencent Inc. , China"},{"name":"Yankai Lin","slug":"yankai-lin","given":"Yankai","family":"Lin","affiliation":"Pattern Recognition Center , WeChat AI , Tencent Inc. , China"},{"name":"Lei Li","slug":"lei-li","given":"Lei","family":"Li","affiliation":"MOE Key Lab of Computational Linguistics , School of Computer Science , Peking University"},{"name":"Xuancheng Ren","slug":"xuancheng-ren","given":"Xuancheng","family":"Ren","affiliation":"MOE Key Lab of Computational Linguistics , School of Computer Science , Peking University"},{"name":"Peng Li","slug":"peng-li","given":"Peng","family":"Li","affiliation":"Institute for AI Industry Research (AIR) , Tsinghua University"},{"name":"Jie Zhou","slug":"jie-zhou","given":"Jie","family":"Zhou","affiliation":"Pattern Recognition Center , WeChat AI , Tencent Inc. , China"},{"name":"Xu Sun","slug":"xu-sun","given":"Xu","family":"Sun","affiliation":"MOE Key Lab of Computational Linguistics , School of Computer Science , Peking University"}],"abstract":"Graph Contrastive Learning (GCL) has proven highly effective in promoting the performance of Semi-Supervised Node Classification (SSNC). However, existing GCL methods are generally transferred from other fields like CV or NLP, whose underlying working mechanism remains underexplored. In this work, we first deeply probe the working mechanism of GCL in SSNC, and find that the promotion brought by GCL is severely unevenly distributed: the improvement mainly comes from subgraphs with less annotated information, which is fundamentally different from contrastive learning in other fields. However, existing GCL methods generally ignore this uneven distribution of annotated information and apply GCL evenly to the whole graph. To remedy this issue and further improve GCL in SSNC, we propose the Topology InFormation gain-Aware Graph Contrastive Learning (TIFA-GCL) framework that considers the annotated information distribution across graph in GCL. Extensive experiments on six benchmark graph datasets, including the enormous OGB-Products graph, show that TIFA-GCL can bring a larger improvement than existing GCL methods in both transductive and inductive settings. Further experiments demonstrate the generalizability and interpretability of TIFA-GCL.","doi":"10.24963/ijcai.2022/395","url":"https://ijcai.org/proceedings/2022/395","pdf":"https://ijcai.org/proceedings/2022/0395.pdf","markdown":"https://ijcai.org/proceedings/2022/395.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/395","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":395,"pages":{"first":2852,"last":2858},"keywords":["Machine Learning: Sequence and Graph Learning","Machine Learning: Semi-Supervised Learning","Data Mining: Mining Graphs"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986838","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984981","length_sec":348,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bai et al., 2021] S. Bai, F. Zhang, and P. H. Torr. Hypergraph Con- volution and Hypergraph Attention. Pattern Recognit., 2021.","title":"Hypergraph Convolution and Hypergraph Attention","authors":"S Bai, F Zhang, P H Torr","year":"2021","venue":"Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bianchi et al., 2019] F. M Bianchi, D. Grattarola, L. Livi, and C. Alippi. Graph Neural Networks with Convolutional Arma Fil- ters. arXiv: 1901.01343, 2019.","title":"Graph Neural Networks with Convolutional Arma Filters","authors":"F Bianchi, D Grattarola, L Livi, C Alippi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Buchnik and Cohen, 2018] Eliav Buchnik and Edith Cohen. Boot- strapped Graph Diffusions: Exposing the Power of Nonlinearity. Proc. ACM Meas. Anal. Comput. Syst., 2(1):10:1-10:19, 2018.","title":"Bootstrapped Graph Diffusions: Exposing the Power of Nonlinearity","authors":"Cohen Buchnik, Eliav Buchnik, Edith Cohen","year":"2018","venue":"Proc. ACM Meas. Anal. Comput. Syst","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] S. Chen, Z. W., X. Z., X. Z., and D. P. Probing Negative Sampling Strategies to Learn Graph Representations via Unsupervised Contrastive Learning. arXiv: 2104.06317, 2021.","title":"Probing Negative Sampling Strategies to Learn Graph Representations via Unsupervised Contrastive Learning","authors":"S Chen, Z W, X Z, X Z, D P","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In NeurIPS, 2016.","title":"Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feng et al., 2020] Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Khar- lamov, and Jie Tang. Graph random neural networks for semi- supervised learning on graphs. In NeurIPS, 2020.","title":"Graph random neural networks for semisupervised learning on graphs","authors":"Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, Jie Tang","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hamilton et al., 2017] W. L. Hamilton, Z. Ying, and J. L. Inductive Representation Learning on Large Graphs. In NeurIPS, 2017.","title":"Inductive Representation Learning on Large Graphs","authors":"W L Hamilton, Z Ying, J L","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hassani and Khasahmadi, 2020] Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive Multi-View Representation Learning on Graphs. In ICML, 2020.","title":"Contrastive Multi-View Representation Learning on Graphs","authors":"Khasahmadi Hassani, Kaveh Hassani, Amir Hosein, Khasahmadi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In CVPR, 2020.","title":"Momentum Contrast for Unsupervised Visual Representation Learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2020] Weihua Hu, Matthias Fey, Marinka Zitnik, Yux- iao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open Graph Benchmark: Datasets for Machine Learn- ing on Graphs. In NeurIPS, 2020.","title":"Open Graph Benchmark: Datasets for Machine Learning on Graphs","authors":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, Jure Leskovec","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jovanovic et al., 2021] Nikola Jovanovic, Zhao Meng, Lukas Faber, and Roger Wattenhofer. Towards Robust Graph Con- trastive Learning. arXiv: 2102.13085, 2021.","title":"Towards Robust Graph Contrastive Learning","authors":"Nikola Jovanovic, Zhao Meng, Lukas Faber, Roger Wattenhofer","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kefato and Girdzijauskas, 2021] Zekarias T. Kefato and Sarunas Girdzijauskas. Self-supervised Graph Neural Networks without Explicit Negative Sampling. arXiv: 2103.14958, 2021.","title":"Self-supervised Graph Neural Networks without Explicit Negative Sampling","authors":"T Zekarias, Sarunas Kefato, Girdzijauskas","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kipf and W., 2017] T. N. Kipf and M. W. Semi-supervised Classi- fication with Graph Convolutional Networks. In ICLR, 2017.","title":"Semi-supervised Classification with Graph Convolutional Networks","authors":"T N Kipf, M W","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Morris et al., 2019] Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and Leman go Neural: Higher-order Graph Neural Networks. In AAAI, 2019.","title":"Weisfeiler and Leman go Neural: Higher-order Graph Neural Networks","authors":"Christopher Morris, Martin Ritzert, Matthias Fey, Jan William L Hamilton, Gaurav Eric Lenssen, Martin Rattan, Grohe","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Qiu et al., 2020] Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. GCC: Graph Contrastive Coding for Graph Neural Network Pre- Training. In KDD, 2020.","title":"GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training","authors":"Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, Jie Tang","year":"2020","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Qu et al., 2021] Y. Qu, D. S., Y. S., S. S., W. C., and J. H. CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding. In ICLR, 2021.","title":"CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding","authors":"Y Qu, D S, Y S, S S, W C, J H","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Robinson et al., 2021] Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. Contrastive Learning with Hard Negative Samples. In ICLR, 2021.","title":"Contrastive Learning with Hard Negative Samples","authors":"Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, Stefanie Jegelka","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A Unified Embedding for Face Recog- nition and Clustering. In CVPR, 2015.","title":"Facenet: A Unified Embedding for Face Recognition and Clustering","authors":"Florian Schroff, Dmitry Kalenichenko, James Philbin","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective Classification in Network Data. AI magazine, 29(3):93-93, 2008.","title":"Collective Classification in Network Data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, Tina Eliassi-Rad","year":"2008","venue":"AI magazine","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shchur et al., 2018] Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of Graph Neural Network Evaluation. arXiv: 1811.05868, 2018.","title":"Pitfalls of Graph Neural Network Evaluation","authors":"Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, Stephan Günnemann","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sohn et al., 2020] K. Sohn, D. Berthelot, C. Li, Z. Zhang, N. Car- lini, E. D. Cubuk, A. Kurakin, H. Zhang, and C. Raffel. Fix- match: Simplifying Semi-supervised Learning with Consistency and Confidence. In NeurIPS, 2020.","title":"Fixmatch: Simplifying Semi-supervised Learning with Consistency and Confidence","authors":"K Sohn, D Berthelot, C Li, Z Zhang, N Carlini, E D Cubuk, A Kurakin, H Zhang, C Raffel","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Aran- txa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph Attention Networks. In ICLR, 2018.","title":"Graph Attention Networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Velickovic et al., 2019] Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. Deep Graph Infomax. In ICLR, 2019.","title":"Deep Graph Infomax","authors":"Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, R, Devon Hjelm","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Verma et al., 2018] Nitika Verma, Edmond Boyer, and Jakob Ver- beek. Feastnet: Feature-steered Graph Convolutions for 3d Shape Analysis. In CVPR, 2018.","title":"Feastnet: Feature-steered Graph Convolutions for 3d Shape Analysis","authors":"Nitika Verma, Edmond Boyer, Jakob Verbeek","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Verma et al., 2019] Vikas Verma, Meng Qu, Alex Lamb, Yoshua Bengio, Juho Kannala, and Jian Tang. GraphMix: Regularized Training of Graph Neural Networks for Semi-Supervised Learn- ing. arXiv: 1909.11715, 2019.","title":"GraphMix: Regularized Training of Graph Neural Networks for Semi-Supervised Learning","authors":"Vikas Verma, Meng Qu, Alex Lamb, Yoshua Bengio, Juho Kannala, Jian Tang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wan et al., 2021] S. Wan, S. Pan, J. Yang, and C. Gong. Con- trastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning. In AAAI, 2021.","title":"Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning","authors":"S Wan, S Pan, J Yang, C Gong","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang and Leskovec, 2020] Hongwei Wang and Jure Leskovec. Unifying Graph Convolutional Neural Networks and Label Prop- agation. arXiv: 2002.06755, 2020.","title":"Unifying Graph Convolutional Neural Networks and Label Propagation","authors":"Hongwei Wang, Jure Leskovec","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang and Liu, 2021] Chenguang Wang and Ziwen Liu. Graph Representation Learning by Ensemble Aggregating Subgraphs via Mutual Information Maximization. arXiv: 2103.13125, 2021.","title":"Graph Representation Learning by Ensemble Aggregating Subgraphs via Mutual Information Maximization","authors":"Chenguang Wang, Ziwen Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2020] Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, Juncheng Liu, and Bryan Hooi. NodeAug: Semi-Supervised Node Classification with Data Augmentation. In KDD, 2020.","title":"NodeAug: Semi-Supervised Node Classification with Data Augmentation","authors":"Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, Juncheng Liu, Bryan Hooi","year":"2020","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xie et al., 2021] Qizhe Xie, Zihang Dai, Eduard H. Hovy, Thang Luong, and Quoc Le. Unsupervised Data Augmentation for Con- sistency Training. In NeurIPS, 2021.","title":"Unsupervised Data Augmentation for Consistency Training","authors":"Xie","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2016] Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. Revisiting Semi-supervised Learning with Graph Embeddings. In ICML, 2016.","title":"Revisiting Semi-supervised Learning with Graph Embeddings","authors":"Zhilin Yang, William W Cohen, Ruslan Salakhutdinov","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2020] Zhen Yang, Ming Ding, Chang Zhou, Hongxia Yang, Jingren Zhou, and Jie Tang. Understanding Negative Sam- pling in Graph Representation Learning. In KDD, 2020.","title":"Understanding Negative Sampling in Graph Representation Learning","authors":"Zhen Yang, Ming Ding, Chang Zhou, Hongxia Yang, Jingren Zhou, Jie Tang","year":"2020","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph Contrastive Learning with Augmentations. In NeurIPS, 2020.","title":"Graph Contrastive Learning with Augmentations","authors":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zeng et al., 2020] Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivas- tava, Rajgopal Kannan, and Viktor K. Prasanna. GraphSAINT: Graph Sampling Based Inductive Learning Method. In ICLR, 2020.","title":"GraphSAINT: Graph Sampling Based Inductive Learning Method","authors":"Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, Viktor K Prasanna","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhu et al., 2020] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep Graph Contrastive Representa- tion Learning. arXiv: 2006.04131, 2020.","title":"Deep Graph Contrastive Representation Learning","authors":"Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhu et al., 2021] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph Contrastive Learning with Adaptive Augmentation. In WWW, 2021.","title":"Graph Contrastive Learning with Adaptive Augmentation","authors":"Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}