Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks

Li, Pengyong; Wang, Jun; Li, Ziliang; Qiao, Yixuan; Liu, Xianggen; Ma, Fei; Gao, Peng; Song, Sen; Xie, Guotong

doi:10.24963/ijcai.2021/371

Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks

Pengyong Li, Jun Wang, Ziliang Li, Yixuan Qiao, Xianggen Liu, Fei Ma, Peng Gao, Sen Song, Guotong Xie

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 2694-2700.

https://doi.org/10.24963/ijcai.2021/371

Keywords

Machine Learning: Unsupervised Learning
Machine Learning: Deep Learning
Data Mining: Mining Graphs, Semi Structured Data, Complex Data

Abstract

Self-supervised learning has gradually emerged as a powerful technique for graph representation learning. However, transferable, generalizable, and robust representation learning on graph data still remains a challenge for pre-training graph neural networks. In this paper, we propose a simple and effective self-supervised pre-training strategy, named Pairwise Half-graph Discrimination (PHD), that explicitly pre-trains a graph neural network at graph-level. PHD is designed as a simple binary classification task to discriminate whether two half-graphs come from the same source. Experiments demonstrate that the PHD is an effective pre-training strategy that offers comparable or superior performance on 13 graph classification tasks compared with state-of-the-art strategies, and achieves notable improvements when combined with node-level strategies. Moreover, the visualization of learned representation revealed that PHD strategy indeed empowers the model to learn graph-level knowledge like the molecular scaffold. These results have established PHD as a powerful and effective self-supervised learning strategy in graph-level representation learning.

Pengyong Li, Jun Wang, Ziliang Li, Yixuan Qiao, Xianggen Liu, Fei Ma, Peng Gao, Sen Song, and Guotong Xie. 2021. Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 2694–2700. https://doi.org/10.24963/ijcai.2021/371

@inproceedings{ijcai2021p371,
  title     = {Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks},
  author    = {Li, Pengyong and Wang, Jun and Li, Ziliang and Qiao, Yixuan and Liu, Xianggen and Ma, Fei and Gao, Peng and Song, Sen and Xie, Guotong},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Prof. Zhi-Hua Zhou},
  pages     = {2694--2700},
  year      = {2021},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2021/371},
  url       = {https://doi.org/10.24963/ijcai.2021/371},
}

TY  - CPAPER
TI  - Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks
AU  - Li, Pengyong
AU  - Wang, Jun
AU  - Li, Ziliang
AU  - Qiao, Yixuan
AU  - Liu, Xianggen
AU  - Ma, Fei
AU  - Gao, Peng
AU  - Song, Sen
AU  - Xie, Guotong
T2  - Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2021
SP  - 2694
EP  - 2700
DO  - 10.24963/ijcai.2021/371
UR  - https://doi.org/10.24963/ijcai.2021/371
ER  -