{"type":"ScholarlyArticle","title":"Multi-Scale Subgraph Contrastive Learning","authors":[{"name":"Yanbei Liu","slug":"yanbei-liu","given":"Yanbei","family":"Liu","affiliation":"School of Life Sciences , Tiangong University"},{"name":"Yu Zhao","slug":"yu-zhao","given":"Yu","family":"Zhao","affiliation":"School of Electronics and Information Engineering , Tiangong University"},{"name":"Xiao Wang","slug":"xiao-wang","given":"Xiao","family":"Wang","affiliation":"School of Software , Beihang University"},{"name":"Lei Geng","slug":"lei-geng","given":"Lei","family":"Geng","affiliation":"School of Life Sciences , Tiangong University"},{"name":"Zhitao Xiao","slug":"zhitao-xiao","given":"Zhitao","family":"Xiao","affiliation":"School of Life Sciences , Tiangong University"}],"abstract":"Graph-level contrastive learning, aiming to learn the representations for each graph by contrasting two augmented graphs, has attracted considerable attention. Previous studies usually simply assume that a graph and its augmented graph as a positive pair, otherwise as a negative pair. However, it is well known that graph structure is always complex and multi-scale, which gives rise to a fundamental question: after graph augmentation, will the previous assumption still hold in reality? By an experimental analysis, we discover the semantic information of an augmented graph structure may be not consistent as original graph structure, and whether two augmented graphs are positive or negative pairs is highly related with the multi-scale structures. Based on this finding, we propose a multi-scale subgraph contrastive learning architecture which is able to characterize the fine-grained semantic information. Specifically, we generate global and local views at different scales based on subgraph sampling, and construct multiple contrastive relationships according to their semantic associations to provide richer self-supervised signals. Extensive experiments and parametric analyzes on eight graph classification real-world datasets well demonstrate the effectiveness of the proposed method.","doi":"10.24963/ijcai.2023/246","url":"https://ijcai.org/proceedings/2023/246","pdf":"https://ijcai.org/proceedings/2023/0246.pdf","markdown":"https://ijcai.org/proceedings/2023/246.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/246","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Data Mining","track":"Main Track","number":246,"pages":{"first":2215,"last":2223},"keywords":["Machine Learning: ML: Representation learning","Data Mining: DM: Mining graphs","Machine Learning: ML: Self-supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Adhikari et al., 2018] Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, and B Aditya Prakash. Sub2vec: Feature learning for subgraphs. In PAKDD, pages 170-182, 2018.","title":"Sub2vec: Feature learning for subgraphs","authors":"Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, Aditya Prakash","year":"2018","venue":"PAKDD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aggarwal et al., 2001] Charu C Aggarwal, Alexander Hin- neburg, and Daniel A Keim. On the surprising behavior of distance metrics in high dimensional space. In Inter- national Conference on Database Theory, pages 420-434, 2001.","title":"On the surprising behavior of distance metrics in high dimensional space","authors":"C Charu, Alexander Aggarwal, Daniel A Hinneburg, Keim","year":"2001","venue":"International Conference on Database Theory","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019a] Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as a universal machine learning framework for molecules and crystals. Chemistry of Materials, 31(9):3564-3572, 2019.","title":"Graph networks as a universal machine learning framework for molecules and crystals","authors":"Chen","year":"2019","venue":"Chemistry of Materials","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2019b] Ting Chen, Song Bian, and Yizhou Sun. Are powerful graph neural nets necessary? a dissection on graph classification. arXiv preprint arXiv:1905.04579, 2019.","title":"Are powerful graph neural nets necessary? a dissection on graph classification","authors":"Chen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597-1607, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chu et al., 2021] Guanyi Chu, Xiao Wang, Chuan Shi, and Xunqiang Jiang. Cuco: Graph representation with cur- riculum contrastive learning. In IJCAI, pages 2300-2306, 2021.","title":"Cuco: Graph representation with curriculum contrastive learning","authors":"Guanyi Chu, Xiao Wang, Chuan Shi, Xunqiang Jiang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/317"},{"position":7,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In ICML, pages 1263-1272, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, S Samuel, Patrick F Schoenholz, Oriol Riley, George E Vinyals, Dahl","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In KDD, pages 855-864, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hassani and Khasahmadi, 2020] Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view repre- sentation learning on graphs. In ICML, pages 4116-4126, 2020.","title":"Contrastive multi-view representation learning on graphs","authors":"Kaveh Hassani, Amir Hosein, Khasahmadi","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2020] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural net- works. In ICLR, 2020.","title":"Strategies for pre-training graph neural networks","authors":"Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, Jure Leskovec","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jain et al., 2016] Anubhav Jain, Yongwoo Shin, and Kristin A Persson. Computational predictions of energy materials using density functional theory. Nature Reviews Materials, 1(1):1-13, 2016.","title":"Computational predictions of energy materials using density functional theory","authors":"Anubhav Jain, Yongwoo Shin, Kristin A Persson","year":"2016","venue":"Nature Reviews Materials","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang et al., 2017] Biaobin Jiang, Kyle Kloster, David F Gleich, and Michael Gribskov. Aptrank: an adaptive pagerank model for protein function prediction on bi- relational graphs. Bioinformatics, 33(12):1829-1836, 2017.","title":"Aptrank: an adaptive pagerank model for protein function prediction on birelational graphs","authors":"Jiang","year":"2017","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiao et al., 2020] Yizhu Jiao, Yun Xiong, Jiawei Zhang, Yao Zhang, Tianqi Zhang, and Yangyong Zhu. Sub-graph contrast for scalable self-supervised graph representation learning. In ICDM, pages 222-231, 2020.","title":"Sub-graph contrast for scalable self-supervised graph representation learning","authors":"Yizhu Jiao, Yun Xiong, Jiawei Zhang, Yao Zhang, Tianqi Zhang, Yangyong Zhu","year":"2020","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jin et al., 2021] Ming Jin, Yizhen Zheng, Yuan-Fang Li, Chen Gong, Chuan Zhou, and Shirui Pan. Multi-scale con- trastive siamese networks for self-supervised graph repre- sentation learning. In IJCAI, 2021.","title":"Multi-scale contrastive siamese networks for self-supervised graph representation learning","authors":"Ming Jin, Yizhen Zheng, Yuan-Fang Li, Chen Gong, Chuan Zhou, Shirui Pan","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/204"},{"position":15,"raw":"[Kavun et al., 2012] Sergii V Kavun, Irina V Mykhalchuk, Nataliya I Kalashnykova, and Oleksandr G Zyma. A method of internet-analysis by the tools of graph theory. In Intelligent Decision Technologies, pages 35-44, 2012.","title":"A method of internet-analysis by the tools of graph theory","authors":"Irina V Sergii V Kavun, Nataliya I Mykhalchuk, Oleksandr G Kalashnykova, Zyma","year":"2012","venue":"Intelligent Decision Technologies","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Variational graph auto-encoders. In NeurIPS, 2016.","title":"Variational graph auto-encoders","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Morris et al., 2020] Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Mar- ion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv:2007.08663, 2020.","title":"Tudataset: A collection of benchmark datasets for learning with graphs","authors":"Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, Marion Neumann","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Narayanan et al., 2017] Annamalai Narayanan, Mahinthan Chandramohan, Rajasekar Venkatesan, Lihui Chen, Yang Liu, and Shantanu Jaiswal. graph2vec: Learning dis- tributed representations of graphs. arXiv:1707.05005, 2017.","title":"graph2vec: Learning distributed representations of graphs","authors":"Annamalai Narayanan, Mahinthan Chandramohan, Rajasekar Venkatesan, Lihui Chen, Yang Liu, Shantanu Jaiswal","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Newman, 2013] Mark EJ Newman. Spectral methods for community detection and graph partitioning. Physical Re- view E, 88(4):042822, 2013.","title":"Spectral methods for community detection and graph partitioning","authors":"Mark Ej Newman","year":"2013","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Aaron Van Den Oord, Yazhe Li, Oriol Vinyals","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shervashidze et al., 2011] Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(9), 2011.","title":"Weisfeiler-lehman graph kernels","authors":"Nino Shervashidze, Pascal Schweitzer, Erik, Jan Van Leeuwen, Kurt Mehlhorn, Karsten M Borgwardt","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sun et al., 2020] Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In ICLR, 2020.","title":"Infograph: Unsupervised and semisupervised graph-level representation learning via mutual information maximization","authors":"Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, Jian Tang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Suresh et al., 2021] Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to im- prove graph contrastive learning. In NeurIPS, pages 15920-15933, 2021.","title":"Adversarial graph augmentation to improve graph contrastive learning","authors":"Susheel Suresh, Pan Li, Cong Hao, Jennifer Neville","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tang and Liu, 2010] Lei Tang and Huan Liu. Graph min- ing applications to social network analysis. Managing and Mining Graph Data, pages 487-513, 2010.","title":"Graph mining applications to social network analysis","authors":"Liu ; ] Lei Tang, Huan Tang, Liu","year":"2010","venue":"Managing and Mining Graph Data","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Velickovic et al., 2019] Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In ICLR, 2019.","title":"Deep graph infomax","authors":"Petar Velickovic, William Fedus, Pietro William L Hamilton, Yoshua Liò, Devon Bengio, Hjelm","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Welling and Kipf, 2017] Max Welling and Thomas N Kipf. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Kipf Welling, Max Welling, Thomas N Kipf","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xia et al., 2022] Jun Xia, Lirong Wu, Jintao Chen, Bozhen Hu, and Stan Z Li. Simgrace: A simple framework for graph contrastive learning without data augmentation. In WWW, pages 1070-1079, 2022.","title":"Simgrace: A simple framework for graph contrastive learning without data augmentation","authors":"Jun Xia, Lirong Wu, Jintao Chen, Bozhen Hu, Stan Z Li","year":"2022","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural net- works? In ICLR, 2019.","title":"How powerful are graph neural networks?","authors":"Xu","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yanardag and Vishwanathan, 2015] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In KDD, pages 1365-1374, 2015.","title":"Pinar Yanardag and SVN Vishwanathan. Deep graph kernels","authors":"Vishwanathan Yanardag","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph con- trastive learning with augmentations. In NeurIPS, pages 5812-5823, 2020.","title":"Graph contrastive learning with augmentations","authors":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[You et al., 2021] Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning auto- mated. In ICML, pages 12121-12132, 2021.","title":"Graph contrastive learning automated","authors":"You","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2019] Guo Zhang, Hao He, and Dina Katabi. Circuit-gnn: Graph neural networks for distributed circuit design. In ICML, pages 7364-7373, 2019.","title":"Circuit-gnn: Graph neural networks for distributed circuit design","authors":"Guo Zhang, Hao He, Dina Katabi","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhou et al., 2020] Fan Zhou, Qing Yang, Ting Zhong, Da- jiang Chen, and Ning Zhang. Variational graph neural net- works for road traffic prediction in intelligent transporta- tion systems. IEEE Transactions on Industrial Informat- ics, 17(4):2802-2812, 2020.","title":"Variational graph neural networks for road traffic prediction in intelligent transportation systems","authors":"Fan Zhou, Qing Yang, Ting Zhong, Dajiang Chen, Ning Zhang","year":"2020","venue":"IEEE Transactions on Industrial Informatics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}