{"type":"ScholarlyArticle","title":"RoSA: A Robust Self-Aligned Framework for Node-Node Graph Contrastive Learning","authors":[{"name":"Yun Zhu","slug":"yun-zhu","given":"Yun","family":"Zhu","affiliation":"Zhejiang University"},{"name":"Jianhao Guo","slug":"jianhao-guo","given":"Jianhao","family":"Guo","affiliation":"Zhejiang University"},{"name":"Fei Wu","slug":"fei-wu","given":"Fei","family":"Wu","affiliation":"Zhejiang University"},{"name":"Siliang Tang","slug":"siliang-tang","given":"Siliang","family":"Tang","affiliation":"Zhejiang University"}],"abstract":"Graph contrastive learning has gained significant progress recently. However, existing works have rarely explored non-aligned node-node contrasting. In this paper, we propose a novel graph contrastive learning method named RoSA that focuses on utilizing non-aligned augmented views for node-level representation learning. First, we leverage the earth mover's distance to model the minimum effort to transform the distribution of one view to the other as our contrastive objective, which does not require alignment between views. Then we introduce adversarial training as an auxiliary method to increase sampling diversity and enhance the robustness of our model. Experimental results show that RoSA outperforms a series of graph contrastive learning frameworks on homophilous, non-homophilous and dynamic graphs, which validates the effectiveness of our work. To the best of our awareness, RoSA is the first work focuses on the non-aligned node-node graph contrastive learning problem. Our codes are available at: https://github.com/ZhuYun97/RoSA","doi":"10.24963/ijcai.2022/527","url":"https://ijcai.org/proceedings/2022/527","pdf":"https://ijcai.org/proceedings/2022/0527.pdf","markdown":"https://ijcai.org/proceedings/2022/527.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/527","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":527,"pages":{"first":3795,"last":3801},"keywords":["Machine Learning: Self-supervised Learning","Machine Learning: Robustness","Machine Learning: Sequence and Graph Learning","Machine Learning: Representation learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986583","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984726","length_sec":339,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey E. Hinton. A simple frame- work for contrastive learning of visual representations. In Proc. of ICML, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E Hinton","year":"2020","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Light- speed computation of optimal transport. In Proc. of NeurIPS, 2013.","title":"Sinkhorn distances: Lightspeed computation of optimal transport","authors":"Marco Cuturi","year":"2013","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gao et al., 2021] Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence embeddings. In Proc. of EMNLP, 2021.","title":"Simcse: Simple contrastive learning of sentence embeddings","authors":"Tianyu Gao, Xingcheng Yao, Danqi Chen","year":"2021","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proc. of ICLR, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Grill et al., 2020] Jean-Bastien Grill, Florian Strub, Flo- rent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Ávila Pires, Zhao- han Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent -A new approach to self-supervised learn- ing. In Proc. of NeurIPS, 2020.","title":"Bootstrap your own latent -A new approach to self-supervised learning","authors":"Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Ávila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hamilton et al., 2017] William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Proc. of ICONIP, 2017.","title":"Inductive representation learning on large graphs","authors":"Rex William L Hamilton, Jure Ying, Leskovec","year":"2017","venue":"Proc. of ICONIP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross B. Girshick. Momentum contrast for unsupervised visual representation learning. In Proc. of CVPR, 2020.","title":"Momentum contrast for unsupervised visual representation learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross B Girshick","year":"2020","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hjelm et al., 2019] R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning deep rep- resentations by mutual information estimation and maxi- mization. In Proc. of ICLR, 2019.","title":"Learning deep representations by mutual information estimation and maximization","authors":"R, Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, Yoshua Bengio","year":"2019","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jiao et al., 2020] Yizhu Jiao, Yun Xiong, Jiawei Zhang, Yao Zhang, Tianqi Zhang, and Yangyong Zhu. Sub-graph contrast for scalable self-supervised graph representation learning. In Proc. of ICDM, 2020.","title":"Sub-graph contrast for scalable self-supervised graph representation learning","authors":"Yizhu Jiao, Yun Xiong, Jiawei Zhang, Yao Zhang, Tianqi Zhang, Yangyong Zhu","year":"2020","venue":"Proc. of ICDM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In Proc. of ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kong et al., 2020] Kezhi Kong, Guohao Li, Mucong Ding, Zuxuan Wu, Chen Zhu, Bernard Ghanem, Gavin Taylor, and Tom Goldstein. Flag: Adversarial data augmentation for graph neural networks. arXiv e-prints, pages arXiv- 2010, 2020.","title":"Flag: Adversarial data augmentation for graph neural networks","authors":"Kong","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2020] Songtao Liu, Zeming Li, and Jian Sun. Self-emd: Self-supervised object detection without ima- genet. arXiv e-prints, pages arXiv-2011, 2020.","title":"Self-emd: Self-supervised object detection without imagenet","authors":"Songtao Liu, Zeming Li, Jian Sun","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In Proc. of ICLR, 2018.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pei et al., 2020] Hongbin Pei, Bingzhe Wei, Kevin Chen- Chuan Chang, Yu Lei, and Bo Yang. Geom-gcn: Geo- metric graph convolutional networks. In Proc. of ICLR, 2020.","title":"Geom-gcn: Geometric graph convolutional networks","authors":"Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan, Yu Chang, Bo Lei, Yang","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Peng et al., 2020] Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Jun- zhou Huang. Graph Representation Learning via Graphi- cal Mutual Information Maximization. In Proc. of WWW, 2020.","title":"Graph Representation Learning via Graphical Mutual Information Maximization","authors":"Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, Junzhou Huang","year":"2020","venue":"Proc. of WWW","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rubner et al., 2000] Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth mover's distance as a met- ric for image retrieval. International journal of computer vision, 2000.","title":"The earth mover's distance as a metric for image retrieval","authors":"Yossi Rubner, Carlo Tomasi, Leonidas J Guibas","year":"2000","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Thakoor et al., 2021] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Rémi Munos, Petar Veličković, and Michal Valko. Bootstrapped representa- tion learning on graphs. In ICLR 2021 Workshop on Geo- metrical and Topological Representation Learning, 2021.","title":"Bootstrapped representation learning on graphs","authors":"Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Rémi Munos, Petar Veličković, Michal Valko","year":"2021","venue":"ICLR 2021 Workshop on Geometrical and Topological Representation Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Van den Oord et al., 2018] Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with con- trastive predictive coding. arXiv e-prints, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Oord Van Den","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Velickovic et al., 2019] Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. Deep graph infomax. In Proc. of ICLR, 2019.","title":"Deep graph infomax","authors":"Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, R, Devon Hjelm","year":"2019","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2021] Lirong Wu, Haitao Lin, Zhangyang Gao, Cheng Tan, and Stan Z Li. Self-supervised on graphs: Contrastive, generative, or predictive. arXiv e-prints, pages arXiv-2105, 2021.","title":"Self-supervised on graphs: Contrastive, generative, or predictive","authors":"Lirong Wu, Haitao Lin, Zhangyang Gao, Cheng Tan, Stan Z Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph con- trastive learning with augmentations. Proc. of NeurIPS, 2020.","title":"Graph contrastive learning with augmentations","authors":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhu et al., 2020a] Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced ad- versarial training for natural language understanding. In Proc. of ICLR, 2020.","title":"Freelb: Enhanced adversarial training for natural language understanding","authors":"Zhu","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhu et al., 2020b] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep Graph Con- trastive Representation Learning. In ICML Workshop on Graph Representation Learning and Beyond, 2020.","title":"Deep Graph Contrastive Representation Learning","authors":"Zhu","year":"2020","venue":"ICML Workshop on Graph Representation Learning and Beyond","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhu et al., 2021a] Yanqiao Zhu, Yichen Xu, Qiang Liu, and Shu Wu. An empirical study of graph contrastive learning. In Proc. of NeurIPS, 2021.","title":"An empirical study of graph contrastive learning","authors":"Zhu","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu et al., 2021b] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In Proc. of WWW, 2021.","title":"Graph contrastive learning with adaptive augmentation","authors":"Zhu","year":"2021","venue":"Proc. of WWW","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}