{"type":"ScholarlyArticle","title":"Proximity Enhanced Graph Neural Networks with Channel Contrast","authors":[{"name":"Wei Zhuo","slug":"wei-zhuo","given":"Wei","family":"Zhuo","affiliation":"Shenzhen Campus of Sun Yat-sen University , China"},{"name":"Guang Tan","slug":"guang-tan","given":"Guang","family":"Tan","affiliation":"Shenzhen Campus of Sun Yat-sen University , China"}],"abstract":"We consider graph representation learning in an unsupervised manner. Graph neural networks use neighborhood aggregation as a core component that results in feature smoothing among nodes in proximity. While successful in various prediction tasks, such a paradigm falls short of capturing nodes' similarities over a long distance, which proves to be important for high-quality learning. To tackle this problem, we strengthen the graph with three types of additional graph views, in which each node is directly linked to a set of nodes with the highest similarity in terms of node features, neighborhood features or local structures. Not restricted by connectivity in the original graph, the generated views provide new and complementary perspectives from which to look at the relationship between nodes. Inspired by the recent success of contrastive learning approaches, we propose a self-supervised method that aims to learn node representations by maximizing the agreement between representations across generated views and the original graph, without the requirement of any label information. We also propose a channel-level contrast approach that greatly reduces computation cost. Extensive experiments on six assortative graphs and three disassortative graphs demonstrate the effectiveness of our approach.","doi":"10.24963/ijcai.2022/340","url":"https://ijcai.org/proceedings/2022/340","pdf":"https://ijcai.org/proceedings/2022/0340.pdf","markdown":"https://ijcai.org/proceedings/2022/340.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/340","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":340,"pages":{"first":2448,"last":2455},"keywords":["Machine Learning: Self-supervised Learning","Data Mining: Mining Graphs"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986890","length_sec":96,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985033","length_sec":320,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, et al. A simple framework for contrastive learning of visual representations. In ICML, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2021] Yuzhou Chen, Baris Coskunuzer, and Yulia Gel. Topological relational learning on graphs. NeurIPS, 34, 2021.","title":"Topological relational learning on graphs","authors":"Chen","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In SIGKDD, 2016.","title":"node2vec: Scalable feature learning for networks","authors":"Leskovec Grover, Aditya Grover, Jure Leskovec","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hamilton et al., 2017] William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"William L Hamilton, Rex Ying, Jure Leskovec","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hassani and Khasahmadi, 2020] Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. In ICML, 2020.","title":"Contrastive multi-view representation learning on graphs","authors":"Kaveh Hassani, Amir Hosein, Khasahmadi","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Henderson et al., 2012] Keith Henderson, Brian Gallagher, Tina Eliassi-Rad, et al. Rolx: structural role extraction & mining in large graphs. In SIGKDD, 2012.","title":"Rolx: structural role extraction & mining in large graphs","authors":"Keith Henderson, Brian Gallagher, Tina Eliassi-Rad","year":"2012","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hu et al., 2020] Ziniu Hu, Yuxiao Dong, Kuansan Wang, et al. Gpt-gnn: Generative pre-training of graph neural networks. In SIGKDD, 2020.","title":"Gpt-gnn: Generative pre-training of graph neural networks","authors":"Hu","year":"2020","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Variational graph auto-encoders. NIPS Workshop on Bayesian Deep Learning, 2016.","title":"Variational graph auto-encoders","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":"NIPS Workshop on Bayesian Deep Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kong et al., 2019] Lingpeng Kong, Cyprien de Masson d'Autume, Wang Ling, et al. A mutual information max- imization perspective of language representation learning. arXiv preprint arXiv:1910.08350, 2019.","title":"A mutual information maximization perspective of language representation learning","authors":"Kong","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ma et al., 2021] Yao Ma, Xiaorui Liu, Neil Shah, et al. Is homophily a necessity for graph neural networks? arXiv preprint arXiv:2106.06134, 2021.","title":"Is homophily a necessity for graph neural networks","authors":"Yao Ma, Xiaorui Liu, Neil Shah","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mernyei and Cangea, 2020] Péter Mernyei and Cȃtȃlina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks. arXiv preprint arXiv:2007.02901, 2020.","title":"Péter Mernyei and Cȃtȃlina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks","authors":"Cangea Mernyei","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nikolentzos et al., 2018] Giannis Nikolentzos, Polykarpos Meladianos, Antoine Jean-Pierre Tixier, et al. Kernel graph convolutional neural networks. In ICANN, 2018.","title":"Kernel graph convolutional neural networks","authors":"Giannis Nikolentzos, Polykarpos Meladianos, Antoine, Jean-Pierre Tixier","year":"2018","venue":"ICANN","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Park et al., 2019] Jiwoong Park, Minsik Lee, Hyung Jin Chang, et al. Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In IEEE ICCV, 2019.","title":"Symmetric graph convolutional autoencoder for unsupervised graph representation learning","authors":"Park","year":"2019","venue":"IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Park et al., 2020] Chanyoung Park, Donghyun Kim, Jiawei Han, and Hwanjo Yu. Unsupervised attributed multiplex network embedding. In AAAI, 2020.","title":"Unsupervised attributed multiplex network embedding","authors":"Park","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pei et al., 2019] Hongbin Pei, Bingzhe Wei, Kevin Chen- Chuan Chang, et al. Geom-gcn: Geometric graph con- volutional networks. In ICLR, 2019.","title":"Geom-gcn: Geometric graph convolutional networks","authors":"Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan, Chang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pele and Werman, 2009] Ofir Pele and Michael Werman. Fast and robust earth mover's distances. In IEEE ICCV, 2009.","title":"Fast and robust earth mover's distances","authors":"Werman Pele, Ofir Pele, Michael Werman","year":"2009","venue":"IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Peng et al., 2020] Zhen Peng, Wenbing Huang, Minnan Luo, et al. Graph representation learning via graphical mu- tual information maximization. In The Web Conf., 2020.","title":"Graph representation learning via graphical mutual information maximization","authors":"Peng","year":"2020","venue":"The Web Conf","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In SIGKDD, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ribeiro et al., 2017] Leonardo FR Ribeiro, Pedro HP Saverese, and Daniel R Figueiredo. struc2vec: Learning node representations from structural identity. In SIGKDD, 2017.","title":"Daniel R Figueiredo. struc2vec: Learning node representations from structural identity","authors":"Ribeiro","year":"2017","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shervashidze et al., 2011] Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, et al. Weisfeiler- lehman graph kernels. Journal of Machine Learning Research, 2011.","title":"Weisfeilerlehman graph kernels","authors":"Nino Shervashidze, Pascal Schweitzer, Erik, Jan Van Leeuwen","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sinha et al., 2015] Arnab Sinha, Zhihong Shen, Yang Song, et al. An overview of microsoft academic service (mas) and applications. In WWW, 2015.","title":"An overview of microsoft academic service (mas) and applications","authors":"Arnab Sinha, Zhihong Shen, Yang Song","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tang et al., 2015] Jian Tang, Meng Qu, Mingzhe Wang, et al. Line: Large-scale information network embedding. In WWW, 2015.","title":"Line: Large-scale information network embedding","authors":"Jian Tang, Meng Qu, Mingzhe Wang","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, et al. Graph attention networks. ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Veličković et al., 2019] Petar Veličković, William Fedus, William L. Hamilton, et al. Deep graph infomax. In ICLR, 2019.","title":"Deep graph infomax","authors":"Petar Veličković, William Fedus, William L Hamilton","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2020] Xiao Wang, Meiqi Zhu, Deyu Bo, et al. Am-gcn: Adaptive multi-channel graph convolutional net- works. In SIGKDD, 2020.","title":"Am-gcn: Adaptive multi-channel graph convolutional networks","authors":"Xiao Wang, Meiqi Zhu, Deyu Bo","year":"2020","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ying et al., 2018] Zhitao Ying, Jiaxuan You, Christopher Morris, et al. Hierarchical graph representation learning with differentiable pooling. In NeurIPS, 2018.","title":"Hierarchical graph representation learning with differentiable pooling","authors":"Ying","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, et al. Graph contrastive learning with augmentations. NeurIPS, 2020.","title":"Graph contrastive learning with augmentations","authors":"You","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Chen, 2018] Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. In NeurIPS, 2018.","title":"Link prediction based on graph neural networks","authors":"Chen ; ] Muhan Zhang, Yixin Zhang, Chen","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2013] Yan-Ming Zhang, Kaizhu Huang, Guanggang Geng, et al. Fast knn graph construction with locality sensitive hashing. In ECML PKDD, 2013.","title":"Fast knn graph construction with locality sensitive hashing","authors":"Yan-Ming Zhang, Kaizhu Huang, Guanggang Geng","year":"2013","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhu et al., 2020a] Yanqiao Zhu, Yichen Xu, Feng Yu, et al. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131, 2020.","title":"Deep graph contrastive representation learning","authors":"Zhu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhu et al., 2020b] Yanqiao Zhu, Yichen Xu, Feng Yu, et al. Graph contrastive learning with adaptive augmentation. arXiv preprint arXiv:2010.14945, 2020.","title":"Graph contrastive learning with adaptive augmentation","authors":"Zhu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}