{"type":"ScholarlyArticle","title":"Bootstrapping Informative Graph Augmentation via A Meta Learning Approach","authors":[{"name":"Hang Gao","slug":"hang-gao","given":"Hang","family":"Gao","affiliation":"University of Chinese Academy of Sciences; Institute of Software Chinese Academy of Sciences"},{"name":"Jiangmeng Li","slug":"jiangmeng-li","given":"Jiangmeng","family":"Li","affiliation":"University of Chinese Academy of Sciences; Institute of Software Chinese Academy of Sciences"},{"name":"Wenwen Qiang","slug":"wenwen-qiang","given":"Wenwen","family":"Qiang","affiliation":"University of Chinese Academy of Sciences; Institute of Software Chinese Academy of Sciences"},{"name":"Lingyu Si","slug":"lingyu-si","given":"Lingyu","family":"Si","affiliation":"University of Chinese Academy of Sciences; Institute of Software Chinese Academy of Sciences"},{"name":"Fuchun Sun","slug":"fuchun-sun","given":"Fuchun","family":"Sun","affiliation":"Tsinghua University"},{"name":"Changwen Zheng","slug":"changwen-zheng","given":"Changwen","family":"Zheng","affiliation":"Institute of Software Chinese Academy of Sciences"}],"abstract":"Recent works explore learning graph representations in a self-supervised manner. In graph contrastive learning, benchmark methods apply various graph augmentation approaches. However, most of the augmentation methods are non-learnable, which causes the issue of generating unbeneficial augmented graphs. Such augmentation may degenerate the representation ability of graph contrastive learning methods. Therefore, we motivate our method to generate augmented graph with a learnable graph augmenter, called MEta Graph Augmentation (MEGA). We then clarify that a \"good\" graph augmentation must have uniformity at the instance-level and informativeness at the feature-level. To this end, we propose a novel approach to learning a graph augmenter that can generate an augmentation with uniformity and informativeness. The objective of the graph augmenter is to promote our feature extraction network to learn a more discriminative feature representation, which motivates us to propose a meta-learning paradigm. Empirically, the experiments across multiple benchmark datasets demonstrate that MEGA outperforms the state-of-the-art methods in graph self-supervised learning tasks. Further experimental studies prove the effectiveness of different terms of MEGA. Our codes are available at https://github.com/hang53/MEGA.","doi":"10.24963/ijcai.2022/416","url":"https://ijcai.org/proceedings/2022/416","pdf":"https://ijcai.org/proceedings/2022/0416.pdf","markdown":"https://ijcai.org/proceedings/2022/416.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/416","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":416,"pages":{"first":3001,"last":3007},"keywords":["Machine Learning: Self-supervised Learning","Machine Learning: Meta-Learning","Machine Learning: Representation learning","Machine Learning: Sequence and Graph Learning","Machine Learning: Unsupervised Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986615","length_sec":72,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984758","length_sec":353,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Adhikari et al., 2018] Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, and B Aditya Prakash. Sub2vec: Fea- ture learning for subgraphs. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2018.","title":"Sub2vec: Feature learning for subgraphs","authors":"Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, Aditya Prakash","year":"2018","venue":"Pacific-Asia Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Inter- national conference on machine learning. PMLR, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chuang et al., 2020] Ching-Yao Chuang, Joshua Robin- son, Lin Yen-Chen, Antonio Torralba, and Stefanie Jegelka. Debiased contrastive learning. arXiv preprint arXiv:2007.00224, 2020.","title":"Debiased contrastive learning","authors":"Ching-Yao Chuang, Joshua Robinson, Lin Yen-Chen, Antonio Torralba, Stefanie Jegelka","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In Proceedings of the 34th ICML. PMLR, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"Proceedings of the 34th ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Grill et al., 2020] Jean-Bastien Grill, Florian Strub, Flo- rent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhao- han Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learn- ing. Advances in Neural Information Processing Systems, 33:21271-21284, 2020.","title":"Bootstrap your own latent-a new approach to self-supervised learning","authors":"Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855-864, 2016.","title":"node2vec: Scalable feature learning for networks","authors":"Leskovec Grover, Aditya Grover, Jure Leskovec","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hassani and Khasahmadi, 2020] Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view repre- sentation learning on graphs. In International Conference on Machine Learning, pages 4116-4126. PMLR, 2020.","title":"Contrastive multi-view representation learning on graphs","authors":"Kaveh Hassani, Amir Hosein, Khasahmadi","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2020a] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Neural Information Process- ing Systems (NeurIPS), 2020.","title":"Open graph benchmark: Datasets for machine learning on graphs","authors":"Hu","year":"2020","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2020b] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural net- works. In International Conference on Learning Repre- sentations (ICLR), 2020.","title":"Strategies for pre-training graph neural networks","authors":"Hu","year":"2020","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kriege et al., 2020] Nils M Kriege, Fredrik D Johansson, and Christopher Morris. A survey on graph kernels. Ap- plied Network Science, 5(1):1-42, 2020.","title":"A survey on graph kernels","authors":"Nils M Kriege, Christopher Fredrik D Johansson, Morris","year":"2020","venue":"Applied Network Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2019] Shikun Liu, Andrew Davison, and Edward Johns. Self-supervised generalisation with meta auxiliary learning. Advances in Neural Information Processing Sys- tems, 32, 2019.","title":"Self-supervised generalisation with meta auxiliary learning","authors":"Liu","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Morris et al., ] Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neu- mann. Tudataset: A collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop on Graph Representation Learning and Beyond. [Oord et al., 2018]","title":"Tudataset: A collection of benchmark datasets for learning with graphs","authors":"Morris","year":"2018","venue":"ICML 2020 Workshop on Graph Representation Learning and Beyond","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Aaron Van Den Oord, Yazhe Li, Oriol Vinyals","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Schmidhuber, 2014] Jürgen Schmidhuber. Learning com- plex, extended sequences using the principle of history compression. Neural Computation, 4(2):234-242, 2014.","title":"Learning complex, extended sequences using the principle of history compression","authors":"Jürgen Schmidhuber","year":"2014","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sun et al., 2020] Fan-Yun Sun, Jordon Hoffman, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, 2020.","title":"Infograph: Unsupervised and semisupervised graph-level representation learning via mutual information maximization","authors":"Fan-Yun Sun, Jordon Hoffman, Vikas Verma, Jian Tang","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Suresh et al., 2021] Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to im- prove graph contrastive learning. Advances in Neural In- formation Processing Systems, 34, 2021.","title":"Adversarial graph augmentation to improve graph contrastive learning","authors":"Susheel Suresh, Pan Li, Cong Hao, Jennifer Neville","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tian et al., 2020] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XI 16, pages 776-794. Springer, 2020.","title":"Contrastive multiview coding","authors":"Tian","year":"2020","venue":"Computer Vision-ECCV 2020: 16th European Conference","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Veličković et al., 2018] Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. arXiv preprint arXiv:1809.10341, 2018.","title":"Deep graph infomax","authors":"Petar Veličković, William Fedus, Pietro William L Hamilton, Yoshua Liò, Devon Bengio, Hjelm","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wan et al., 2020] Sheng Wan, Shirui Pan, Jian Yang, and Chen Gong. Contrastive and generative graph convolu- tional networks for graph-based semi-supervised learning. arXiv preprint arXiv:2009.07111, 2020.","title":"Contrastive and generative graph convolutional networks for graph-based semi-supervised learning","authors":"Sheng Wan, Shirui Pan, Jian Yang, Chen Gong","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2018] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural net- works? In International Conference on Learning Rep- resentations, 2018.","title":"How powerful are graph neural networks","authors":"Xu","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph con- trastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812-5823, 2020.","title":"Graph contrastive learning with augmentations","authors":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zbontar et al., 2021] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In Inter- national Conference on Machine Learning, pages 12310- 12320. PMLR, 2021.","title":"Barlow twins: Selfsupervised learning via redundancy reduction","authors":"Jure Zbontar, Li Jing, Ishan Misra, Yann Lecun, Stéphane Deny","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}