{"type":"ScholarlyArticle","title":"TGNN: A Joint Semi-supervised Framework for Graph-level Classification","authors":[{"name":"Wei Ju","slug":"wei-ju","given":"Wei","family":"Ju","affiliation":"School of Computer Science , Peking University , China"},{"name":"Xiao Luo","slug":"xiao-luo","given":"Xiao","family":"Luo","affiliation":"School of Mathematical Sciences , Peking University , China"},{"name":"Meng Qu","slug":"meng-qu","given":"Meng","family":"Qu","affiliation":"Mila -Québec AI Institute , Université de Montréal , Canada"},{"name":"Yifan Wang","slug":"yifan-wang","given":"Yifan","family":"Wang","affiliation":"School of Computer Science , Peking University , China"},{"name":"Chong Chen","slug":"chong-chen","given":"Chong","family":"Chen","affiliation":"DAMO Academy , Alibaba Group , China"},{"name":"Minghua Deng","slug":"minghua-deng","given":"Minghua","family":"Deng","affiliation":"School of Mathematical Sciences , Peking University , China"},{"name":"Xian-Sheng Hua","slug":"xian-sheng-hua","given":"Xian-Sheng","family":"Hua","affiliation":"DAMO Academy , Alibaba Group , China"},{"name":"Ming Zhang","slug":"ming-zhang","given":"Ming","family":"Zhang","affiliation":"School of Computer Science , Peking University , China"}],"abstract":"This paper studies semi-supervised graph classification, a crucial task with a wide range of applications in social network analysis and bioinformatics. Recent works typically adopt graph neural networks to learn graph-level representations for classification, failing to explicitly leverage features derived from graph topology (e.g., paths). Moreover, when labeled data is scarce, these methods are far from satisfactory due to their insufficient topology exploration of unlabeled data. We address the challenge by proposing a novel semi-supervised framework called Twin Graph Neural Network (TGNN). To explore graph structural information from complementary views, our TGNN has a message passing module and a graph kernel module. To fully utilize unlabeled data, for each module, we calculate the similarity of each unlabeled graph to other labeled graphs in the memory bank and our consistency loss encourages consistency between two similarity distributions in different embedding spaces. The two twin modules collaborate with each other by exchanging instance similarity knowledge to fully explore the structure information of both labeled and unlabeled data. We evaluate our TGNN on various public datasets and show that it achieves strong performance.","doi":"10.24963/ijcai.2022/295","url":"https://ijcai.org/proceedings/2022/295","pdf":"https://ijcai.org/proceedings/2022/0295.pdf","markdown":"https://ijcai.org/proceedings/2022/295.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/295","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":295,"pages":{"first":2122,"last":2128},"keywords":["Data Mining: Mining Graphs","Machine Learning: Semi-Supervised Learning","Data Mining: Mining Semi Structured Data"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986830","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984973","length_sec":351,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Adhikari et al., 2018] Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, and B Aditya Prakash. Sub2vec: Feature learning for subgraphs. In PAKDD, 2018.","title":"Sub2vec: Feature learning for subgraphs","authors":"Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, Aditya Prakash","year":"2018","venue":"PAKDD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Dexiong Chen, Laurent Jacob, and Julien Mairal. Convolutional kernel networks for graph- structured data. In ICML, 2020.","title":"Convolutional kernel networks for graphstructured data","authors":"Chen","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In ICML, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, S Samuel, Patrick F Schoenholz, Oriol Riley, George E Vinyals, Dahl","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Grandvalet and Bengio, 2005] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimiza- tion. In NeurIPS, 2005.","title":"Semi-supervised learning by entropy minimization","authors":"Bengio Grandvalet, Yves Grandvalet, Yoshua Bengio","year":"2005","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hao et al., 2020] Zhongkai Hao, Chengqiang Lu, Zhenya Huang, Hao Wang, Zheyuan Hu, Qi Liu, Enhong Chen, and Cheekong Lee. Asgn: An active semi-supervised graph neural network for molecular property prediction. In KDD, 2020.","title":"Asgn: An active semi-supervised graph neural network for molecular property prediction","authors":"Hao","year":"2020","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hinton et al., 2014] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. In NeurIPS Workshop, 2014.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2014","venue":"NeurIPS Workshop","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ju et al., 2022] Wei Ju, Xiao Luo, Zeyu Ma, Junwei Yang, Minghua Deng, and Ming Zhang. Ghnn: Graph harmonic neural networks for semi-supervised graph-level classifi- cation. Neural Networks, 2022.","title":"Ghnn: Graph harmonic neural networks for semi-supervised graph-level classification","authors":"Wei Ju, Xiao Luo, Zeyu Ma, Junwei Yang, Minghua Deng, Ming Zhang","year":"2022","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kashima et al., 2003] Hisashi Kashima, Koji Tsuda, and Akihiro Inokuchi. Marginalized kernels between labeled graphs. In ICML, 2003.","title":"Marginalized kernels between labeled graphs","authors":"Hisashi Kashima, Koji Tsuda, Akihiro Inokuchi","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR. 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kojima et al., 2020] Ryosuke Kojima, Shoichi Ishida, Masateru Ohta, Hiroaki Iwata, Teruki Honma, and Yasushi Okuno. kgcn: a graph-based deep learning frame- work for chemical structures. Journal of Cheminformatics, 2020.","title":"kgcn: a graph-based deep learning framework for chemical structures","authors":"Ryosuke Kojima, Shoichi Ishida, Masateru Ohta, Hiroaki Iwata, Teruki Honma, Yasushi Okuno","year":"2020","venue":"Journal of Cheminformatics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Laine and Aila, 2017] Samuli Laine and Timo Aila. Tem- poral ensembling for semi-supervised learning. In ICLR, 2017.","title":"Temporal ensembling for semi-supervised learning","authors":"Laine, Aila ;, Samuli Laine, Timo Aila","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee and others, 2013] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML Workshop, 2013.","title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","authors":"Dong-Hyun Lee","year":"2013","venue":"ICML Workshop","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2019] Jia Li, Yu Rong, Hong Cheng, Helen Meng, Wenbing Huang, and Junzhou Huang. Semi-supervised graph classification: A hierarchical graph perspective. In WWW, 2019.","title":"Semi-supervised graph classification: A hierarchical graph perspective","authors":"Jia Li, Yu Rong, Hong Cheng, Helen Meng, Wenbing Huang, Junzhou Huang","year":"2019","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Long et al., 2021] Qingqing Long, Yilun Jin, Yi Wu, and Guojie Song. Theoretically improving graph neural net- works via anonymous walk graph kernels. In WWW, 2021.","title":"Theoretically improving graph neural networks via anonymous walk graph kernels","authors":"Qingqing Long, Yilun Jin, Yi Wu, Guojie Song","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lu et al., 2019] Chengqiang Lu, Qi Liu, Chao Wang, Zhenya Huang, Peize Lin, and Lixin He. Molecular prop- erty prediction: A multilevel quantum interactions model- ing perspective. In AAAI, 2019.","title":"Molecular property prediction: A multilevel quantum interactions modeling perspective","authors":"Chengqiang Lu, Qi Liu, Chao Wang, Zhenya Huang, Peize Lin, Lixin He","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Luo et al., 2022] Xiao Luo, Wei Ju, Meng Qu, Chong Chen, Minghua Deng, Xian-Sheng Hua, and Ming Zhang. Dual- graph: Improving semi-supervised graph classification via dual contrastive learning. In ICDE, 2022.","title":"Dualgraph: Improving semi-supervised graph classification via dual contrastive learning","authors":"Xiao Luo, Wei Ju, Meng Qu, Chong Chen, Minghua Deng, Xian-Sheng Hua, Ming Zhang","year":"2022","venue":"ICDE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Miyato et al., 2018] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE TPAMI, 2018.","title":"Virtual adversarial training: a regularization method for supervised and semi-supervised learning","authors":"Miyato","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Narayanan et al., 2017] Annamalai Narayanan, Mahinthan Chandramohan, Rajasekar Venkatesan, Lihui Chen, Yang Liu, and Shantanu Jaiswal. graph2vec: Learning distributed representations of graphs. arXiv preprint arXiv:1707.05005, 2017.","title":"graph2vec: Learning distributed representations of graphs","authors":"Annamalai Narayanan, Mahinthan Chandramohan, Rajasekar Venkatesan, Lihui Chen, Yang Liu, Shantanu Jaiswal","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nikolentzos and Vazirgiannis, 2020] Giannis Nikolentzos and Michalis Vazirgiannis. Random walk graph neural networks. In NeurIPS, 2020.","title":"Random walk graph neural networks","authors":"Giannis Nikolentzos, Michalis Vazirgiannis","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shervashidze et al., 2011] Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. JMLR, 2011.","title":"Weisfeiler-lehman graph kernels","authors":"Nino Shervashidze, Pascal Schweitzer, Erik, Jan Van Leeuwen, Kurt Mehlhorn, Karsten M Borgwardt","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Spielman, 2007] Daniel A Spielman. Spectral graph theory and its applications. In FOCS, 2007.","title":"Spectral graph theory and its applications","authors":"Daniel A Spielman","year":"2007","venue":"FOCS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun et al., 2020] Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In ICLR, 2020.","title":"Infograph: Unsupervised and semisupervised graph-level representation learning via mutual information maximization","authors":"Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, Jian Tang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. In NeurIPS, 2017.","title":"Mean teachers are better role models: Weightaveraged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR. 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vishwanathan et al., 2010] S Vichy N Vishwanathan, Nicol N Schraudolph, Risi Kondor, and Karsten M Borgwardt. Graph kernels. JMLR, 2010.","title":"Graph kernels","authors":"N Vichy, Nicol N Vishwanathan, Risi Schraudolph, Karsten M Kondor, Borgwardt","year":"2010","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural net- works? In ICLR, 2019.","title":"How powerful are graph neural networks?","authors":"Xu","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yanardag and Vishwanathan, 2015] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In KDD, 2015.","title":"Pinar Yanardag and SVN Vishwanathan. Deep graph kernels","authors":"Vishwanathan Yanardag","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ying et al., 2018] Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differen- tiable pooling. In NeurIPS, 2018.","title":"Hierarchical graph representation learning with differentiable pooling","authors":"Ying","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph con- trastive learning with augmentations. In NeurIPS, 2020.","title":"Graph contrastive learning with augmentations","authors":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[You et al., 2021] Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning auto- mated. In ICML, 2021.","title":"Graph contrastive learning automated","authors":"You","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}