{"type":"ScholarlyArticle","title":"Stabilizing and Enhancing Link Prediction through Deepened Graph Auto-Encoders","authors":[{"name":"Xinxing wu","slug":"xinxing-wu","given":"Xinxing","family":"wu","affiliation":"University of Kentucky , Lexington , Kentucky , U.S.A."},{"name":"Qiang Cheng","slug":"qiang-cheng","given":"Qiang","family":"Cheng","affiliation":"University of Kentucky , Lexington , Kentucky , U.S.A."}],"abstract":"Graph neural networks have been widely used for a variety of learning tasks. Link prediction is a relatively under-studied graph learning task, with current state-of-the-art models based on one- or two-layer shallow graph auto-encoder (GAE) architectures. In this paper, we overcome the limitation of current methods for link prediction of non-Euclidean network data, which can only use shallow GAEs and variational GAEs. Our proposed methods innovatively incorporate standard auto-encoders (AEs) into the architectures of GAEs to capitalize on the intimate coupling of node and edge information in complex network data. Empirically, extensive experiments on various datasets demonstrate the competitive performance of our proposed approach. Theoretically, we prove that our deep extensions can inclusively express multiple polynomial filters with different orders. The codes of this paper are available at https://github.com/xinxingwu-uk/DGAE.","doi":"10.24963/ijcai.2022/498","url":"https://ijcai.org/proceedings/2022/498","pdf":"https://ijcai.org/proceedings/2022/0498.pdf","markdown":"https://ijcai.org/proceedings/2022/498.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/498","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":498,"pages":{"first":3587,"last":3593},"keywords":["Machine Learning: Learning Graphical Models","Machine Learning: Relational Learning","Machine Learning: Representation learning","Data Mining: Networks"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986574","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984717","length_sec":303,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bengio et al., 2006] Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. Greedy layer-wise train- ing of deep networks. In NeurIPS, 2006.","title":"Greedy layer-wise training of deep networks","authors":"Yoshua Bengio, Pascal Lamblin, Dan Popovici, Hugo Larochelle","year":"2006","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In ICML, 2020.","title":"Simple and deep graph convolutional networks","authors":"Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, Yaliang Li","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. JMLR, 9(11):2579-2605, 2008.","title":"Visualizing data using t-SNE","authors":"Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fortunato, 2010] Santo Fortunato. Community detection in graphs. Phys. Rep., 486(3-5):75-174, 2010.","title":"Community detection in graphs","authors":"Santo Fortunato","year":"2010","venue":"Phys. Rep","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gasteiger et al., 2019] Johannes Gasteiger, Stefan Weißen- berger, and Stephan Günnemann. Diffusion improves graph learning. In NeurIPS, 2019.","title":"Diffusion improves graph learning","authors":"Gasteiger","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gong and Cheng, 2019] Liyu Gong and Qiang Cheng. Ex- ploiting edge features for graph neural networks. In CVPR, 2019.","title":"Exploiting edge features for graph neural networks","authors":"Cheng ; ] Liyu Gong, Qiang Gong, Cheng","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Grover et al., 2019] Aditya Grover, Aaron Zweig, and Ste- fano Ermon. Graphite: Iterative generative modeling of graphs. In ICML, 2019.","title":"Graphite: Iterative generative modeling of graphs","authors":"Aditya Grover, Aaron Zweig, Stefano Ermon","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hardt and Ma, 2017] Moritz Hardt and Tengyu Ma. Identity matters in deep learning. In ICLR, 2017.","title":"Identity matters in deep learning","authors":"Ma ; Hardt, Moritz Hardt, Tengyu Ma","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hinton and Salakhutdinov, 2006] Geoffrey E. Hinton and Ruslan R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Sci., 313(5786):504-507, 2006.","title":"Reducing the dimensionality of data with neural networks","authors":"Geoffrey E Hinton, Ruslan R Salakhutdinov","year":"2006","venue":"Sci","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hinton et al., 2012] Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhut- dinov. Improving neural networks by preventing co- adaptation of feature detectors. arXiv:1207.0580, 2012.","title":"Improving neural networks by preventing coadaptation of feature detectors","authors":"Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R Salakhutdinov","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2016] Thomas N. Kipf and Max Welling. Variational graph auto-encoders. arXiv:1611.07308, 2016.","title":"Variational graph auto-encoders","authors":"Thomas N Kipf, Max Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proc. IEEE, 86(11):2278- 2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proc. IEEE","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2018] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, 2018.","title":"Deeper insights into graph convolutional networks for semi-supervised learning","authors":"Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liben-Nowell and Kleinberg, 2003] David Liben-Nowell and Jon Kleinberg. The link-prediction problem for social networks. In CIKM, 2003.","title":"The link-prediction problem for social networks","authors":"Liben-Nowell, Kleinberg, David Liben-Nowell, Jon Kleinberg","year":"2003","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pan et al., 2018] Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. Adversarially regu- larized graph autoencoder for graph embedding. In IJCAI, 2018.","title":"Adversarially regularized graph autoencoder for graph embedding","authors":"Pan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/362"},{"position":18,"raw":"[Pei et al., 2020] Hongbin Pei, Bingzhe Wei, Kevin Chen- Chuan Chang, Yu Lei, and Bo Yang. Geom-GCN: Geo- metric graph convolutional networks. In ICLR, 2020.","title":"Geom-GCN: Geometric graph convolutional networks","authors":"Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan, Yu Chang, Bo Lei, Yang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In SIGKDD, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rong et al., 2020] Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph con- volutional networks on node classification. In ICLR, 2020.","title":"Dropedge: Towards deep graph convolutional networks on node classification","authors":"Yu Rong, Wenbing Huang, Tingyang Xu, Junzhou Huang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rozemberczki et al., 2020] Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale attributed node em- bedding. arXiv:1909.13021, 2020.","title":"Multi-scale attributed node embedding","authors":"Carl Benedek Rozemberczki, Rik Allen, Sarkar","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Salha et al., 2020] Guillaume Salha, Romain Hennequin, and Michalis Vazirgiannis. Simple and effective graph au- toencoders with one-hop linear model. In ECML PKDD, 2020.","title":"Simple and effective graph autoencoders with one-hop linear model","authors":"Guillaume Salha, Romain Hennequin, Michalis Vazirgiannis","year":"2020","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Scarselli et al., 2009] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Trans. Neural. Netw., 20(1):61-80, 2009.","title":"The graph neural network model","authors":"Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, Gabriele Monfardini","year":"2009","venue":"IEEE Trans. Neural. Netw","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Eliassi- Rad. Collective classification in network data. AI Mag., 29(3):93-106, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, Tina Eliassi-Rad","year":"2008","venue":"AI Mag","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tang and Liu, 2011] Lei Tang and Huan Liu. Leveraging social media networks for classification. Data Min. Knowl. Discov., 23(3):447-478, 2011.","title":"Leveraging social media networks for classification","authors":"Lei Tang, Huan Liu","year":"2011","venue":"Data Min. Knowl. Discov","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2016] Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. In SIGKDD, 2016.","title":"Structural deep network embedding","authors":"Wang","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2017] Hao Wang, Xingjian Shi, and Dit-Yan Yeung. Relational deep learning: A deep latent variable model for link prediction. In AAAI, 2017.","title":"Relational deep learning: A deep latent variable model for link prediction","authors":"Hao Wang, Xingjian Shi, Dit-Yan Yeung","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2019] Guangtao Wang, Rex Ying, Jing Huang, and Jure Leskovec. Improving graph attention networks with large margin-based constraints. arXiv:1910.11945, 2019.","title":"Improving graph attention networks with large margin-based constraints","authors":"Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wu et al., 2019] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In ICML, 2019.","title":"Simplifying graph convolutional networks","authors":"Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, Kilian Weinberger","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2018] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In ICML, 2018.","title":"Representation learning on graphs with jumping knowledge networks","authors":"Xu","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}