{"type":"ScholarlyArticle","title":"A Degeneracy Framework for Scalable Graph Autoencoders","authors":[{"name":"Guillaume Salha","slug":"guillaume-salha","given":"Guillaume","family":"Salha","affiliation":"Deezer Research & Development , Paris , France; École Polytechnique , Palaiseau , France"},{"name":"Romain Hennequin","slug":"romain-hennequin","given":"Romain","family":"Hennequin","affiliation":"Deezer Research & Development , Paris , France"},{"name":"Viet Anh Tran","slug":"viet-anh-tran","given":"Viet Anh","family":"Tran","affiliation":"Deezer Research & Development , Paris , France"},{"name":"Michalis Vazirgiannis","slug":"michalis-vazirgiannis","given":"Michalis","family":"Vazirgiannis","affiliation":"École Polytechnique , Palaiseau , France"}],"abstract":"In this paper, we present a general framework to scale graph autoencoders (AE) and graph variational autoencoders (VAE). This framework leverages graph degeneracy concepts to train models only from a dense subset of nodes instead of using the entire graph. Together with a simple yet effective propagation mechanism, our approach significantly improves scalability and training speed while preserving performance. We evaluate and discuss our method on several variants of existing graph AE and VAE, providing the first application of these models to large graphs with up to millions of nodes and edges. We achieve empirically competitive results w.r.t. several popular scalable node embedding methods, which emphasizes the relevance of pursuing further research towards more scalable graph AE and VAE.","doi":"10.24963/ijcai.2019/465","url":"https://ijcai.org/proceedings/2019/465","pdf":"https://ijcai.org/proceedings/2019/0465.pdf","markdown":"https://ijcai.org/proceedings/2019/465.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/465","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":465,"pages":{"first":3353,"last":3359},"keywords":["Machine Learning: Data Mining","Machine Learning: Unsupervised Learning","Machine Learning Applications: Networks","Machine Learning Applications: Big data ; Scalability"],"videos":[],"references":[{"position":1,"raw":"[Baldi, 2012] Pierre Baldi. Autoencoders, unsupervised learning, and deep architectures. ICML workshop on un- supervised and transfer learning, 2012.","title":"Autoencoders, unsupervised learning, and deep architectures","authors":"Pierre Baldi","year":"2012","venue":"ICML workshop on unsupervised and transfer learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Batagelj and Zaversnik, 2003] Vladimir Batagelj and Mat- jaz Zaversnik. An o(m) algorithm for cores decomposition of networks. arXiv preprint cs/0310049, 2003.","title":"An o(m) algorithm for cores decomposition of networks","authors":"Vladimir Batagelj, Matjaz Zaversnik","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berg et al., 2018] Rianne van den Berg, Thomas N Kipf, and Max Welling. Graph convolutional matrix completion. KDD Deep Learning day, 2018.","title":"Graph convolutional matrix completion","authors":"Rianne Van Den, Thomas N Berg, Max Kipf, Welling","year":"2018","venue":"KDD Deep Learning day","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blondel et al., 2008] Vincent Blondel, Jean-Loup Guil- laume, and Etienne. Lefebvre. Fast unfolding of commu- nities in large networks. J. Stat. Mech, 2008.","title":"Fast unfolding of communities in large networks","authors":"Vincent Blondel, Jean-Loup Guillaume, Etienne Lefebvre","year":"2008","venue":"J. Stat. Mech","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bruna et al., 2014] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. Spectral networks and locally connected networks on graphs. ICLR, 2014.","title":"Spectral networks and locally connected networks on graphs","authors":"Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann Lecun","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chakrabarti and Faloutsos, 2006] Deepayan Chakrabarti and Christos Faloutsos. Graph mining: Laws, generators, and algorithms. ACM computing surveys, 38(1):2, 2006.","title":"Deepayan Chakrabarti and Christos Faloutsos. Graph mining: Laws, generators, and algorithms","authors":"Faloutsos Chakrabarti","year":"2006","venue":"ACM computing surveys","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2018] Jie Chen, Tengfei Ma, and Cao Xiao. Fastgcn: fast learning with graph convolutional networks via importance sampling. ICLR, 2018.","title":"Fastgcn: fast learning with graph convolutional networks via importance sampling","authors":"Jie Chen, Tengfei Ma, Cao Xiao","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. NIPS, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Giatsidis et al., 2014] Christos Giatsidis, Fragkiskos Malliaros, Dimitrios Thilikos, and Michalis Vazirgian- nis. Corecluster: A degeneracy based graph clustering framework. AAAI, 2014.","title":"Corecluster: A degeneracy based graph clustering framework","authors":"Christos Giatsidis, Fragkiskos Malliaros, Dimitrios Thilikos, Michalis Vazirgiannis","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gori et al., 2005] Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph do- mains. IJCNN, 2005.","title":"A new model for learning in graph domains","authors":"Marco Gori, Gabriele Monfardini, Franco Scarselli","year":"2005","venue":"IJCNN","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. SIGKDD, 2016.","title":"node2vec: Scalable feature learning for networks","authors":"Leskovec Grover, Aditya Grover, Jure Leskovec","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Grover et al., 2018] Aditya Grover, Aaron Zweig, and Ste- fano Ermon. Graphite: Iterative generative modeling of graphs. arXiv preprint arXiv:1803.10459, 2018.","title":"Graphite: Iterative generative modeling of graphs","authors":"Aditya Grover, Aaron Zweig, Stefano Ermon","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. NIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. ICLR, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kipf and Welling, 2016a] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICLR, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"Welling ; Kipf, N Thomas, Max Kipf, Welling","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kipf and Welling, 2016b] Thomas N Kipf and Max Welling. Variational graph auto-encoders. NIPS Work- shop on Bayesian Deep Learning, 2016.","title":"Variational graph auto-encoders","authors":"Welling ; Kipf, N Thomas, Max Kipf, Welling","year":"2016","venue":"NIPS Workshop on Bayesian Deep Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lovász, 2007] László Lovász. Eigenvalues of graphs. Tech- nical report, Eotvos Lorand University, 2007.","title":"Eigenvalues of graphs","authors":"László Lovász, Lovász","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ma et al., 2018] Tengfei Ma, Jie Chen, and Cao Xiao. Con- strained generation of semantically valid graphs via regu- larizing variational autoencoders. NeurIPS, 2018.","title":"Constrained generation of semantically valid graphs via regularizing variational autoencoders","authors":"Ma","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Malliaros et al., 2019] Fragkiskos Malliaros, Christos Giat- sidis, Apostolos Papadopoulos, and Michalis Vazirgiannis. The core decomposition of networks: Theory, algorithms and applications. 2019.","title":"Fragkiskos Malliaros, Christos Giatsidis, Apostolos Papadopoulos, and Michalis Vazirgiannis","authors":"Malliaros","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nikolentzos et al., 2018] Giannis Nikolentzos, Polykarpos Meladianos, Stratis Limnios, and Michalis Vazirgiannis. A degeneracy framework for graph similarity. IJCAI, 2018.","title":"A degeneracy framework for graph similarity","authors":"Giannis Nikolentzos, Polykarpos Meladianos, Stratis Limnios, Michalis Vazirgiannis","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/360"},{"position":21,"raw":"[Pan et al., 2018] Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. Adversarially regu- larized graph autoencoder. IJCAI, 2018.","title":"Adversarially regularized graph autoencoder","authors":"Pan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. SIGKDD, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rumelhart et al., 1986] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning internal repre- sentations by error propagation. Parallel Distributed Pro- cessing, Vol 1, 1986.","title":"Learning internal representations by error propagation","authors":"Geoffrey E David E Rumelhart, Ronald J Hinton, Williams","year":"1986","venue":"Parallel Distributed Processing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Scarselli et al., 2009] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. Neural Networks, 20(1):61-80, 2009.","title":"The graph neural network model","authors":"Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, Gabriele Monfardini","year":"2009","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Simonovsky and Komodakis, 2018] Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using vae. ICANN, 2018.","title":"Graphvae: Towards generation of small graphs using vae","authors":"Martin Simonovsky, Nikos Komodakis","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Taheri et al., 2018] Aynaz Taheri, Kevin Gimpel, and Tanya Berger-Wolf. Learning graph representations with recur- rent neural network autoencoders. KDD DL Day, 2018.","title":"Learning graph representations with recurrent neural network autoencoders","authors":"Taheri","year":"2018","venue":"KDD DL Day","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tang et al., 2015] Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large- scale information network embedding. WWW, 2015.","title":"Line: Largescale information network embedding","authors":"Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, Qiaozhu Mei","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tixier et al., 2016] Antoine Tixier, Fragkiskos Malliaros, and Michalis Vazirgiannis. A graph degeneracy-based ap- proach to keyword extraction. EMNLP, 2016.","title":"A graph degeneracy-based approach to keyword extraction","authors":"Tixier","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tran, 2018] Phi Vu Tran. Learning to make predictions on graphs with autoencoders. DSAA, 2018.","title":"Phi Vu Tran. Learning to make predictions on graphs with autoencoders","authors":"Tran","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tschannen et al., 2018] Michael Tschannen, Olivier Bachem, and Mario Lucic. Recent advances in autoencoder-based representation learning. NeurIPS Bayesian DL workshop, 2018.","title":"Recent advances in autoencoder-based representation learning","authors":"Olivier Michael Tschannen, Mario Bachem, Lucic","year":"2018","venue":"NeurIPS Bayesian DL workshop","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2016] Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. SIGKDD, 2016.","title":"Structural deep network embedding","authors":"Wang","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2017] Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. Mgae: Marginalized graph autoencoder for graph clustering. CIKM, 2017.","title":"Mgae: Marginalized graph autoencoder for graph clustering","authors":"Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, Jing Jiang","year":"2017","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wu et al., 2019] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu. A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596, 2019.","title":"A comprehensive survey on graph neural networks","authors":"Wu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xu et al., 2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural net- works? ICLR, 2019.","title":"How powerful are graph neural networks? ICLR","authors":"Xu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}