{"type":"ScholarlyArticle","title":"Variational Graph Embedding and Clustering with Laplacian Eigenmaps","authors":[{"name":"Zitai Chen","slug":"zitai-chen","given":"Zitai","family":"Chen","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China; Guangdong Key Laboratory for Big Data Analysis and Simulation of Public Opinion , School of Communication and Design , Sun Yat-sen University , Guangzhou , China"},{"name":"Chuan Chen","slug":"chuan-chen","given":"Chuan","family":"Chen","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China; Guangdong Key Laboratory for Big Data Analysis and Simulation of Public Opinion , School of Communication and Design , Sun Yat-sen University , Guangzhou , China"},{"name":"Zong Zhang","slug":"zong-zhang","given":"Zong","family":"Zhang","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China"},{"name":"Zibin Zheng","slug":"zibin-zheng","given":"Zibin","family":"Zheng","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China; Guangdong Key Laboratory for Big Data Analysis and Simulation of Public Opinion , School of Communication and Design , Sun Yat-sen University , Guangzhou , China"},{"name":"Qingsong Zou","slug":"qingsong-zou","given":"Qingsong","family":"Zou","affiliation":"School of Data and Computer Science , Sun Yat-sen University , Guangzhou , China; Guangdong Province Key Laboratory of Computational Science , Sun Yat-sen University , Guangzhou , China"}],"abstract":"As a fundamental machine learning problem, graph clustering has facilitated various real-world applications, and tremendous efforts had been devoted to it in the past few decades. However, most of the existing methods like spectral clustering suffer from the sparsity, scalability, robustness and handling high dimensional raw information in clustering. To address this issue, we propose a deep probabilistic model, called Variational Graph Embedding and Clustering with Laplacian Eigenmaps (VGECLE), which learns node embeddings and assigns node clusters simultaneously. It represents each node as a Gaussian distribution to disentangle the true embedding position and the uncertainty from the graph. With a Mixture of Gaussian (MoG) prior, VGECLE is capable of learning an interpretable clustering by the variational inference and generative process. In order to learn the pairwise relationships better, we propose a Teacher-Student mechanism encouraging node to learn a better Gaussian from its instant neighbors in the stochastic gradient descent (SGD) training fashion. By optimizing the graph embedding and the graph clustering problem as a whole, our model can fully take the advantages in their correlation. To our best knowledge, we are the first to tackle graph clustering in a deep probabilistic viewpoint. We perform extensive experiments on both synthetic and real-world networks to corroborate the effectiveness and efficiency of the proposed framework.","doi":"10.24963/ijcai.2019/297","url":"https://ijcai.org/proceedings/2019/297","pdf":"https://ijcai.org/proceedings/2019/0297.pdf","markdown":"https://ijcai.org/proceedings/2019/297.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/297","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":297,"pages":{"first":2144,"last":2150},"keywords":["Machine Learning: Deep Learning","Machine Learning Applications: Networks","Machine Learning: Clustering","Machine Learning: Probabilistic Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Belkin and Niyogi, 2003] Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality re- duction and data representation. Neural Computation, 15(6):1373-1396, 2003.","title":"Laplacian eigenmaps for dimensionality reduction and data representation","authors":"Niyogi Belkin, Mikhail Belkin, Partha Niyogi","year":"2003","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bojchevski and Günnemann, 2018] Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In ICLR, 2018.","title":"Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking","authors":"Aleksandar Bojchevski, Stephan Günnemann","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bühler and Hein, 2009] Thomas Bühler and Matthias Hein. Spectral clustering based on the graph p-laplacian. In ICML, pages 81-88. ACM, 2009.","title":"Spectral clustering based on the graph p-laplacian","authors":"Hein Bühler, Thomas Bühler, Matthias Hein","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chakraborty et al., 2014] Tanmoy Chakraborty, Sriram Srinivasan, Niloy Ganguly, Animesh Mukherjee, and Sanjukta Bhowmick. On the permanence of vertices in network communities. In KDD, pages 1396-1405. ACM, 2014.","title":"On the permanence of vertices in network communities","authors":"Tanmoy Chakraborty, Sriram Srinivasan, Niloy Ganguly, Animesh Mukherjee, Sanjukta Bhowmick","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Craddock et al., 2012] R. Cameron Craddock, G.Andrew James, Paul E. Holtzheimer III, Xiaoping P. Hu, and He- len S. Mayberg. A whole brain fmri atlas generated via spatially constrained spectral clustering. Human Brain Mapping, 33(8):1914-1928, 2012.","title":"A whole brain fmri atlas generated via spatially constrained spectral clustering","authors":"R Cameron Craddock, G Andrew James, Paul E Holtzheimer, Xiaoping P Hu, Helen S Mayberg","year":"2012","venue":"Human Brain Mapping","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dilokthanakul et al., 2016] Nat Dilokthanakul, Pedro A. M. Mediano, Marta Garnelo, Matthew C. H. Lee, Hugh Sal- imbeni, Kai Arulkumaran, and Murray Shanahan. Deep unsupervised clustering with gaussian mixture variational autoencoders. CoRR, abs/1611.02648, 2016.","title":"Deep unsupervised clustering with gaussian mixture variational autoencoders","authors":"Dilokthanakul","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fortunato and Hric, 2016] Santo Fortunato and Darko Hric. Community detection in networks: A user guide. Physics Reports, 659:1 -44, 2016.","title":"Santo Fortunato and Darko Hric. Community detection in networks: A user guide","authors":"Hric Fortunato","year":"2016","venue":"Physics Reports","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fortunato, 2010] Santo Fortunato. Community detection in graphs. Physics Reports, 486(3):75 -174, 2010.","title":"Santo Fortunato. Community detection in graphs","authors":"Fortunato","year":"2010","venue":"Physics Reports","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In KDD, pages 855-864, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hsu et al., 2019] Wei-Ning Hsu, Yu Zhang, Ron Weiss, Heiga Zen, Yonghui Wu, Yuan Cao, and Yuxuan Wang. Hierarchical generative modeling for controllable speech synthesis. In ICLR, 2019.","title":"Hierarchical generative modeling for controllable speech synthesis","authors":"Wei-Ning Hsu, Yu Zhang, Ron Weiss, Heiga Zen, Yonghui Wu, Yuan Cao, Yuxuan Wang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Javed et al., 2018] Muhammad Aqib Javed, Muham- mad Shahzad Younis, Siddique Latif, Junaid Qadir, and Adeel Baig. Community detection in networks: A mul- tidisciplinary review. Journal of Network and Computer Applications, 108:87 -111, 2018.","title":"Community detection in networks: A multidisciplinary review","authors":"Javed","year":"2018","venue":"Journal of Network and Computer Applications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang et al., 2017] Zhuxi Jiang, Yin Zheng, Huachun Tan, Bangsheng Tang, and Hanning Zhou. Variational deep embedding: An unsupervised and generative approach to clustering. In IJCAI, pages 1965-1972. AAAI Press, 2017.","title":"Variational deep embedding: An unsupervised and generative approach to clustering","authors":"Jiang","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/273"},{"position":13,"raw":"[Kingma and Salimans, 2016] Diederik P Kingma and Tim Salimans. Improving variational autoencoders with in- verse autoregressive flow. In NeurIPS, 2016.","title":"Improving variational autoencoders with inverse autoregressive flow","authors":"P Diederik, Tim Kingma, Salimans","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Welling, 2013] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Kuang et al., ] Da Kuang, Chris Ding, and Haesun Park. Symmetric Nonnegative Matrix Factorization for Graph Clustering, pages 106-117.","title":"Symmetric Nonnegative Matrix Factorization for Graph Clustering","authors":"Kuang","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Leskovec et al., 2010] Jure Leskovec, Kevin J. Lang, and Michael Mahoney. Empirical comparison of algorithms for network community detection. In WWW, pages 631- 640. ACM, 2010.","title":"Empirical comparison of algorithms for network community detection","authors":"Jure Leskovec, Kevin J Lang, Michael Mahoney","year":"2010","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2019] Rui Li, Fanghua Ye, Shaoan Xie, Chuan Chen, and Zibin Zheng. Digging into it: Community de- tection via hidden attributes analysis. Neurocomputing, 331:97 -107, 2019.","title":"Digging into it: Community detection via hidden attributes analysis","authors":"Rui Li, Fanghua Ye, Shaoan Xie, Chuan Chen, Zibin Zheng","year":"2019","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Newman, 2006] M. E. J. Newman. Modularity and com- munity structure in networks. Proceedings of the National Academy of Sciences, 103(23):8577-8582, 2006.","title":"Modularity and community structure in networks","authors":"M E J Newman","year":"2006","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: online learning of social rep- resentations. In KDD, pages 701-710, 2014.","title":"Deepwalk: online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi and Malik, 2000] Jianbo Shi and Jitendra Malik. Nor- malized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888- 905, 2000.","title":"Normalized cuts and image segmentation","authors":"Jianbo Shi, Jitendra Malik","year":"2000","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2017] Bing-Jie Sun, Huawei Shen, Jinhua Gao, Wentao Ouyang, and Xueqi Cheng. A non-negative sym- metric encoder-decoder approach for community detec- tion. In CIKM, pages 597-606. ACM, 2017.","title":"A non-negative symmetric encoder-decoder approach for community detection","authors":"Sun","year":"2017","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tian et al., 2014] Fei Tian, Bin Gao, Qing Cui, Enhong Chen, and Tie-Yan Liu. Learning deep representations for graph clustering. In AAAI, pages 1293-1299, 2014.","title":"Learning deep representations for graph clustering","authors":"Bin Fei Tian, Qing Gao, Enhong Cui, Tie-Yan Chen, Liu","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[von Luxburg, 2007] Ulrike von Luxburg. A tutorial on spec- tral clustering. Statistics and Computing, 17(4):395-416, 2007.","title":"A tutorial on spectral clustering","authors":"Ulrike Von, Luxburg","year":"2007","venue":"Statistics and Computing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2016] Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. In KDD, pages 1225-1234, 2016.","title":"Structural deep network embedding","authors":"Wang","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2016] Liang Yang, Xiaochun Cao, Dongxiao He, Chuan Wang, Xiao Wang, and Weixiong Zhang. Mod- ularity based community detection with deep learning. In IJCAI, pages 2252-2258. AAAI Press, 2016.","title":"Modularity based community detection with deep learning","authors":"Liang Yang, Xiaochun Cao, Dongxiao He, Chuan Wang, Xiao Wang, Weixiong Zhang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ye et al., 2018] Fanghua Ye, Chuan Chen, and Zibin Zheng. Deep autoencoder-like nonnegative matrix factorization for community detection. In CIKM, CIKM '18, pages 1393-1402, New York, NY, USA, 2018. ACM.","title":"Deep autoencoder-like nonnegative matrix factorization for community detection","authors":"Ye","year":"2018","venue":"CIKM, CIKM '18","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu et al., 2018] Dingyuan Zhu, Peng Cui, Daixin Wang, and Wenwu Zhu. Deep variational network embedding in wasserstein space. In KDD, pages 2827-2836. ACM, 2018.","title":"Deep variational network embedding in wasserstein space","authors":"Dingyuan Zhu, Peng Cui, Daixin Wang, Wenwu Zhu","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}