{"type":"ScholarlyArticle","title":"Unsupervised Deep Graph Structure and Embedding Learning","authors":[{"name":"Xiaobo Shen","slug":"xiaobo-shen","given":"Xiaobo","family":"Shen","affiliation":"Nanjing University of Science and Technology"},{"name":"Lei Shi","slug":"lei-shi","given":"Lei","family":"Shi","affiliation":"Nanjing University of Science and Technology"},{"name":"Xiuwen Gong","slug":"xiuwen-gong","given":"Xiuwen","family":"Gong","affiliation":"University of Technology Sydney"},{"name":"Shirui Pan","slug":"shirui-pan","given":"Shirui","family":"Pan","affiliation":"Griffith University"}],"abstract":"Graph Neural Network (GNN) is powerful in graph embedding learning, but its performance has been shown to be heavily degraded under adversarial attacks. Deep graph structure learning (GSL) is proposed to defend attack by jointly learning graph structure and graph embedding, typically in node classification task. Label supervision is expensive in real-world applications, and thus unsupervised GSL is more challenging and still remains less studied. To fulfill this gap, this paper proposes a new unsupervised GSL method, i.e., unsupervised property GNN (UPGNN). UPGNN first refines graph structure by exploring properties of low rank, sparsity, feature smoothness. UPGNN employs graph mutual information loss to learn graph embedding by maximizing its correlation with refined graph. The proposed UPGNN learns graph structure and embedding without label supervision, and thus can be applied various downstream tasks. We further propose Accelerated UPGNN (AUPGNN) to reduce computational complexity, providing a efficient alternative to UPGNN. Our extensive experiments on node classification and clustering demonstrate the effectiveness of the proposed method over the state-of-the-arts especially under heavy perturbation.","doi":"10.24963/ijcai.2024/259","url":"https://ijcai.org/proceedings/2024/259","pdf":"https://ijcai.org/proceedings/2024/0259.pdf","markdown":"https://ijcai.org/proceedings/2024/259.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/259","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":259,"pages":{"first":2342,"last":2350},"keywords":["Machine Learning: ML: Sequence and graph learning","Machine Learning: ML: Unsupervised learning","Data Mining: DM: Mining graphs"],"videos":[],"references":[{"position":1,"raw":"[Belghazi et al., 2018] Mohamed I. Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Hjelm R D. Mine: mutual in- formation neural estimation. In ICML, pages 531-540, 2018.","title":"Mine: mutual information neural estimation","authors":"Mohamed I Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, Hjelm R D","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2016] Shaosheng Cao, Wei Lu, and Qiongkai Xu. Deep neural networks for learning graph represen- tations. In AAAI, pages 1145-1152, 2016.","title":"Deep neural networks for learning graph representations","authors":"Cao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dai et al., 2018] Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. Adversarial attack on graph structured data. In ICML, pages 1115-1124, 2018.","title":"Adversarial attack on graph structured data","authors":"Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, Le Song","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Elinas et al., 2020] Pantelis Elinas, Edwin V Bonilla, and Louis Tiao. Variational inference for graph convolutional networks in the absence of graph data and adversarial set- tings. NIPS, 33:18648-18660, 2020.","title":"Variational inference for graph convolutional networks in the absence of graph data and adversarial settings","authors":"Pantelis Elinas, Edwin V Bonilla, Louis Tiao","year":"2020","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Entezari et al., 2020] Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E Papalexakis. All you need is low (rank) defending against adversarial at- tacks on graphs. In WSDM, pages 169-177, 2020.","title":"All you need is low (rank) defending against adversarial attacks on graphs","authors":"Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, Evangelos E Papalexakis","year":"2020","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fatemi et al., 2021] Bahare Fatemi, Layla El Asri, and Seyed Mehran Kazemi. Slaps: Self-supervision im- proves structure learning for graph neural networks. NIPS, 34:22667-22681, 2021.","title":"Slaps: Self-supervision improves structure learning for graph neural networks","authors":"Fatemi","year":"2021","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Feng and Wang, 2022] Long Feng and Junhui Wang. Pro- jected robust PCA with application to smooth image re- covery. JMLR, 23:249:1-249:41, 2022.","title":"Projected robust PCA with application to smooth image recovery","authors":"Wang ; Feng, Long Feng, Junhui Wang","year":"2022","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2020] Xiang Gao, Wei Hu, and Zongming Guo. Exploring structure-adaptive graph learning for robust semi-supervised classification. In ICME, pages 1-6. IEEE, 2020.","title":"Exploring structure-adaptive graph learning for robust semi-supervised classification","authors":"Xiang Gao, Wei Hu, Zongming Guo","year":"2020","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gasteiger et al., 2019] Johannes Gasteiger, Stefan Weißen- berger, and Stephan Günnemann. Diffusion improves graph learning. NIPS, 32, 2019.","title":"Diffusion improves graph learning","authors":"Gasteiger","year":"2019","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In SIGKDD, pages 855-864, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hjelm et al., 2018] R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representa- tions by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670, 2018.","title":"Learning deep representations by mutual information estimation and maximization","authors":"Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jin et al., 2020] Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. In SIGKDD, pages 66-74, 2020.","title":"Graph structure learning for robust graph neural networks","authors":"Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, Jiliang Tang","year":"2020","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jin et al., 2021] Wei Jin, Tyler Derr, Yiqi Wang, Yao Ma, Zitao Liu, and Jiliang Tang. Node similarity preserving graph convolutional networks. In WSDM, pages 148-156, 2021.","title":"Node similarity preserving graph convolutional networks","authors":"Wei Jin, Tyler Derr, Yiqi Wang, Yao Ma, Zitao Liu, Jiliang Tang","year":"2021","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Variational graph auto-encoders. NeurIPS, 2016.","title":"Variational graph auto-encoders","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2018] Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. Adaptive graph convolutional neural net- works. In AAAI, volume 32, 2018.","title":"Adaptive graph convolutional neural networks","authors":"Ruoyu Li, Sheng Wang, Feiyun Zhu, Junzhou Huang","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nowozin et al., 2016] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neu- ral samplers using variational divergence minimization. NIPS, 29, 2016.","title":"f-gan: Training generative neural samplers using variational divergence minimization","authors":"Sebastian Nowozin, Botond Cseke, Ryota Tomioka","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Peng et al., 2020] Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Jun- zhou Huang. Graph representation learning via graphical mutual information maximization. In Proceedings of The Web Conference 2020, pages 259-270, 2020.","title":"Graph representation learning via graphical mutual information maximization","authors":"Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, Junzhou Huang","year":"2020","venue":"Proceedings of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Peng et al., 2022] Zhen Peng, Minnan Luo, Wenbing Huang, Jundong Li, Qinghua Zheng, Fuchun Sun, and Junzhou Huang. Learning representations by graphical mutual information estimation and maximization. TPAMI, 2022.","title":"Learning representations by graphical mutual information estimation and maximization","authors":"Zhen Peng, Minnan Luo, Wenbing Huang, Jundong Li, Qinghua Zheng, Fuchun Sun, Junzhou Huang","year":"2022","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In SIGKDD, pages 701-710, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Raguet et al., 2013] Hugo Raguet, Jalal Fadili, and Gabriel Peyré. A generalized forward-backward splitting. SIAM Journal on Imaging Sciences, 6(3):1199-1226, 2013.","title":"A generalized forward-backward splitting","authors":"Raguet","year":"2013","venue":"SIAM Journal on Imaging Sciences","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Richard et al., 2012] Emile Richard, Pierre-André Savalle, and Nicolas Vayatis. Estimation of simultaneously sparse and low rank matrices. ICML, 2012.","title":"Estimation of simultaneously sparse and low rank matrices","authors":"Emile Richard, Pierre-André Savalle, Nicolas Vayatis","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Savalle et al., 2012] Pierre-André Savalle, Emile Richard, and Nicolas Vayatis. Estimation of simultaneously sparse and low rank matrices. In ICML, 2012.","title":"Estimation of simultaneously sparse and low rank matrices","authors":"Savalle","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. ICLR, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Veličković et al., 2019] Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In ICLR, 2019.","title":"Deep graph infomax","authors":"Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, Devon Hjelm","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wan and Kokel, 2021] Guihong Wan and Harsha Kokel. Graph sparsification via meta-learning. AAAI, 2021.","title":"Graph sparsification via meta-learning","authors":"Wan, Kokel ;, Guihong Wan, Harsha Kokel","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2017a] Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. MGAE: marginal- ized graph autoencoder for graph clustering. In CIKM, pages 889-898, 2017.","title":"MGAE: marginalized graph autoencoder for graph clustering","authors":"Wang","year":"2017","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2017b] Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. Community preserv- ing network embedding. In AAAI, pages 203-209, 2017.","title":"Community preserving network embedding","authors":"Wang","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wu et al., 2019] Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. Adversarial examples on graph data: Deep insights into attack and de- fense. IJCAI, 2019.","title":"Adversarial examples on graph data: Deep insights into attack and defense","authors":"Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, Liming Zhu","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2020] Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and Anil K Jain. Adversarial attacks and defenses in images, graphs and text: A review. IJAC, 17:151-178, 2020.","title":"Adversarial attacks and defenses in images, graphs and text: A review","authors":"Han Xu, Yao Ma, Hao-Chen, Debayan Liu, Hui Deb, Ji-Liang Liu, Anil K Tang, Jain","year":"2020","venue":"IJAC","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2015] Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. Network represen- tation learning with rich text information. In IJCAI, pages 2111-2117, 2015.","title":"Network representation learning with rich text information","authors":"Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, Edward Y Chang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/299"},{"position":32,"raw":"[Yang et al., 2019] Liang Yang, Zesheng Kang, Xiaochun Cao, Di Jin, Bo Yang, and Yuanfang Guo. Topology op- timization based graph convolutional network. In IJCAI, pages 4054-4061, 2019.","title":"Topology optimization based graph convolutional network","authors":"Liang Yang, Zesheng Kang, Xiaochun Cao, Di Jin, Bo Yang, Yuanfang Guo","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/563"},{"position":33,"raw":"[Zhang and Zitnik, 2020] Xiang Zhang and Marinka Zitnik. Gnnguard: Defending graph neural networks against ad- versarial attacks. NeurIPS, 33:9263-9275, 2020.","title":"Gnnguard: Defending graph neural networks against adversarial attacks","authors":"Xiang Zhang, Marinka Zitnik","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2019a] Xiaotong Zhang, Han Liu, Qimai Li, and Xiao-Ming Wu. Attributed graph clustering via adap- tive graph convolution. IJCAI, 2019.","title":"Attributed graph clustering via adaptive graph convolution","authors":"Zhang","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/601"},{"position":35,"raw":"[Zhang et al., 2019b] Yingxue Zhang, Soumyasundar Pal, Mark Coates, and Deniz Ustebay. Bayesian graph con- volutional neural networks for semi-supervised classifica- tion. In AAAI, volume 33, pages 5829-5836, 2019.","title":"Bayesian graph convolutional neural networks for semi-supervised classification","authors":"Zhang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhu et al., 2019] Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. Robust graph convolutional networks against adversarial attacks. In SIGKDD, pages 1399-1407, 2019.","title":"Robust graph convolutional networks against adversarial attacks","authors":"Dingyuan Zhu, Ziwei Zhang, Peng Cui, Wenwu Zhu","year":"2019","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhu et al., 2021] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In WWW, pages 2069-2080, 2021.","title":"Graph contrastive learning with adaptive augmentation","authors":"Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zügner and Günnemann, 2019] Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In ICLR, 2019.","title":"Adversarial attacks on graph neural networks via meta learning","authors":"Daniel Zügner, Stephan Günnemann","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zügner et al., 2018] Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural net- works for graph data. In SIGKDD, pages 2847-2856, 2018.","title":"Adversarial attacks on neural networks for graph data","authors":"Daniel Zügner, Amir Akbarnejad, Stephan Günnemann","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":"/proceedings/2019/872"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}