{"type":"ScholarlyArticle","title":"Multi-level Graph Contrastive Prototypical Clustering","authors":[{"name":"Yuchao Zhang","slug":"yuchao-zhang","given":"Yuchao","family":"Zhang","affiliation":"School of Computer Science , Northwestern Polytechnical University , Xi'an 710072 , Shaanxi , P. R. China; School of Artificial Intelligence, Optics and Electronics (iOPEN) , Northwestern Polytechnical University , Xi'an 710072 , Shaanxi , P. R. China"},{"name":"Yuan Yuan","slug":"yuan-yuan","given":"Yuan","family":"Yuan","affiliation":"School of Artificial Intelligence, Optics and Electronics (iOPEN) , Northwestern Polytechnical University , Xi'an 710072 , Shaanxi , P. R. China"},{"name":"Qi Wang","slug":"qi-wang","given":"Qi","family":"Wang","affiliation":"School of Artificial Intelligence, Optics and Electronics (iOPEN) , Northwestern Polytechnical University , Xi'an 710072 , Shaanxi , P. R. China"}],"abstract":"Recently, graph neural networks (GNNs) have drawn a surge of investigations in deep graph clustering. Nevertheless, existing approaches predominantly are inclined to semantic-agnostic since GNNs exhibit inherent limitations in capturing global underlying semantic structures. Meanwhile, multiple objectives are imposed within one latent space, whereas representations from different granularities may presumably conflict with each other, yielding severe performance degradation for clustering. To this end, we propose a novel Multi-Level Graph Contrastive Prototypical Clustering (MLG-CPC) framework for end-to-end clustering. Specifically, a Prototype Discrimination (ProDisc) objective function is proposed to explicitly capture semantic information via cluster assignments. Moreover, to alleviate the issue of objectives conflict, we introduce to perceive representations of different granularities within individual feature-, prototypical-, and cluster-level spaces by the feature decorrelation, prototype contrast, and cluster space consistency respectively. Extensive experiments on four benchmarks demonstrate the superiority of the proposed MLG-CPC against the state-of-the-art graph clustering approaches.","doi":"10.24963/ijcai.2023/513","url":"https://ijcai.org/proceedings/2023/513","pdf":"https://ijcai.org/proceedings/2023/0513.pdf","markdown":"https://ijcai.org/proceedings/2023/513.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/513","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":513,"pages":{"first":4611,"last":4619},"keywords":["Machine Learning: ML: Unsupervised learning","Machine Learning: ML: Multi-view learning","Machine Learning: ML: Clustering"],"videos":[],"references":[{"position":1,"raw":"[Bo et al., 2020] Deyu Bo, Xiao Wang, Chuan Shi, Meiqi Zhu, Emiao Lu, and Peng Cui. Structural deep clustering network. In Proceedings of The Web Conference, pages 1400-1410, 2020.","title":"Structural deep clustering network","authors":"Deyu Bo, Xiao Wang, Chuan Shi, Meiqi Zhu, Emiao Lu, Peng Cui","year":"2020","venue":"Proceedings of The Web Conference","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caron et al., 2018] Mathilde Caron, Piotr Bojanowski, Ar- mand Joulin, and Matthijs Douze. Deep clustering for un- supervised learning of visual features. In Proceedings of the European Conference on Computer Vision, pages 132- 149, 2018.","title":"Deep clustering for unsupervised learning of visual features","authors":"Mathilde Caron, Piotr Bojanowski, Armand Joulin, Matthijs Douze","year":"2018","venue":"Proceedings of the European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Caron et al., 2020] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by con- trasting cluster assignments. Advances in Neural Informa- tion Processing Systems, 33:9912-9924, 2020.","title":"Unsupervised learning of visual features by contrasting cluster assignments","authors":"Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, Armand Joulin","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Inter- national Conference on Machine Learning, pages 1597- 1607, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ermolov et al., 2021] Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening for self-supervised representation learning. In International Conference on Machine Learning, pages 3015-3024, 2021.","title":"Whitening for self-supervised representation learning","authors":"Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gasteiger et al., 2018] Johannes Gasteiger, Aleksandar Bo- jchevski, and Stephan Günnemann. Predict then propa- gate: Graph neural networks meet personalized pagerank. In International Conference on Learning Representations, 2018.","title":"Predict then propagate: Graph neural networks meet personalized pagerank","authors":"Gasteiger","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neu- ral message passing for quantum chemistry. In Inter- national Conference on Machine Learning, pages 1263- 1272, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, S Samuel, Patrick F Schoenholz, Oriol Riley, George E Vinyals, Dahl","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gong et al., 2022] Lei Gong, Sihang Zhou, Wenxuan Tu, and Xinwang Liu. Attributed graph clustering with dual redundancy reduction. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 3015- 3021, 2022.","title":"Attributed graph clustering with dual redundancy reduction","authors":"Lei Gong, Sihang Zhou, Wenxuan Tu, Xinwang Liu","year":"2022","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2022/418"},{"position":9,"raw":"[Guo et al., 2017] Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. Improved deep embedded clustering with local structure preservation. In Proceedings of the Inter- national Joint Conference on Artificial Intelligence, pages 1753-1759, 2017.","title":"Improved deep embedded clustering with local structure preservation","authors":"Xifeng Guo, Long Gao, Xinwang Liu, Jianping Yin","year":"2017","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/243"},{"position":10,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in Neural Information Processing Sys- tems, 30:1025-1035, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hartigan and Wong, 1979] John A Hartigan and Manchek A Wong. Algorithm as 136: A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics), 28(1):100-108, 1979.","title":"Algorithm as 136: A k-means clustering algorithm","authors":"Wong ; ] Hartigan, A John, Manchek A Hartigan, Wong","year":"1979","venue":"Journal of the royal statistical society. series c (applied statistics)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hassani and Khasahmadi, 2020] Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view repre- sentation learning on graphs. In International Conference on Machine Learning, pages 4116-4126, 2020.","title":"Contrastive multi-view representation learning on graphs","authors":"Kaveh Hassani, Amir Hosein, Khasahmadi","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Keriven, 2022] Nicolas Keriven. Not too little, not too much: a theoretical analysis of graph (over) smoothing. arXiv preprint arXiv:2205.12156, 2022.","title":"Not too little, not too much: a theoretical analysis of graph (over) smoothing","authors":"Nicolas Keriven","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Variational graph auto-encoders. Advances in Neural In- formation Processing Systems, 2016.","title":"Variational graph auto-encoders","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Rep- resentations, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2020] Xuelong Li, Han Zhang, Rong Wang, and Feiping Nie. Multiview clustering: A scalable and parameter-free bipartite graph fusion method. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 44(1):330-344, 2020.","title":"Multiview clustering: A scalable and parameter-free bipartite graph fusion method","authors":"Xuelong Li, Han Zhang, Rong Wang, Feiping Nie","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2021] Junnan Li, Pan Zhou, Caiming Xiong, and Steven CH Hoi. Prototypical contrastive learning of unsu- pervised representations. In International Conference on Learning Representations, 2021.","title":"Prototypical contrastive learning of unsupervised representations","authors":"Li","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lin et al., 2022] Shuai Lin, Chen Liu, Pan Zhou, Zi-Yuan Hu, Shuojia Wang, Ruihui Zhao, Yefeng Zheng, Liang Lin, Eric Xing, and Xiaodan Liang. Prototypical graph contrastive learning. IEEE Transactions on Neural Net- works and Learning Systems, 2022.","title":"Prototypical graph contrastive learning","authors":"Shuai Lin, Chen Liu, Pan Zhou, Zi-Yuan Hu, Shuojia Wang, Ruihui Zhao, Yefeng Zheng, Liang Lin, Eric Xing, Xiaodan Liang","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2022] Yanfeng Liu, Qiang Li, Yuan Yuan, Qian Du, and Qi Wang. Abnet: Adaptive balanced network for multiscale object detection in remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 60:1-14, 2022.","title":"Abnet: Adaptive balanced network for multiscale object detection in remote sensing imagery","authors":"Yanfeng Liu, Qiang Li, Yuan Yuan, Qian Du, Qi Wang","year":"2022","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2023] Yanfeng Liu, Zhitong Xiong, Yuan Yuan, and Qi Wang. Distilling knowledge from super resolution for efficient remote sensing salient object detection. IEEE Transactions on Geoscience and Remote Sensing, 61:1- 16, 2023.","title":"Distilling knowledge from super resolution for efficient remote sensing salient object detection","authors":"Yanfeng Liu, Zhitong Xiong, Yuan Yuan, Qi Wang","year":"2023","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pan et al., 2018] Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. Adversarially regu- larized graph autoencoder for graph embedding. In Inter- national Joint Conference on Artificial Intelligence, pages 2609-2615, 2018.","title":"Adversarially regularized graph autoencoder for graph embedding","authors":"Pan","year":"2018","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/362"},{"position":22,"raw":"[Park et al., 2019] Jiwoong Park, Minsik Lee, Hyung Jin Chang, Kyuewang Lee, and Jin Young Choi. Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In Proceedings of the Interna- tional Conference on Computer Vision, pages 6519-6528, 2019.","title":"Symmetric graph convolutional autoencoder for unsupervised graph representation learning","authors":"Park","year":"2019","venue":"Proceedings of the International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shchur et al., 2018] Oleksandr Shchur, Maximil- ian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868, 2018.","title":"Pitfalls of graph neural network evaluation","authors":"Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, Stephan Günnemann","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Ad- vances in Neural Information Processing Systems, pages 4080-4090, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Topping et al., 2022] Jake Topping, Francesco Di Gio- vanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M Bronstein. Understanding over-squashing and bottlenecks on graphs via curvature. In International Con- ference on Learning Representations, 2022.","title":"Understanding over-squashing and bottlenecks on graphs via curvature","authors":"Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, Michael M Bronstein","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tu et al., 2021] Wenxuan Tu, Sihang Zhou, Xinwang Liu, Xifeng Guo, Zhiping Cai, En Zhu, and Jieren Cheng. Deep fusion clustering network. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9978-9987, 2021.","title":"Deep fusion clustering network","authors":"Wenxuan Tu, Sihang Zhou, Xinwang Liu, Xifeng Guo, Zhiping Cai, En Zhu, Jieren Cheng","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In Interna- tional Conference on Learning Representations, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Velickovic et al., 2019] Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2019.","title":"Deep graph infomax","authors":"Petar Velickovic, William Fedus, Pietro William L Hamilton, Yoshua Liò, Devon Bengio, Hjelm","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang and Isola, 2020] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Inter- national Conference on Machine Learning, pages 9929- 9939, 2020.","title":"Understanding contrastive representation learning through alignment and uniformity on the hypersphere","authors":"Wang, Isola ;, Tongzhou Wang, Phillip Isola","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2019] Chun Wang, Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, and Chengqi Zhang. At- tributed graph clustering: a deep attentional embedding approach. In Proceedings of the International Joint Con- ference on Artificial Intelligence, pages 3670-3676, 2019.","title":"Attributed graph clustering: a deep attentional embedding approach","authors":"Chun Wang, Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Chengqi Zhang","year":"2019","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2019/509"},{"position":32,"raw":"[Wang et al., 2022a] Qi Wang, Yanfeng Liu, Zhitong Xiong, and Yuan Yuan. Hybrid feature aligned network for salient object detection in optical remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 60:1- 15, 2022.","title":"Hybrid feature aligned network for salient object detection in optical remote sensing imagery","authors":"Wang","year":"2022","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2022b] Qi Wang, Yanling Miao, Mulin Chen, and Yuan Yuan. Spatial-spectral clustering with anchor graph for hyperspectral image. IEEE Transactions on Geo- science and Remote Sensing, 60:1-13, 2022.","title":"Spatial-spectral clustering with anchor graph for hyperspectral image","authors":"Wang","year":"2022","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xia et al., 2022] Jun Xia, Lirong Wu, Ge Wang, Jintao Chen, and Stan Z Li. Progcl: Rethinking hard negative mining in graph contrastive learning. In International Conference on Machine Learning, pages 24332-24346, 2022.","title":"Progcl: Rethinking hard negative mining in graph contrastive learning","authors":"Jun Xia, Lirong Wu, Ge Wang, Jintao Chen, Stan Z Li","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xie et al., 2016] Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learn- ing, pages 478-487, 2016.","title":"Unsupervised deep embedding for clustering analysis","authors":"Junyuan Xie, Ross Girshick, Ali Farhadi","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xu et al., 2021] Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. Self-supervised graph-level representation learning with local and global structure. In International Conference on Machine Learning, pages 11548-11558, 2021.","title":"Self-supervised graph-level representation learning with local and global structure","authors":"Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, Jian Tang","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2017] Bo Yang, Xiao Fu, Nicholas D Sidiropoulos, and Mingyi Hong. Towards k-means- friendly spaces: Simultaneous deep learning and clustering. In International Conference on Machine Learning, pages 3861-3870, 2017.","title":"Towards k-meansfriendly spaces: Simultaneous deep learning and clustering","authors":"Bo Yang, Xiao Fu, Nicholas D Sidiropoulos, Mingyi Hong","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yuan et al., 2018] Yuan Yuan, Jie Fang, Xiaoqiang Lu, and Yachuang Feng. Remote sensing image scene classifica- tion using rearranged local features. IEEE Transactions on Geoscience and Remote Sensing, 57(3):1779-1792, 2018.","title":"Remote sensing image scene classification using rearranged local features","authors":"Yuan Yuan, Jie Fang, Xiaoqiang Lu, Yachuang Feng","year":"2018","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zbontar et al., 2021] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In Inter- national Conference on Machine Learning, pages 12310- 12320, 2021.","title":"Barlow twins: Selfsupervised learning via redundancy reduction","authors":"Jure Zbontar, Li Jing, Ishan Misra, Yann Lecun, Stéphane Deny","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhu et al., 2021] Hao Zhu, Ke Sun, and Peter Koniusz. Contrastive laplacian eigenmaps. Advances in Neural In- formation Processing Systems, 34:5682-5695, 2021.","title":"Contrastive laplacian eigenmaps","authors":"Ke Hao Zhu, Peter Sun, Koniusz","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}