Dual Contrastive Graph-Level Clustering with Multiple Cluster Perspectives Alignment

1 Introduction

Graph-structured data is one of the most common data structures in the real world, and some examples include molecular data [Rong et al., 2020], biological data [Agarwal, 2006], and social networks [Aggarwal, 2011], etc. Clustering on graph data [Schaeffer, 2007; Li et al., 2023] is a fundamental research topic in machine learning, which aims to partition the nodes of a single graph (node-level) or a set of graphs (graph-level) into different groups without supervision information. Early works on graph data clustering were mainly based on graph cut [Schaeffer, 2007] and spectral clustering (SC) [Ng et al., 2001; Fan et al., 2022; Chen et al., 2022;

∗Corresponding author.

Chen et al., 2023a], but such approaches are difficult and time-consuming to handle complex graph structures or very large graphs. Therefore, learning representative features for graph data is an intuitive way to improve the efficiency and performance of graph clustering.

The upsurge of deep learning and graph neural networks (GNNs) [Wu et al., 2020] facilitate the progress of graph clustering. Many GNN-based deep clustering approaches [Wang et al., 2017; Bo et al., 2020; Tu et al., 2021; Liu et al., 2022] have exhibited remarkable success in node clustering. Recently, graph contrastive learning (GCL) [Zhu et al., 2021a] has attracted significant interest from researchers due to its powerful feature learning capability. GCL learns the representation of graph data by imposing various forms of augmentation to the graphs and then contrasting between positive and negative pairs. By introducing the concept of GCL into learning node features, several GCL-based graph clustering methods [Zhao et al., 2021; Xia et al., 2022; Liu et al., 2023b; Liu et al., 2023a] have further raised the performance bar of node clustering.

However, there are still very limited efforts delving into graph-level clustering. This disparity stems from the fact that node-level clustering operates within a single graph, where node representations can be learned more easily through information aggregation between nodes and their neighbors [Wu et al., 2020]. Conversely, graph-level tasks are specific to a series of graphs, which presents a more complex and challenging task of measuring the relationships between them. Recent advances in graph-level tasks have focused on graph-level classification, mostly based on graph kernels and graph-level representation learning methods [Yang et al., 2023]. Graph kernels [Vishwanathan et al., 2010; Sun and Fan, 2024], such as the Weisfeiler-Lehman kernel (WL) [Shervashidze et al., 2011] and Short-Path kernel (SP) [Borgwardt and Kriegel, 2005], measure similarity between graphs using well-defined kernel functions. While graph representation learning [Xu et al., 2019; Sun et al., 2020; You et al., 2020; Cai et al., 2024a; Wu et al., 2024b] generally leverages GNNs [Kipf and Welling, 2017; Xu et al., 2019; Wu et al., 2024a] as the backbone, together with readout functions, to aggregate node features for learning graphlevel features. Graph-level classification is then achieved via a two-stage strategy that combines a classifier with either graph kernels or graph representation learning methods.

Nevertheless, these approaches tend to be a trivial solution when implemented in graph-level clustering, primarily due to the following challenges.

• supervision information is used to train the classifier, whereas clustering is not. Consequently, graph-level clustering demands a much higher requirement for the discriminability of learned features than graph-level classification.
• The aforementioned approaches typically construct as a two-stage model, where the learned similarity matrix or graph-level representations cannot be optimized for specific clustering tasks. Previous works on deep clustering [Yang et al., 2016; Xie et al., 2016; Cai et al., 2022a] have demonstrated the benefits of incorporating an explicit clustering objective into feature learning.
• Relying on a single cluster perspective hardly exploits comprehensive cluster information. Different cluster perspectives may reveal different cluster information. For example, k-means [Hartigan and Wong, 1979] focuses more on the Euclidean distance relationship between data, while SC [Ng et al., 2001] highlights connectivity and reveals clusters under subspace structures.

These challenges naturally pose an intriguing research question: Can we advance graph-level clustering by designing a joint optimization framework and leveraging information from different cluster perspectives?

To answer this question, we propose a dual contrastive graph-level clustering (DCGLC) 1 in this paper. Figure 1 shows the network structure of DCGLC. Specifically, we leverage graph contrastive learning to enhance the effectiveness of graph-level representations by maximizing the consistency between positive sample pairs and inconsistency between negative sample pairs. Additionally, we introduce an explicit clustering objective into graph contrastive learning, which enables our model to be optimized for clustering and learns discriminative graph-level representations. Furthermore, we propose a contrastive mechanism to align clustering information from both the Euclidean-based and subspacebased cluster perspectives, where the former focuses on the geometric relationships between graphs while the latter captures their structural relationships. The proposed DCGLC encourages the incorporation of clustering information from different cluster perspectives and ensures the consistency of clustering assignment across them, which leads to more reliable clustering results. Besides, the graph-level features contrast, and cluster-perspective contrast can be mutually improved, facilitating the learning of cluster-friendly representations. We summarize the main contributions of this work as follows:

1 Code is available at: https://github.com/wownice333/DCGLC

• We propose DCGLC, an end-to-end clustering method unifying the optimization of graph contrastive learning with information integration of different cluster perspectives.
• We propose a cluster-perspective contrast mechanism to align the cluster information, which results in more reliable clustering assignments by considering both geometric and structural relationships between graphs.
• We demonstrate the effectiveness of DCGLC through extensive experiments on various types of graph benchmarks. Experimental results show that DCGLC outperforms stateof-the-art baselines in graph-level clustering.

2 RelatedWork

Deep Graph Clustering. Clustering [Xie et al., 2016; Zhang et al., 2022; Cai et al., 2022b; Fu et al., 2022; Yao et al., 2023; Zhang et al., 2023; Cai et al., 2024b] is a fundamental task in machine learning, and is crucial for realworld data analysis. With the upsurge of GNNs, deep graph clustering, especially node clustering, has received increasing attention and advancement in recent years. For instance, Wang et al. [2017] proposed a marginalized graph autoencoder, which leverages both structural information and intrinsic properties of graph data to learn effective node features for clustering. Bo et al. [2020] combined graph convolutional network (GCN) [Kipf and Welling, 2017; Chen et al., 2023b; Chen et al., 2023c] and an integrated clustering module to learn effective node features for graph clustering. Tu et al. [2021] proposed a dynamic fusion mechanism to fuse the structural information and attributes and further designed a triplet self-supervised strategy to produce reliable cluster assignment. As an emerging technique, GCL-based methods [Zhao et al., 2021; Pan and Kang, 2021; Liu et al., 2022; Liu et al., 2023b] are also widely studied. Zhao et al. [2021] proposed a contrastive node clustering framework that encourages the similarity of positive pairs generated by attribute masks, and the dissimilarity of negative pairs constituted from out-of-cluster samples. Pan and Kang [2021] leveraged contrastive learning to learn consensus representation from multiple views for node clustering. Liu et al. [2023b] proposed a contrastive graph clustering method that effectively mines hard samples in clustering. Despite remarkable advancements in node clustering, a substantial research gap persists in graph-level clustering, because measuring the relationships between graphs is more complicated than nodes within a single graph.

Graph Representation Learning. Graph representation learning [Xu et al., 2019; Wu et al., 2020; Sun et al., 2024] aims to learn graph-level representation by aggregating node features learned with powerful GNNs, and has witnessed remarkable progress. For example, Sun et al. [2020] maximized the mutual information between the global and substructural information of data to learn comprehensive graphlevel representations. You et al. [2020] proposed GraphCL, a graph augmentation mechanism to learn effective graph-level representations based on GCL [Zhu et al., 2021a] for graph classification. We also follow the graph augmentation strategies proposed in [You et al., 2020] in this study, e.g., node dropping, edge perturbation, attribute masking, and subgraph, to augment graph data. You et al. [2021] proposed JOAO to investigate the automatic graph augmentation mechanisms in GCL. However, the majority of existing approaches focused on classification tasks and still very limited efforts on graphlevel clustering. Besides, these approaches usually construct two-stage models, which poses a greater challenge for graphlevel clustering as no supervision information can be utilized.

3 ProposedMethod

3.1 ProblemFormulation

Let G = {G1, . . . , GN} be a set of N graphs and each graph is formulated as Gi = {Vi,Ei, {x(i) v }v∈Vi }, where Vi is the node set, Ei is the edge set, and x(i) v is the feature vector of node v on graph i. We regard the problem that aims to partition the graph set into C different groups without any supervision information as graph-level clustering, i.e., to let G = G(1) ∪G(2) · · · ∪G(C) , s.t. G(i) ∩G(j) = ∅, ∀i ̸= j. Different from the graph-level classification and node-level clustering [Wu et al., 2020], graph-level clustering is still a challenging task in machine learning and has rarely been studied [Xu et al., 2022] because measuring the relationships between graphs in an unsupervised setting is much more complex than: 1) nodes within a single graph and 2) in a supervised setting. Therefore, the important issue in graph-level clustering is the exploration of learning graph-level features that contain clustering information and are sufficiently discriminative. To this end, we propose to achieve graph-level clustering by solving the following general problem:

where ℓr|ψ(·) denotes the representation learning objective with the aim of learning effective graph-level features, and ℓc|ϕ(·) is the clustering objective with the aim of capturing cluster information. A GNN [Kipf and Welling, 2017; Zhou et al., 2020] based model hθ(·), which has become a paradigm for many graph-based tasks [Xu et al., 2019; Park and Neville, 2019; Bo et al., 2020], is generally leveraged to facilitate the learning of graph representations from graph data. In the following section, we will show how to solve (1) and achieve graph-level clustering through the joint training of learning effective graph-level representation and leveraging comprehensive cluster information.

3.2 Graph-Level Representation Contrast

We first present the representation learning module, where we attempt to explore the capability of GCL [Zhu et al., 2021a] in learning effective representation for graph-level clustering. As an emerging technique in graph representation learning, GCL-based methods surpass state-of-the-art mutual information (MI) maximization-based methods [Veliˇckovi´c et al., 2019; Sun et al., 2020] in many tasks such as graph-level classification [You et al., 2020; Zhu et al., 2021b] and nodelevel clustering [Zhao et al., 2021; Xia et al., 2022]. Specifically, given a graph Gj, the learned features of node i in the k-th GNN layer can be defined as:

where a(k) i denotes the aggregated neighbour features of node i, i.e., a(k) i = AGGREGATE(k) (h(k-1) v : v ∈ N(i)), and N(i) is the neighbor set of node i. COMBINE(·) indicates the operation of integrating the information between node i and its neighbours. In particular, the initial feature h(0) i = xi. Subsequently, the graph-level representation of graph Gj , i.e., Hθ(Gj), can be formulated as follows:

where hiθ denotes the concatenated features of node i among all K layers, i.e., hiθ = CONCAT({h(k) i }Kk=1 ), and nGj is the node numbers of graph Gj . Note that the READOUT(·) function (e.g., Sum, Average, or Max/Min) aims to integrate the local features within a graph to obtain graph-level features [Kipf and Welling, 2017; Xu et al., 2019], and we utilize the sum readout function in our implementation.

Given a graph G, we follow the graph augmentation strategy A in [You et al., 2020] to produce an augmented graph

ˆGi ∼ Ai( ˆGi|G). Note that a minibatch with Nb graphs in training will generate Nb augmented graphs. Therefore, the original graph Gi and its corresponding augmented graph ˆGi can be regarded as a positive pair in contrastive learning, and the combination of Gi with other augmented graphs { ˆGj, i ̸= j} can be regarded as negative pairs then. For each graph pair such as Gi and ˆGi, their graph-level representations Hθ(Gi), Hθ( ˆGi) can be obtained through a GNN encoder following Eqs. (2) and (3). Besides, the contrastive loss is generally calculated in the low-dimensional latent space, therefore a projection head fψ(·) is applied to project the graph-level representation Hθ(Gi), Hθ( ˆGi) into ui|θ,ψ and ˆui|θ,ψ. We use NT-Xent loss [Sohn, 2016] as the contrastive loss in this paper, the NT-Xent loss is to maximize the consistency among positive pairs and the inconsistency among negative pairs. It is worth noting that we do not further generate other negative samples, instead, we regard the other Nb - 1 augmented samples in a minibatch as negative samples. For the Nb graphs in a minibatch, we aim to align the learned graph-level representations from the two sets of augmented graphs, which can be achieved by minimizing

where sim(ui|θ,ψ, ˆuj|θ,ψ) = u⊤i|θ,ψ ˆuj|θ,ψ ∥ui|θ,ψ∥∥ˆuj|θ,ψ∥ denotes the cosine similarity function, and τ indicates the temperature parameter. The loss function aims to minimize the distance between feature vectors of the same samples while maximizing the distance between feature vectors of different samples, resulting in learning more discriminative graph-level representations. Apparently, an intuitive way to achieve graphlevel clustering is to directly perform k-means [Hartigan and Wong, 1979] or SC [Ng et al., 2001] in the learned graphlevel representation. However, there are two important issues in this solution:

1. Graph-level representation learning and clustering are two separate processes without any explicitly defined clustering objective, which implies that they may fail to learn cluster-friendly graph-level representations. The empirical results in the experiment section also demonstrate that those two-stage clustering models, i.e., the simple combination of k-means/SC and graph kernels/graph representation learning, tend to be trivial solutions.
2. The Euclidean-based (e.g., k-means) and subspace-based clustering (e.g., SC) criteria offer cluster information from different perspectives. Euclidean-based methods prioritize distance information between data, while subspace-based methods emphasize the angle information. Combining these two perspectives can potentially provide richer information for graph-level clustering.

3.3 Multiple Cluster Perspectives Contrast

To address these issues, we propose a cluster-perspective contrast approach that aligns clustering information derived from multiple perspectives, which also introduces an explicit clustering objective to guide the optimization. Specifically, inspired by [Xie et al., 2016; Cai et al., 2022a], we cluster from two perspectives by utilizing two clustering heads, i.e., 1) Euclidian-based and 2) subspace-based, respectively. It should be noted that the clustering task generally performs in a low-dimensional embedding space, therefore we further project the graph-level features Hθ(Gi) into the cluster embedding Zθ,ϕ with a projector gϕ(·) parameterized by ϕ. The Euclidian-based clustering head measures the discrepancy between the graph-level features and each cluster center with Euclidian distance, which can be defined as follows:

where µc is the c-th cluster center of C clusters in total and can be initialized by k-means. zi|θ,ϕ = gϕ(Hθ(Gi)) ∈ Rdz is the cluster embedding for graph Gi and dz is the dimension of cluster embedding. Another clustering head is based on subspace clustering, with the assumption that the data can be divided by several subspaces when they are projected into a low-dimensional embedding space. Consequently, the subspace-based clustering head can be formulated by the similarity between the cluster embedding and the defined subspace base as follows:

where D ∈ RCd×d is the subspace base with d-dimension. It can be initialized by the column space of clusters generated by performing k-means on Zθ,ϕ. ∥z⊤i|θ,ϕD(c) ∥2F quantifies the 'closeness' (cosine similarity) between the graph-level feature zi|θ,ϕ and the c-th subspace D(c) . η is the smooth parameter and is fixed as 2. Particularly, the subspace D should obey two rules: 1) ∥D(c) i ∥ = 1, i = 1, . . . , d, c = 1, . . . ,C; 2) ∥D(i)⊤D(j) ∥F ≤ τ, i ̸= j, τ → 0. In practice, we impose two constraints by:

where ⊙ denotes the Hadamard product, I is an identity matrix with the same size as D. O is a special matrix whose elements in each d-size block are 0 and the others are 1. The first term ensures each base vector is of unit length, preventing the value of max∥z⊤j|θ,ϕD(c) ∥2F from being zero. The second term encourages the subspace bases to be mutually orthogonal, which is critical for maintaining distinct and separate subspaces.

After defining the learnable cluster assignment QEu and QSub from the distance and subspace perspectives, we can further yield a refined assignment, e.g., PEu, defined as:

where Nb represents the number of graphs in a batch, and the effect of PEu is to emphasize the high-confidence assignments in QEu. Similarly, we can derive PSub from QSub via:

Algorithm 1 Training flows of DCGLC.

Input: Graph set G, number of clusters C, batch size Nb, learning rate α, total training epochs T .

Output: The clustering results C.

• 1: Initialize the parameters θ, ϕ, ψ and cluster centers µ;
• 2: while not converge do
• 3: for t = 1, 2, . . . , T do
• 4: Generate augmented graph set ˆG fromG;
• 5: Obtain graph representations Hθ(Gi) and Hθ( ˆGi) with Eq. (2) and (3);
• 6: Obtain the cluster embeddings Zi, and cluster assignments QEu and QSub with Eqs. (5) and (6);
• 7: Compute refined cluster assignments PEu and PSub via Eqs. (8) and (9);
• 8: end for
• 9: Compute the total loss with Eqs. (4), (7), (10), (11), and (12);
• 10: Back-propagation and update network parameters θ, ϕ, ψ, and the cluster centers µ;
• 11: end while
• 12: Compute final cluster assignment QEu and QSub;
• 13: Obtain the clustering labels with Eq. (13);
• 14: return The clustering results C.

Note that the refined assignments serve as the pseudo labels to provide guidance for the update of QEu and QSub, which can be optimized by minimizing the discrepancy between the cluster assignment distribution and the refined assignment distribution, i.e.:

where KL(·||·) denotes the Kullback-Leibler (KL) divergence between two distributions. As previously discussed, our objective is to learn reliable and consistent clustering assignments by incorporating more comprehensive cluster information from multiple cluster perspectives (i.e., Euclidianbased one denoted by QEu and subspace-based one denoted by QSub) for each graph. However, clustering from different perspectives may yield inconsistent results due to diverse evaluation criteria, which ultimately affects the consistency of clustering and leads to suboptimal clustering performance. To address this challenge, we propose a cluster-perspective contrast objective ℓc|θ,ϕ for the cluster information alignment of different perspectives, which is defined as follows:

where C denotes the number of clusters, and the definitions of τ and sim(·, ·) are same to Eq. (4). Notably, we also demonstrate that aligning different cluster perspectives outperforms solely utilizing a single clustering head with a comprehensive experiment via an ablation study (in Section 4.2).

3.4 Training Strategy

The two modules in the proposed method are jointly optimized by minimizing the following total objective function:

where λ and β are trade-off parameters that control the contributions of ℓc|θ,ϕ and ℓKL|θ,ϕ to the total loss. Compared to (1), we not only comprise a well-defined representation learning module and clustering module, but also incorporate cluster information from different cluster perspectives. The dual contrastive learning in graph-level feature pairs and multiple cluster perspectives allows us to obtain more reliable cluster assignments. More importantly, our approach stands out as an end-to-end graph-level clustering method that is able to produce clustering results directly from the network, eliminating the reliance on performing other clustering methods (e.g., k-means and SC) on the learned graph-level representations. The clustering results C for each graph can be obtain from the trained QEu and QSub by:

Algorithm 1 summarizes the detailed training flows of the proposed DCGLC.

3.5 Complexity Analysis

Here, we further theoretically analyze the time and space complexity for the proposed DCGLC method. Considering N graph with average n nodes and |E| edges per graph, the number of GIN layers and clusters are K and C, and the dimensions of nodes and each hidden layer are m and dh. We analyze the time and space complexity of DCGLC as follows:

• The time complexity of the first layer and the other layers in GIN are approximate O(Nn|E|mdh) and O(Nn|E|d2h ), where |E| typically can be ignored when the adjacency matrix is sparse. Besides, the time complexity of each cluster head is O(NKdhC). Therefore, the time complexity of DCGLC is O(Ndh(nm + (K - 1)ndh + 2KC)).
• Regarding space complexity, the GIN backbone in DCGLC needs to store the neighbor list and feature matrices of the entire graph to represent the graph structure. Thus the space complexity of these processes is O(Nn +N|E| +Nnm). Besides, the computation process of the GIN needs to store the intermediate results of each GIN layer. Thus the space complexity of this process for the GIN with K layers is O(Nndh(m+ (K - 1)dh)). The space complexity of the two cluster heads is O(NKdhC). Overall, the space complexity of DCGLC is O(Nn+N|E|+Nnm+Nndh(m+ (K - 1)dh) + 2NKdhC).

From the above discussion, we can observe that the time and space complexities of DCGLC are mainly associated with the number of nodes and graphs, i.e., n and N, which is competitive to GCL-based baselines such as GraphCL [You et al., 2020], JOAO [You et al., 2021], and GLCC [Ju et al., 2023].

3.6 Discussion with Previous Works

Numerous works have paved the way for this work, and we briefly discuss the connections between our work and them.

The GCL methods, e.g., GraphCL [You et al., 2020] and JOAO [You et al., 2021], can be applied to graph-level clustering by performing k-means [Hartigan and Wong, 1979] or SC [Ng et al., 2001] on the learned representations. Compared with them, DCGLC allows graph-level representations for clustering tasks to be learned in an end-to-end manner, which guides graph representation learning with an explicitly defined clustering objective.

Some works also delved into end-to-end frameworks for jointly optimizing representation learning and clustering. For example, Xie et al. [2016] and Cai et al. [2022a] respectively investigated distance-based and subspace-based endto-end clustering models. However, they are not committed to clustering graph-level data and considered from a single cluster perspective. Ju et al. [2023] proposed a graph-level contrastive clustering method for learning clusters through instance-level and cluster-level contrast. However, the cluster-level contrast in [Ju et al., 2023; Li et al., 2021] focuses on aligning the logits prediction from the original and augmented data, which is quite different from ours. DCGLC introduces two cluster heads to explicitly define clusters, and focuses on leveraging comprehensive cluster information from different cluster perspectives to learn more reliable cluster assignments, which is substantiated by the ablation study results in Section 4.2.

4 Experiment

In this section, we conduct a series of evaluations to demonstrate the superiority of the proposed DCGLC.

4.1 Experimental Settings

Datasets. We consider three types of graph benchmark datasets to evaluate the clustering performance, including seven molecule datasets (MUTAG, BZR, PTC-MR, PTCMM, COX2, ER MD, and AIDS), three biological datasets (DD, PROTEINS and ENZYMES), and two social network datasets (IMDB-BINARY and REDDIT-MULTI-5K). Details of these benchmarks refer to Appendix A.

BaselineMethods. We compare the proposed method with several state-of-the-art baselines, which can be broadly classified into three categories including: 1) Graph Kernel: SP [Borgwardt and Kriegel, 2005], GK [Shervashidze et al., 2009], RW [Vishwanathan et al., 2010], WL [Shervashidze et al., 2011], LT [Johansson et al., 2014], WL-OA [Kriege et al., 2016], 2) Unsupervised Graph Representation Learning: InfoGraph [Sun et al., 2020], GraphCL [You et al., 2020], JOAO [You et al., 2021], 3) Graph-Level Clustering: GWF [Xu et al., 2022], GLCC [Ju et al., 2023]. Note that the network structure of InfoGraph [Sun et al., 2020], GraphCL [You et al., 2020], JOAO [You et al., 2021], and GLCC [Ju et al., 2023] are exactly the same as ours to guarantee a fair comparison.

Implementation Details. We employ a 4-layer GIN [Xu et al., 2019] as the backbone network for the proposed DCGLC, with the aggregated dimension set to 16. We utilize an MLPbased feature projection head and cluster projection head, with the dimension of the latent layer and the clustering embedding layer both set to 10. The number of clusters is set to the number of categories in the dataset, the batch size is fixed to 64, and the training epoch is set to 300 for all datasets. We use Adam as the optimizer and follow the setting in [You et al., 2021] to augment graphs automatically in each epoch. Other details and the setup of baseline models refer to Appendix B.

Evaluation Metrics. We use clustering accuracy (ACC), normalized mutual information (NMI), and adjusted rand index (ARI) as the evaluation metrics, and report the means and standard deviations of the three metrics based on 10 independent runs for each dataset.

4.2 Experimental Results

Comparison with State-of-the-art Baselines. Tables 12 present the experimental results on eight graph benchmarks from different data types (more results refer to Appendix C), where we compare the proposed DCGLC method with eleven state-of-the-art baselines. From this table, we have the following observations: 1) The proposed DCGLC

DCGLC

59.98±1.11 2.60±0.89 3.66±0.92 65.48±0.24 4.32±0.70 7.34±0.15 78.63±0.16 3.05±0.82 5.42±3.15 66.95±0.18 10.12±0.88 11.29±0.25

Table 2: Clustering performance (mean ± std) on PTC-MR/MM, COX2, and ER MD (%). The results highlighted in bold and underlined

denote the best and runner-up performance respectively.

-100 -80 -60 -40 -20 0 20 40 60 80 100 -80 -60 -40 -20 0 20 40 60 80 100 0 1 method demonstrates significant superiority over state-of-theart graph kernels and graph-level clustering baselines. Particularly, compared to the GCL-based baselines, e.g., GraphCL, JOAO, and GLCC, the proposed DCGLC exhibits notable advantages across three different types of graph data. For example, on the molecule dataset MUTAG, DCGLC exceeds JOAO+KM with an improvement of 7.50%, 6.23%, and 19.28% in terms of ACC, NMI, and ARI. Besides, on the biology dataset DD, DCGLC also outperforms the runnerup GLCC by more than 14.00% across all metrics. These results clearly indicate that the dual contrastive mechanism in DCGLC can facilitate the learning of clustering-friendly graph representations. 2) Empirical results show that the one clustering perspective may not consistently yield superior performance. For instance, GWF+KM outperforms GWF+SC on PTC-MR and BZR, while the opposite is true for MUTAG and ER MD. This phenomenon is also observed in other state-of-the-art baselines, e.g., GraphCL, and JOAO. These observations suggest that different clustering perspectives reveal distinct information for clustering, which may exhibit different validity for various graph data. Besides, this also highlights the rationale for leveraging the cluster information from multiple clustering perspectives to promote graph-level clustering.

Visualization of Clustering Embedding. We employ tSNE [Van der Maaten and Hinton, 2008] to visualize the clustering embeddings to facilitate a more intuitive comparison. As shown in Figure 2, it is evident that InfoGraph yields a dis-

-100 -80 -60 -40 -20 0 20 40 60 80 100 -80 -60 -40 -20 0 20 40 60 80 0 1 organized result with no discernible clusters. While GraphCL and JOAO demonstrate better performance compared to InfoGraph, the samples of the same color are still widely dispersed. Besides, although GWF shows clearer boundaries between classes, the gap between the two classes is small, resulting in blurred clustering boundaries. Compared with them, the clustering structures of DCGLC are more compact, with different clusters more separated and samples from the same cluster being significantly closer in proximity. The improved compactness and separability of the clusters indicate the effectiveness of DCGLC in capturing and leveraging the underlying structure and relationships within the graph data for more accurate clustering.

Impact of λ and β. We evaluate the impact of the variation of two hyper-parameters, λ, and β, on the clustering performance. Specifically, we conduct the parameter analysis by intentionally setting the values of λ and β within a broad range of [0.001, 100]. The results depicted in Figure 3 show the performance variations observed on MUTAG (more results refer to Appendix D). The empirical results reveal that excessively high values of λ generally lead to a negative impact on clustering performance. For instance, when λ is set to 100, the performance becomes unsatisfactory. Interestingly, our method demonstrates a remarkable insensitivity to changes in the value of β, as evidenced by its consistently stable performance across different values. Notably, the ARI tends to be more sensitive to variations in both λ and β. Overall, these findings collectively underscore the robustness and stability of DCGLC across a broad range of λ and β values.

Ablation Study. We conduct an ablation study to demonstrate the effectiveness of the proposed method and the rationale of integrating the cluster information from different perspectives. We construct three degradation variants for the proposed DCGLC, including:

1. DCGLC v1: Drop out the cluster-perspective contrast component, and only consider the Euclidian-based clustering head and graph-level features contrast.
2. DCGLC v2: Drop out the cluster-perspective contrast component, and only consider the subspace-based clustering head and graph-level features contrast.
3. DCGLC v3: Drop out the graph-level features contrast component and only train under the guidance of clusterperspective contrast.

In Table 3, we thoroughly evaluate DCGLC in comparison to all degradation variants on three datasets (more results refer to Appendix E), revealing several key insights: 1) The Euclidean-based and subspace-based cluster perspectives each possess their own set of strengths and weaknesses. The former demonstrates superior performance on MUTAG, whereas the latter outperforms on BZR. This highlights the importance of considering different clustering perspectives, as their effectiveness can vary depending on the characteristics of graph data. 2) Relying solely on a single cluster perspective is insufficient for capturing comprehensive cluster information, as we observe consistent performance degradation DCGLC v1 and DCGLC v2. The results provide strong evidence for the validity of incorporating cluster information from different cluster perspectives in graph-level clustering. 3) The importance of graph-level features contrast is demonstrated from the results of DCGLC v3. Notably, removing this component leads to noticeable performance degradation, reaffirming its indispensable role in enabling the network to learn more effective graph representations.

5 Conclusion

In this paper, we proposed dual contrastive graph-level clustering (DCGLC). DCGLC leverages graph contrastive learning and introduces an explicit clustering objective to learn discriminative graph-level representations that are conducive to clustering. We highlight the importance of capturing comprehensive cluster information from multiple cluster perspectives to graph-level clustering, and propose a contrastive mechanism to align cluster information from different cluster perspectives. DCGLC is an end-to-end framework that integrates graph contrastive learning and cluster-perspective contrast, leading to a mutual enhancement during training. Extensive experiments on several popular graph benchmarks compared with state-of-the-art methods fully demonstrate the superiority of DCGLC. The main limitation of this work stems from the exploration of the graph augmentation mechanism beneficial for capturing better clustering structures, which is a promising research issue for future work.

Acknowledgments

This research is supported by the National Natural Science Foundation of China (Grants No.U21A20472, No.62276065, and No.62376236), and Shenzhen Research Institute of Big Data (Grant No.T00120210002).

Additionally, this research is supported by the National Research Foundation Singapore and DSO National Laboratories under the AI Singapore Programme (AISG Award No: AISG2-RP-2020-018). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not reflect the views of National Research Foundation, Singapore.

Contribution Statement

Jinyu Cai and Yunhe Zhang contributed equally to this work as co-first authors.