GBFlow: Grouping Belief-Guided Dual Normalizing Flows for Accurate Spatial Domain Delineation

1 Introduction

Spatially resolved multi-omics technologies enable the joint measurement of molecular profiles and spatial coordinates, providing powerful means to investigate tissue organization, cellular heterogeneity, and microenvironmental interactions [Moses and Pachter, 2022; Duan et al., 2023]. A key analytical task in this context is spatial domain identification, which aims to delineate coherent tissue regions by jointly leveraging molecular similarity and spatial proximity [Cai and Li, 2025; Pham et al., 2023].

∗Corresponding author.

Most existing approaches rely on graph neural networks or graph autoencoders to model spatial and transcriptional relationships [Hu et al., 2021; Li et al., 2022]. These methods construct affinity graphs based on spatial proximity, molecular similarity, or both, and perform neighborhood aggregation to capture local context. Representative examples include HyperGCN [Ma et al., 2024], which models higherorder spatial interactions via hypergraphs, and SpaMGCN [Zhang et al., 2025b], which employs multi-scale adaptive graph convolutions for spatial multi-omics integration. Overall, these studies demonstrate that graph-based structural priors can substantially enhance spatial domain delineation.

However, the effectiveness of graph-based models critically depends on the quality of the predefined affinity graph [Kang et al., 2025; Lu et al., 2025]. In practice, spatial graphs are often constructed using heuristic rules such as k-nearest neighbors or fixed-radius thresholds, which may fail to reflect true biological neighborhoods [Gong et al., 2025]. As a result, spurious cross-region connections frequently arise, especially near tissue boundaries or in heterogeneous regions with complex cellular compositions. Such noisy or misleading edges can propagate erroneous signals across distinct tissue domains, leading to representation distortion, oversmoothing, and degraded clustering performance[Jiang et al., 2025b; Zhang et al., 2025a]. Moreover, once the graph structure is fixed, most GNN-based methods lack mechanisms to adaptively correct unreliable connections during training, making them highly sensitive to graph construction choices. This issue is also widely observed in general clustering tasks, where separate graph construction and clustering learning limits performance and robustness[Jiang et al., 2025a].

More fundamentally, graph autoencoders tend to overregularize latent representations, biasing them toward the imposed topology and limiting their capacity to capture global distributional variability. Conversely, structure-agnostic models lack explicit mechanisms to enforce spatial coherence. Achieving a principled balance between structureawareness and distributional flexibility remains an open challenge.

To address these challenges, we propose GBFlow, a grouping belief-guided dual normalizing flow framework for spatial domain identification. GBFlow refines latent representations through two complementary flows: a structureaware flow that enforces relational consistency using a refined affinity graph, and a parallel structure-free flow that preserves global distributional flexibility. The outputs are adaptively fused to produce robust view-specific embeddings that capture both spatial coherence and molecular similarity. Additionally, a grouping belief-based affinity refinement (GBAR) strategy suppresses unreliable connections and strengthens confident neighborhoods. To handle multi-view spatial omics data, GBFlow integrates preliminary feature extraction, cross-graph feature integration, inter-omics attention, and dynamic graph learning, including structure-aware contrastive learning to enhance global consistency while preserving local diversity. These modules collectively enable progressive refinement of both view-specific and global affinity graphs, balancing structural constraints with distributional flexibility.

Our main contributions are summarized as follows:

• We propose a dual-flow latent refinement framework that jointly integrates structure-aware and structure-free normalizing flows, enabling a flexible yet consistent representation of spatial multi-omics data.
• We introduce a grouping belief-based affinity refinement strategy that effectively suppresses noisy connections and enhances reliable neighborhood relations.
• Extensive experiments on multiple spatial multi-omics datasets demonstrate that GBFlow consistently outperforms state-of-the-art methods in both accuracy and robustness.

2 RelatedWork

2.1 Single-omics Spatial Domain Identification

Early studies on spatial domain identification primarily relied on classical clustering methods applied to gene expression profiles, sometimes combined with spatial smoothing [Sun et al., 2025; Xu et al., 2025]. While simple and interpretable, these approaches often fail to explicitly model spatial relationships, resulting in fragmented or biologically inconsistent domain assignments.

More recent methods increasingly adopt graph neural networks or graph autoencoders to capture both spatial and transcriptional dependencies. Representative approaches include STAGATE [Dong and Zhang, 2022] and GraphST [Long et al., 2023], as well as more advanced models such as SpaMGCN [Zhang et al., 2025b] and HyperGCN [Ma et al., 2024]. Although these methods achieve improved performance, their effectiveness is highly sensitive to the quality of the predefined affinity graph. Some works attempt to mitigate this issue through graph refinement or structure learning, such as UnGSL [Han et al., 2025] and ARO-GNN [Lu and Lin, 2025], but they still rely on a single graph-based transformation and may over-regularize latent representations, limiting their ability to capture global variability.

2.2 Multi-omics Spatial Domain Identification

With the increasing availability of spatial multi-omics data, recent methods aim to integrate heterogeneous modalities to improve spatial domain delineation. A first line of work incorporates histological image information into spatial transcriptomics analysis. SiGra [Tang et al., 2023] integrates gene expression, immunohistochemistry images, and spatial information via a hybrid graph transformer, while STMDA [Wang et al., 2024] employs modality-specific GCNs followed by cross-modal distribution alignment to enhance cellular niche identification.

Beyond image-based integration, other approaches focus on jointly modeling multiple molecular modalities without relying on imaging data. SpatialGlue [Long et al., 2024] constructs omics-specific graphs and integrates them through attention-based cross-modal fusion. PRAGA [Huang et al., 2025] captures dynamic semantic relations across modalities via latent feature graphs, combining spatial priors with a prototype-based contrastive learning framework. While effective, existing spatial multi-omics methods often treat different graphs independently or rely on heuristic fusion strategies, and few explicitly address graph noise or the trade-off between structural consistency and distributional flexibility.

In contrast, our proposed GBFlow explicitly decouples structure-aware and structure-free transformations via dual normalizing flows, and introduces a grouping belief-based affinity refinement strategy to suppress unreliable connections, enabling more robust spatial domain identification.

3 Methods

Given a spatial multi-omics dataset consisting of V views, we denote the multi-view feature matrices as X = {X(v) ∈ Rn×dv }Vv=1 , where n is the number of spatial spots (or cells), and dv is the feature dimension of the v-th view. The central objective of our framework is to learn a robust view-specific latent representations Z(v) ∈ Rn×l for each omics view, along with a unified integrated representation Z ∈ Rn×l . These representations are designed to effectively capture both spatial coherence and molecular similarity, thereby enabling accurate identification of spatial domains through clustering. At the core of the proposed framework (named GBFlow) lies the construction and progressive refinement of multiple graph structures. For each view v, we construct two initial graphs: the spatial proximity graph A(v) 1 and the feature similarity graph A(v) 2 . These are subsequently refined and integrated into a fused graph A(v) 3 , which is further evolved into the final refined affinity graph G(v) . Key learnable components of the model include: the dual-branch normalizing flow outputs ˆZ(v) graph and ˆZ(v) plain , their adaptively fused representation ˜Z(v) , and the dynamic self-expression graph H(v) employed for subspace modeling.

The overall architecture of GBFlow integrates four major modules: (1) Preliminary Feature Learning and Integration, (2) Dual-flow Latent Refinement, (3) Grouping Belief-based Affinity Refinement, and (4) Dynamic Graph Learning. All components are jointly optimized under a unified loss function L.

3.1 Preliminary Feature Learning and Integration Feature Learning

To effectively capture the heterogeneous information in spatial transcriptomics data, we construct multiple complementary graph structures that jointly model spatial proximity and transcriptional similarity. For each view v, we first construct a spatial neighborhood graph A(v) 1 based on the Euclidean distances between spatial coordinates. Recognizing that spatial proximity does not always imply functional similarity, we additionally build a feature-based similarity graph A(v) 2 in the gene expression space. To mitigate spurious connections across distinct regions, we refine the spatial graph by incorporating transcriptional similarity:

where ⊙ denotes element-wise multiplication.

We then obtain a fused graph that balances spatial continuity and expression consistency as follows:

where η(v) ∈ [0, 1] is a learnable parameter.

Using the spatial graph A(v) 1 , feature similarity graphA(v) 2 , and the fused graph A(v) 3 , we employ three graph convolutional encoders to extract view-specific latent representations:

where Fθ(v) k (·) denotes a graph convolutional operation.

Cross-graph Feature Integration

The latent representations obtained from the three graph encoders are first concatenated:

To capture inter-feature correlations and adaptively integrate information from different graph views, we apply learnable importance weights followed by a projection layer:

where u(v) k represents the learned importance coefficient for the representation derived from the k-th graph, and F(·) is a linear transformation.

Inter-omics Integration

To capture cross-modal dependencies across different omics views, we apply a multi-head self-attention mechanism over the view-specific representations. The outputs of all attention heads are concatenated:

where ˜ Zh denotes the output of the h-th attention head. The final refined representation for each omics view is then obtained by splitting ˆ Z along the modality dimension.

3.2 Dual-flow Latent Refinement

Normalizing flows offer an invertible and probabilistic framework for mapping a simple base distribution to a complex target distribution via a sequence of bijective transformations. We extend this paradigm to spatial multi-omics data by introducing a dual-flow architecture that explicitly disentangles relational (graph-structured) consistency from global distributional flexibility.

Graph Normalizing Flow for Structured Transformation

The structure-aware branch employs a Graph Normalizing Flow (GNF) [Liu et al., 2019], which incorporates reversible message passing to enable exact likelihood computation on graph-structured data while preserving invertibility. Following the GRevNet design, at each layer t, the node feature matrix Z(v) t is split along the feature dimension into Z(0) t and Z(1) t . A reversible transformation step consists of two halfsteps:

where F1, F2,G1,G2 are graph-conditioned transformation functions implemented as Graph Neural Networks (GNNs)

that operate on the refined affinity matrix G(v) . This formulation ensures that the entire transformation is invertible:

The Jacobian of each step is block triangular, allowing efficient computation of the log-determinant required for likelihood evaluation.

Vanilla Normalizing Flow for Global Flexibility

In parallel, we employ a structure-free Vanilla Normalizing Flow (VNF) based on RealNVP-style affine coupling layers. This flow operates independently on each node's feature vector without considering graph structure, thereby preserving global distributional flexibility. Given a latent vector z ∈ Rd , we split it into two halves z1, z2 ∈ Rd/2 . A coupling layer is defined as:

where smmlp and tmmlp are Multi-Layer Perceptrons (MLPs). The triangular Jacobian ensures tractable likelihood computation.

Dual-flow Architecture and Fusion

Given the refined multi-omics representation ˆZ(v) , we process it through both flows:

The two complementary representations are then adaptively fused:

where α ∈ [0, 1] is a learnable parameter. This dual-flow design explicitly separates structural constraints from global distribution modeling, allowing GBFlow to balance relational coherence with distributional expressiveness and produce robust latent embeddings for spatial domain identification.

3.3 Grouping Belief based Affinity Refinement

To suppress noisy graph connections, we introduce a Grouping Belief based Affinity Refinement (GBAR) strategy. Given similarity matrices A(v) h (from dynamic subspace) and A(v) a = A(v) 3 , we first construct top-t neighbor masks M(v) h and M(v) a . From these, soft relation matrices are computed as:

Next, hard relation matrices G(v) h and G(v) a are obtained by thresholding with confidence θ. The GBAR loss consists of two terms:

where g(v) = (G(v) h +G(v) a )/2, and β1, β2 are hyperparameters. The total GBAR loss is:

The refined graph G(v) resulting from this procedure is subsequently used in GNF to guide structure-aware latent transformations.

3.4 Dynamic Graph Learning

Shared and Omics-specific Subspace Modeling

To simultaneously capture globally shared structures and omics-specific dependencies, we propose a dynamic subspace modeling strategy. We first construct a global selfexpression matrix: C = ¯c¯c⊤, where ¯c denotes a lowdimensional latent embedding capturing global sample relationships. For each omics v, we additionally learn an omicsspecific self-expression matrix Q(v) to encode modalitydependent connectivity patterns. The adaptive graph structure for each omics is thus defined as:

The self-expression reconstruction objective is:

where L1 enforces graph smoothness and omics disentanglement.

Structure-aware Contrastive Learning

We construct a unified affinity graph by aggregating the fused graphs learned from all omics:

Based on this structure-aware affinity, we design a contrastive objective:

where P v i and N v i denote aggregated similarities over positive and negative sample pairs determined by W:

where ˜ zv i ∈ Rn×l denotes the latent feature vector of sample i in view v before self-expression, obtained from the dual-flow latent refinement module, and ¯¯ zv i ∈ Rn×l represents the selfexpressed feature after applying the dynamic self-expression model, i.e., ¯¯ Z (v) = H(v) ˜ Z (v) . Both are l-dimensional latent embeddings. g(·, ·) is the cosine similarity, and ρW ij is an indicator function defined as:

3.5 Overall Loss Function and Spatial Domain Identification

To ensure that the cyclic autoencoder preserves both structural and semantic information, we employ a reconstruction loss Lrec that penalizes discrepancies in both node features and self-representations. The overall training objective integrates all components of our framework:

For spatial domain identification, we construct the final affinity matrix by combining the global and omics-specific self-expression components:

Spectral clustering is then applied on H to obtain the final spatial domain assignments, yielding coherent and biologically meaningful spatial partitions. The overall algorithm process is detailed in the supplementary materials.

4 Experiments

4.1 Datasets and Preprocessing

In this study, we utilize eight datasets: human breast cancer[Long et al., 2024], mouse brain[Long et al., 2024; Dong and Zhang, 2022], human lymph node[Long et al., 2024], and mouse embryo[Jiang et al., 2023]. Datasetspecific information and experimental settings are provided in the supplementary materials. For the breast cancer and mouse brain datasets, we select the top 3,000 highly variable genes, apply normalization and log transformation, and reduce dimensionality to 40 via PCA. Whole-slide histology images are cropped into 50×50 patches based on spatial coordinates and flattened into 1D vectors. For the lymph node dataset, RNA is filtered to retain genes expressed in at least 10 cells, normalized, log-transformed, and reduced to 100 dimensions via PCA; paired ADT data are similarly normalized, scaled, and PCA-reduced. For the mouse embryo dataset, RNA undergoes filtering, highly variable gene selection, normalization, log transformation, and PCA reduction to 50 dimensions; ATAC features detected in fewer than 6% of cells are removed and projected to 21 dimensions via LSI.

4.2 Baseline andMetrics

We compare GBFlow with seven baseline methods, including GraphST[Long et al., 2023], STAGATE[Dong and Zhang, 2022], PAST[Li et al., 2023], SpatialGlue[Long et al., 2024], PRAGA[Huang et al., 2025], stDCL[Yu et al., 2025], and DUSTED[Zhu et al., 2025]. To comprehensively evaluate spatial domain delineation, we employ a set of standard metrics: Accuracy (ACC), F1-Score, Jaccard coefficient, Adjusted Rand Index (ARI), Mutual Information (MI), Normalized Mutual Information (NMI), Adjusted Mutual Information (AMI), V-Measure, and Completeness (Compl.). Across these metrics, GBFlow consistently achieves top or near-top performance.

4.3 Experimental Results

Qualitative Experimental Results

GBFlow consistently achieves state-of-the-art or highly competitive performance across diverse spatial multi-omics datasets. As shown in Table 1 and Supplementary Table 1, our method yields the highest ACC (76.65%) on the Human breast cancer dataset, outperforming strong baselines such as STAGATE and DUSTED. Notably, in the high-complexity Mouse brain anterior task, which involves 52 distinct clusters, GBFlow demonstrates superior precision by securing top scores in FMI (48.98%), ARI (44.65%), and ACC (47.38%). The model's robustness is further validated on the Human lymph node and Mouse embryo datasets, where it maintains stable and leading performance in terms of NMI and ARI, underscoring its ability to generalize effectively across varying degrees of tissue heterogeneity and cluster scales.

An interesting phenomenon observed in Table 1 is that the ranking of the same method can vary across different evaluation metrics. For example, on the Human lymph node D1 dataset, PRAGA achieves the highest MI (65.42) but a much lower ARI (24.94%), while GBFlow attains a slightly lower MI (65.37%) but leads in ARI (34.41%). This highlights the distinct emphases of different evaluation metrics: information-theoretic measures such as MI and NMI focus more on the overall alignment between feature distributions and label distributions, whereas ARI and FMI place greater emphasis on the pairwise consistency of clustering results. GBFlow's consistent superiority in the latter category directly reflects how its grouping belief module enhances the pairwise accuracy of clustering by reinforcing high-confidence neighborhood relationships.

The performance gains of GBFlow stem from its dualflow latent refinement strategy, which integrates a structureaware Graph Normalizing Flow (GNF) with a structurefree Vanilla Normalizing Flow (VNF). This design enables the model to preserve local spatial coherence while capturing global molecular distribution patterns. In addition, the grouping belief-based affinity refinement module effectively suppresses noise in the initial graphs and reinforces highconfidence neighborhood relationships. By pruning spurious connections and enhancing discriminative features, GBFlow learns more separable latent representations, leading to improved spatial domain identification.

Quantitative Experimental Results

Figure 2 shows GBFlow's ability to delineate histologically meaningful regions in human breast cancer tissue, including desmoplastic changes, lymphocyte infiltration, and necrosis/hemorrhage. Compared with baseline methods, which exhibit over-segmentation (GraphST), boundary blurring (stDCL), or internal noise (PAST, DUSTED), GBFlow produces sharper boundaries and more coherent intra-region structures, reflecting effective integration of multi-omics signals and the benefit of affinity refinement.

Figure 3 illustrates GBFlow's preservation of fine-grained anatomical organization in mouse brain coronal sections. While STAGATE and GraphST show layer fragmentation and stDCL blurs inter-layer transitions, GBFlow maintains clear laminar boundaries (L1-L4) and continuous intra-layer homogeneity. The dual-flow design ensures neighborhood consistency within layers (GNF) and captures global expression variability across layers (VNF), yielding spatially smooth and anatomically faithful partitions. Overall, GBFlow demonstrates higher structural consistency and robustness across diverse tissue types, with quantitative improvements in ACC, ARI, and FMI corroborating these observations.

5 Conclusion

We propose GBFlow, a novel grouping belief-guided dual normalizing flow framework for robust spatial domain identification. By integrating structure-aware and structure-free normalizing flows alongside a grouping belief-based affinity refinement strategy, GBFlow effectively balances relational consistency with global distributional flexibility. Experiments across eight spatial multi-omics datasets demonstrate that GBFlow consistently outperforms state-of-the-art methods, achieving more accurate and robust spatial delineation. Additional analyses including marker gene visualization, latent representation visualization, and scalability analysis are provided in the supplementary material. The framework provides a principled approach to multi-omics integration and is widely applicable to spatially resolved biomedical data analysis.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (62372279, 62322215, 62532017), the Natural Science Foundation of Shandong Province (ZR2025QB62, ZR2023MF119), and the Natural Science Foundation of Hunan Province (2026JJ30018). This study was also supported in part by the High-Performance Computing Center of Central South University.