VGDM: Visual Localization-Guided 3D Dental Segmentation via Extrinsic–Intrinsic Bridging

1 Introduction

3D dental crown segmentation is a fundamental task in digital dental diagnosis and treatment and plays a critical role in downstream applications such as 3D dental modeling, orthodontic design, and treatment planning [Zhang et al., 2025]. The reliability of the segmentation results directly affects the stability of the subsequent process. Unfortunately, in real intraoral scans data(IOS), tooth occlusion, scanning noise, and reconstruction artifacts often destroy the geometric structure between teeth and introduce inconsistent connections with real tooth boundaries into the geometric representation.

In order to solve the automatic crown segmentation task, previous studies have proposed numerous point-based and mesh-based deep learning methods, gradually replacing the early traditional geometric methods [Cui et al., 2021; Xiong et al., 2023]. These methods usually model and propagate features based on local geometric neighborhoods, and have achieved good performance on public tooth segmentation datasets. However, under complex geometry conditions, even if the overall segmentation index is high, the segmentation results are still prone to structure-related errors.

In real dental scans, the restricted scanning angles and artifacts during the reconstruction process often disrupt the connection relationships between adjacent teeth, causing the originally anatomically separated teeth to have unreasonable connections. Current point cloud or mesh-based segmentation methods usually rely on these geometric connectivity relationships to transfer information between neighboring regions. Once the information is transmitted along the incorrect geometric connections, there is a tendency for information to be mixed between adjacent teeth, triggering structural errors such as merging or excessive segmentation.

Such structural errors caused by geometric connectivity are often difficult to naturally eliminate merely by enhancing the feature expression ability of the network. Even if the model can learn discriminative geometric features locally, when the information is transmitted along the incorrect geometric connections, its influence may still spread to adjacent regions, thereby interfering with the determination of instance boundaries. To mitigate this issue, one promising direction is to introduce robust external visual guidance. A rendered 2D view can provide relatively stable boundary cues even when the mesh surface contains unreasonable connections. Meanwhile, exploiting intrinsic feature consistency and geometric smoothness on the surface can regularize propagation under geometric noise and reduce boundary ambiguity.

Based on these observations, we propose VGDM (VisualGuided Diffusion Modulation), a framework that enhances the robustness of feature propagation to geometry-induced spurious connections without explicitly modifying the connectivity of the 3D mesh. The core idea is a visually guided operator modulation strategy, which leverages relatively stable cues from a rendered 2D view to construct an anisotropic modulation field on the 3D surface and modulate the Laplace-Beltrami operator (LBO) accordingly. With the modulated Laplace-Beltrami operator, we employ Laplace diffusion, since the LBO provides an intrinsic surface metric, yielding smoother and more coherent boundaries and making predictions less sensitive to local geometric noise. By increasing the resistance to feature diffusion across adhesion regions, VGDM effectively reduces over-segmentation of individual teeth and incorrect merging of adjacent teeth.

The main contributions are summarized as follows:

• We propose VGDM, a visual-guided diffusion framework for robust tooth instance segmentation under distorted geometric connectivity.
• We design an operator-level diffusion modulation mechanism guided by 2D visual information, which suppresses erroneous feature propagation across adhesion regions without altering mesh topology.
• We develop a dual-stream spectral diffusion framework that improves robustness to imperfect visual cues while preserving intrinsic geometric coherence, leading to more stable tooth instance segmentation.

2 RelatedWork

Accurate crown segmentation from IOS data is fundamental to digital dentistry. While many learning-based methods show promising results on standard public benchmarks [Sun et al., 2020; Ben-Hamadou et al., 2023], due to the scanning noise and reconstruction artifacts in the actual clinical IOS data [Fratila et al., 2025], these factors can introduce unreasonable geometric connections on the mesh surface, which disrupt the originally clear anatomical boundaries between teeth. In such cases, tooth segmentation methods based on local geometric neighborhoods or fixed surface connection relationships are prone to structural errors such as incorrect fusion of adjacent teeth or excessive segmentation of single teeth. Most existing approaches operate on point clouds or meshes and aggregate local neighborhoods [Zanjani et al., 2019; Xiong et al., 2023; Duan and Chen, 2023], sometimes with task-specific post-processing or multi-stage pipelines. Their core assumption is that local Euclidean neighborhoods or fixed surface connectivity reliably reflect tooth boundaries. In challenging IOS (adhesion, noise, partial scans), this assumption breaks and can cause cross-tooth mis-merging or over-segmentation.

Most existing point cloud or mesh-based segmentation methods rely on the geometric neighborhood relationships for feature modeling and information transmission in the spatial domain. The early PointNet series of methods introduced modeling of local geometric structures through local region division and feature aggregation [Qi et al., 2017a; Qi et al., 2017b]; subsequently, methods based on graph neural networks explicitly constructed neighborhood graphs and performed message passing based on geometric connectivity, thereby enhancing the local context modeling ability [Wang et al., 2019]. And spectral methods such as DiffusionNet [Sharp et al., 2022] have further modeled feature propagation as a diffusion process defined on surfaces, achieving modeling of the intrinsic geometric structure through the Laplacian-Beltrami operator [Reuter et al., 2009], demonstrating advantages in maintaining coordinate invariance and resolution stability. Although these methods model the geometric structure at different levels, their common feature is that the feature transmission process is driven by local geometric neighborhoods or surface connectivity, and implicitly assumes that these geometric relationships can reliably reflect the true anatomical boundaries. When artifacts or adhesion phenomena introduce incorrect geometric connections, this type of feature transmission based on geometric connectivity may occur along unreasonable paths, resulting in crossinstance information mixing and structural segmentation errors.

3 Method

3.1 ProblemFormulation

We represent each IOS as a triangular mesh surface M = (V, E,F), where V = {vi}Ni=1 denotes the vertex set with vertex coordinates vi ∈ R3 , and E and F denote the sets of mesh edges and triangular faces, respectively. For each vertex vi, we construct an input feature vector xi ∈ Rd , such as coordinates, Gaussian curvatures, or other local geometric descriptors, and stack them as

Our goal is tooth-wise instance segmentation: given M and X, we predict a per-vertex instance assignment ˆ zi ∈ {1, . . . , ˆK} and a label predict L, where K denotes the (ground-truth) number of tooth instances in the scan (varying across samples) and ˆK denotes the number of predicted instances, and L represents the prediction from the predicted instance to the real tooth FDI label. Equivalently, the model may output per-vertex embeddings ei ∈ Rq and obtain instances via clustering or connected-component grouping. We denote the final set of predicted instances as P = {pj} ˆKj=1 , where each pj ⊆ V is the vertex subset corresponding to a predicted tooth instance.

3.2 Overview

Given the IOS mesh M and its vertex features X, we propose Visual-Guided Diffusion Modulation (VGDM) for tooth instance segmentation. The core idea of VGDM is to use singleview visual cues to constrain information propagation on the mesh - increasing the cross-edge propagation impedance in potential inter-tooth regions, making it more difficult for features to diffuse across tooth gaps, thereby suppressing crosstooth false coupling and reducing instance-level merge/split errors.

As shown in Figure 1, the overall process of VGDM is summarized as follows:

1. Visual Cues Modeling. We render the mesh from a near-occlusal view and apply a 2D vision model to obtain dense toothness predictions. These cues are then projected back onto the mesh to produce vertex-wise visual confidence, and the 2D detections are used to localize candidate tooth regions as local 3D surface patches.
2. Diffusion Modulation. Based on the projected cues, we construct two Laplace-Beltrami operators on the same

patch: the original operator L and a visually modulated operator Lmod. While L preserves intrinsic geometric diffusion, Lmod down-weights diffusion conductance across potential inter-tooth boundaries, preventing feature leakage to adjacent teeth.

(3) Dual-stream DiffusionNet. We adopt DiffusionNet as the 3D backbone (where "diffusion" refers to intrinsic PDEbased feature propagation rather than generative diffusion). VGDM employs a dual-stream architecture that propagates features under L and Lmod in parallel, and fuses the two streams to produce final predictions for each patch. This design improves robustness to adhesion-induced structural errors while mitigating imperfect 2D visual cues.

3.3 Visual Cues Modeling

This section describes how we obtain single-view visual cues and visibility, and map them back to the mesh. We render the mesh from a near-occlusal view (an approximate topdown direction aligned with the occlusal-plane normal) to obtain a 2D representation, and apply a plug-and-play 2D detection model to predict a per-pixel toothness probability map. In our implementation, we use a YOLO-family model, but this component is not tied to any specific architecture and can be replaced by any 2D network that outputs toothness probabilities/confidences. The near-occlusal rendering is used to obtain dense 2D toothness predictions, which are mapped back to the mesh to form a per-vertex visual probability field P and a corresponding visibility mask m, serving as inputs to subsequent operator modulation and dual-stream diffusion. Each 2D tooth proposal is then backprojected onto the 3D mesh to define a local surface patch corresponding to a candidate tooth region. Formally, for a 2D proposal box bt, we define the corresponding patch as Pt = {vi ∈ V | π(vi) ∈ expand(bt)}, where π(·) denotes the projection and expand(·) slightly enlarges the box to include boundary regions.In this way, the 2D boxes induce a set of patch-level regions on the mesh, which serve as the basic units for subsequent visual guidance and diffusion modulation. At this stage, the 2Dmodel also predicts an FDI label L for each proposal, which is used to specify the tooth position of the corresponding 3D patch.

Toothness Prediction andVisibility. During rendering, we also compute per-vertex visibility (e.g., via a z-buffer visibility mask) to indicate whether each vertex is observable from the chosen view. Let P2D denote the predicted 2D toothness probability map and π(vi) the projected pixel coordinate of vertex vi. We obtain per-vertex toothness probabilities by bilinear sampling: Pi = bilinear(P2D, π(vi)) . We additionally obtain a per-vertex visibility mask mi ∈ {0, 1}, where m = 1 for visible vertices and m = 0 for invisible ones. The resulting P and m are used as inputs to diffusion modulation: P provides visual confidence variations for constructing diffusion barriers, while m suppresses unreliable modulation in occluded or non-visible regions.

3.4 DiffusionModulation

This section describes how we construct a pair of spectral operators on the same mesh topology by reweighting surface conductance using single-view visual cues and visibility. The key motivation is that real IOS meshes may contain spurious short-circuit connections (e.g., tooth adhesion), under which spectral diffusion methods-including DiffusionNet-tend to propagate features across incorrect topology, leading to merge/split failures. Instead of explicitly modifying the mesh topology, we intervene before diffusion by modulating the conductance of the Laplacian operator. The goal is to construct a visually modulated Laplacian Lmod such that diffusion within a tooth is largely preserved, diffusion across tooth boundaries is strongly suppressed, and the behavior remains consistent under mesh refinement.

Spectral Method. In surface and mesh learning, spectral method refers to approaches based on the Laplace-Beltrami operator (and its discretization) to characterize intrinsic geometry. The eigenfunctions and eigenvalues of the Laplacian (the spectrum) provide a multi-scale basis for analyzing functions on the surface. Intuitively, the Laplacian governs smoothing and diffusion: low-frequency components capture global structure, while higher frequencies encode local details and boundaries. Spectral diffusion and filtering enable information propagation that depends weakly on the specific mesh discretization, which underlies their relative stability across different resolutions.

Spectral Preliminaries. We use the discrete LaplaceBeltrami operator L together with a mass matrix M to model intrinsic surface diffusion, and compute the generalized eigendecomposition Lϕk = λkMϕk to obtain the first R eigenpairs (λk, ϕk). These eigenpairs are used to construct the spectral operators underlying DiffusionNet and to compute spectral input features such as the heat kernel signature (HKS), which characterizes how heat diffuses locally around each vertex over multiple scales, providing an intrinsic and robust description of local surface geometry. The resulting HKS is included as part of the per-vertex input feature vector to the network. In what follows, we further construct a visually modulated operator Lmod and its corresponding spectrum (λmod k , ϕmod k ) to form a dual-operator setting.

Original Operator. We adopt the standard cotangent discretization of the Laplace-Beltrami operator. For each edge (i, j) ∈ E, the undirected cotangent weight is

where αij and βij are the two angles opposite to edge (i, j) (for boundary edges, only one angle is available). This yields a symmetric stiffness matrix L with off-diagonal entries Lij = -worig ij , together with a lumped (or barycentric) mass matrix M for the generalized eigendecomposition.

Diffusion Modulation. Let eij = vi - vj be the edge vector and using Pi as defined above. We introduce a small constant δ > 0 for numerical stability. Our design principle is to modify the conductance of the diffusion medium prior to constructing the Laplacian. For each edge, we define a refinement-consistent barrier energy qij = (Pi-Pj)2 ∥eij∥+δ . This form can be viewed as a discrete counterpart of the continuous energy ∫ (P ′ (s))2 ds, and behaves consistently under mesh refinement: when an edge is subdivided, the accumulated barrier over the sub-edges remains approximately preserved. We then convert this barrier energy into a conductance scaling factor using an exponential model:

where mi and mj means If there is an endpoint that is not visible, the scaling factor will always be 1, and Gc = √PiPj acts as a confidence gate that globally downweights lowconfidence regions for robustness, and exp(-αqij) enforces exponential suppression of diffusion across visual boundaries (large |Pi - Pj|). The small constant ε prevents numerical degeneracy and is not intended to repair connectivity. Note that this exponential term is fundamentally different from the heat-kernel diffusion exp(-tL): here it defines the diffusion medium before propagation, whereas the latter performs diffusion within a given medium.

The modulated edge weights are then obtained by relative reweighting:wmod ij = worig ij · sij, and the visually modulated Laplacian is reconstructed as Lmod = Dmod -Wmod.

This construction preserves symmetry and non-negativity (wmod ij = wmod ji ≥ 0), ensuring that Lmod remains a valid Laplacian-type operator. After get Lmod,only need to fix M, we can get (λmod k , ϕmod k ) fromLmodϕmod k = λmod k Mϕmod k .

3.5 Dual-streamDiffusionNet

We adopt DiffusionNet [Sharp et al., 2022] as the 3D backbone for feature propagation (here, "diffusion" refers to PDEdriven geometric diffusion on surfaces, not generative diffusion models).Using DiffusionNet as a base module, we build a dual-stream architecture comprising an original spectral decomposition and a visually modulated spectral decomposition. For each sample, the dataloader provides two sets of spectral decompositions: (λk, ϕk) and (λmod k , ϕmod k ); both are fed into the network during training to enable parallel propagation and interaction.

Dual-stream Backbone. Given per-vertex input features xin, we first map them to a common channel width C using a shared linear layer:

We duplicate the result to initialize the two streams. For the t-th block (t = 0, . . . ,Nblock - 1), the two streams are updated by their respective DiffusionNet blocks parameterized by different eigenpairs: (xt+1 orig = Blocktorig (xtorig ; L), xt+1 mod = Blocktmod (xtmod ; Lmod)). The two streams use independent DiffusionNet block parameters while operating on the original and visually modulated Laplacian operators, respectively.

DiffusionNet Block. ADiffusionNetBlock performs intrinsic feature mixing on the vertex set using diffusion/filtering operators parameterized by the Laplace-Beltrami operator L. It is a geometric diffusion layer (not a generative diffusion model): the propagation is driven by surface operators, remains stable across mesh resolutions and sampling densities, and naturally supports variable-length vertex inputs.

Block-wise Cross-stream Exchange. To exploit the complementarity between the two propagation structures, we introduce a cross-stream exchange module after each block. Specifically, we form a difference-enhanced feature f i+1 = concat(xi+1 orig , xi+1 mod , xi+1 orig - xi+1 mod), and use two lightweight mappings Exchtto-orig and Exchtto-mod to generate exchange signals injected into the two streams. We further egNet introduce learnable per-channel gating vectors αtorig , αtmod ∈ RC to control the exchange strength:

Figure 2: Qualitative comparison of dental segmentation results from six models on five challenging cases, including crowded dentition, missing teeth, and the presence of wisdom teeth, as well as scans with noise. Different colors indicate tooth instance labels. Our method preserves cleaner boundaries and remains robust to noise, whereas other methods often introduce boundary artifacts.

where σ(·) denotes the sigmoid function and ⊙ is channelwise multiplication. This design allows the network to learn when and which channels to exchange information, mitigating potential error amplification when the modulated stream is unreliable.

Robust Late Fusion and Output. At the final stage of the network, we adopt a robust late fusion strategy where the original stream serves as the baseline, and the modulated stream provides residual corrections. We denote the final outputs of the two streams as xorig = xNblock orig and xmod = xNblock mod . Let ffuse = concat(xorig, xmod, |xorig -xmod|) The fused feature is computed as x = xorig + Linfuse(ffuse) and through a linear output head and a sigmoid function, each node's probability is output.we output a patch-specific foreground probability pt(vi) (tooth vs. non-tooth) for each patch

On each patch, we extract the largest connected foreground as our tooth segmentation result. For the patch-to-mesh assembly, we employ a tooth-centric instance resolution strategy. Specifically, we group patches by their assigned FDI labels and calculate a confidence score based on the sum of predicted foreground probabilities. For each tooth label, the system automatically selects the patch with the highest score as the optimal candidate, effectively resolving overlap conflicts without complex global optimization.

4 Experiments

4.1 Experiment Setup

We conducted experiments on the Teeth3DS dataset [BenHamadou et al., 2023], which contains 900 upper tooth scan samples and 900 lower tooth scan samples. This dataset provides per-vertex annotations of tooth instances, with the tooth instances labeled using the FDI numbering system to distinguish individual permanent teeth within different quadrants. This numbering system is widely used in clinical dentistry to uniquely identify permanent teeth at different positions. The dataset also includes normal samples as well as samples with complex conditions such as wisdom teeth, tooth loss, or tooth fusion, to assess the model's performance in tooth instance segmentation in scenarios close to real clinical settings.

4.2 Implementation Details

All experiments are implemented in PyTorch. We adopt DiffusionNet as the base network architecture to perform feature propagation and prediction on the mesh surface. The network input includes the 3D coordinates of vertices, Gaussian curvature, Heat Kernel Signature (HKS), and the tooth probability map from the 2D detection model. We use yolo11m-seg as the detector, which is pre-trained before 3D training.

In the dual-stream design, the two branches share the same architecture and training configuration, but perform diffusion propagation with two different operators, L and Lmod, respectively, followed by feature exchange and fusion. The model is trained with Adam using an initial learning rate of 5 × 10-4 for 200 epochs. During the inference stage, we threshold per-vertex probabilities and extract the largest connected component to form tooth instances. To ensure fairness in comparison, all comparison methods use the same post-processing procedure. The parameter of our model is 3.67M. Regarding computational cost, while the pipeline includes 2D detection and patch-wise spectral decomposition, the overhead is well-managed via parallelization. Given that dental CAD workflows are mostly offline, this minor latency is a highly worthwhile trade-off for the significant reduction in structural errors.

4.3 EvaluationMetrics

For more clearly define the instance-level errors of teeth, we define the instance level GT set G = {Gi}Ki=1 and the predicted instance set P = {pj} ˆKj=1 , where Gi and pj represent the set of vertices corresponding to the i-th real tooth instance and the j-th predicted instance (ignoring the gum vertices), respectively. We adopt the instance-level intersection and union ratio IoU(Gi, pj) = |Gi∩pj| |Gi∪pj| , And for each real instance, we take the IoU value of its best matching predicted instance best iou(i) = maxj IoU(Gi, pj). Given the threshold τ , the TSR is defined as

where [·] represents the indicator function, which takes the value 1 if the condition in parentheses holds and 0 otherwise. Secondly, in order to characterize structural errors such as merging and fragmentation, we use the asymmetric coverage ratio:Cov(A, B) = |A∩B| |A| . We can also define the Merge rate to count the cases where multiple true teeth were incorrectly merged into the same predicted instance. We first define the main match as ϕ(i) = argmaxj Cov(Gi, pj), then the overall merge rate is defined as follows.

Similarly, we define the Split rate to count the number of instances where a single real tooth is over-segmented into multiple predicted instances:

In order to facilitate the comparison with the existing work, we use the commonly used evaluation metrics in the literature alignment experiment setting, and report the results according to their data division and evaluation process.

4.4 Main Results

As shown in Table 1, our full pipeline ("ours") achieves competitive performance across all metrics, reaching a TmIOU of 91.20% and a Dice score of 94.25%, which is on par with or superior to most existing methods. Notably, our approach demonstrates stable and balanced performance across different tooth indices, indicating strong robustness of the proposed 3D diffusion-based modulation and dual-stream propagation under realistic conditions. To further analyze the upper limit of our method, we report an "our*" optimized version, in which the errors introduced in the 2D inspection stage (i.e., incorrect label predictions and missed bounding boxes) have been excluded. This setting is used to simulate the maximum potential capability of our 3D framework under the condition of having perfect 2D prior conditions. Under this idealized condition, our method achieves significant improvements, with the mIOU value increasing from 91.20% to 94.98%, and the Dice value increasing from 94.25% to 97.27%. These improvements are particularly significant in the posterior teeth area, as severe occlusal problems and adhesions between teeth are more common here, and in fact, the main reason for the current limitation of posterior teeth performance lies in the 2D visual module. These results indicate that the performance bottleneck of the entire process mainly stems from imperfect 2D predictions, while the proposed 3D diffusion modulation framework itself has a relatively high upper limit of performance.

4.5 Split &Merge Study

While many existing methods report high segmentation accuracy, they still suffer from severe structural errors under challenging intraoral scan conditions, especially tooth adhesion and occlusal noise. Such errors mainly manifest as merging adjacent teeth or over-segmenting a single tooth, which are not adequately reflected by conventional vertex-level metrics. Therefore, we conduct a dedicated split-and-merge study to evaluate the structural correctness of tooth instance segmentation. Table 2 reports the instance-level performance of different methods on the Teeth3DS dataset using TSR, Merge, and Split metrics under multiple thresholds. Generic pointbased methods such as PointNet++ and DGCNN exhibit low TSR values and high merge/split rates, indicating their limited ability to handle distorted local neighborhoods caused by interdental adhesion. Methods specifically designed for dental segmentation (e.g., TSGCNet and DilatedToothSegNet) achieve higher TSR, demonstrating improved tooth separation. However, they still suffer from non-negligible merge and split errors, especially under stricter coverage thresholds, suggesting that incorrect feature propagation across adhesive regions remains an issue. In contrast, our dual-streamVGDM achieves the highest TSR scores (99.70%@0.5 and 99.18% @0.75) while maintaining near-zero merge and split rates across all thresholds. This demonstrates that VGDM is not only capable of correctly separating adjacent teeth but also highly effective in suppressing both false merging and oversegmentation.

Qualitative results in Figure 2 further support this observation. While baseline methods frequently produce merged crowns or fragmented tooth instances in adhesive regions, our method consistently yields clean and anatomically plausible tooth boundaries, closely matching the ground truth. This confirms that the proposed operator-level modulation successfully mitigates diffusion leakage across spurious geometric connections. Moreover, using Laplace-Beltrami diffusion as an intrinsic low-pass regularizer promotes smooth and coherent boundaries, making the predictions less sensitive to local geometric noise.

4.6 Ablation Study

The instance segmentation results of different model variants on the Teeth3DS dataset are shown in Table 3. Firstly, by comparing the full model with the L model, although the mIoU differences among variants are small, it can be found that the dual-stream model is significantly better than the single-stream model in terms of Merge and Split indicators, indicating that the parallel diffusion based on different operators is helpful to alleviate the structural confusion between adjacent teeth. Secondly, by comparing the full model with Lmod model, it can be found that the model cannot give good segmentation results due to the lack of real geometry input.

Furthermore, Table 4 reports performance of the adopted YOLOv11 model under two output forms. The results show that even under relatively general 2D segmentation results, the proposed VGDM can still work stably and show better segmentation results.

5 Conclusion

We introduce VGDM, a novel 3D tooth segmentation framework. Our method has achieved competitive results in the Teeth3DS benchmark test, especially in reducing instancelevel merging and segmentation errors, which are crucial for reliable clinical applications. The proposed framework demonstrates strong robustness under complex anatomical conditions, indicating its suitability for real-world digital dental workflow. More broadly speaking, VGDM highlights the potential of integrating external visual priors and internal geometric operators at the diffusion level. We believe that this paradigm can go beyond tooth segmentation and be extended to other surface-based three-dimensional perception tasks, where false topologies and ambiguous geometries pose fundamental challenges.

Acknowledgements

We are grateful for the valuable comments provided by the anonymous reviewers. This research was partially supported by the National Natural Science Foundation of China (Grant No. 61972221).