IWCNet: Intervention-based Weight Calibration Network for Multi-center Multi-modality MRI Lesion Segmentation

1 Introduction

Multi-modality MRI lesion segmentation leverages complementary contrasts across modalities to jointly characterize lesion morphology, microstructural alterations, and hemodynamic properties, thereby improving lesion delineation, reducing misdiagnosis, and supporting staging and treatmentresponse assessment [Gurney-Champion et al., 2020]. In liver tumor MRI, T1- and T2-weighted images better delineate lesion boundaries and suggest compositional differences [Albiin, 2012], whereas diffusion-weighted imaging

(DWI) is more sensitive to diffusion restriction and related microstructural changes [Parikh et al., 2008], helping further differentiate lesion type and activity. Therefore, accurate multi-modality MRI lesion segmentation can reliably yield key quantitative measures such as lesion volume, the spatial extent of lesion margins, and the proportion of diffusionrestricted subregions, providing direct evidence for treatment planning and response evaluation [Bharwani and Koh, 2013].

Meanwhile, correlation-driven fusion methods have been widely adopted for multi-modality MRI lesion segmentation [Xu et al., 2024; Chen et al., 2025; Cao et al., 2025]. These methods measure pairwise similarity between modality features or attention affinity scores to evaluate cross-modality correlation at each spatial location [Zhou et al., 2022]. They then transform the resulting correlation scores into adaptive fusion weights via normalization or gating, where the weights determine each modality's contribution to the fused representation [Ding et al., 2021]. By learning an adaptive mapping from cross-modality correlations to fusion weights, these methods effectively exploit complementary contrasts to enhance lesion delineation and achieve superior performance under within-domain settings.

However, correlation-driven fusion methods [Zhou et al., 2022; Xu et al., 2024] are prone to fusion weight bias under domain shift in multi-center settings. Joint feature statistics vary across centers, altering the cross-modal correlations these methods rely on and changing their relative strength. Consequently, the correlation-to-weight mapping becomes miscalibrated at unseen centers, leading to systematically over- or under-weighting specific modalities and amplifying center-specific shortcuts rather than lesion-relevant evidence.

In this paper, we propose an Intervention-based Weight Calibration Network (IWCNet) for accurate multi-modality MRI lesion segmentation. Compared with correlation-driven fusion methods, IWCNet models domain shift as a controllable embedding and leverages explicit interventions to mitigate its impact on fusion weight estimation, thereby improving robustness under domain shift. Specifically, IWCNet constructs original and intervened branches by performing interventions on the domain embedding. It measures inter-branch prediction effects to estimate each modality's sensitivity to domain variation and uses these sensitivities to calibrate fusion weights. Additionally, to explicitly manipulate domain variations while preserving anatomical structure, IWCNet introduces Operator-level Domain Modulation (OLDM), which treats domain variation as a controllable variable via operatorlevel domain intervention, thereby constructing structureconsistent domain-variant feature pairs for effect estimation. Furthermore, to prevent fusion weights from over-relying on domain-sensitive shortcuts, IWCNet designs Effect-Driven Consistency Calibration (EDCC), which jointly leverages intervention effects and cross-branch prediction consistency to calibrate weights, thereby anchoring the weights to domaininvariant evidence.

The paper introduces the following contributions:

• A novel multi-center multi-modality MRI lesion segmentation method is proposed to calibrate modality fusion weights using prediction effects under domainembedding interventions, thereby improving robustness to domain shift.
• An Operator-level Domain Modulation module is proposed to construct structure-consistent, domain-variant feature pairs via operator-level intervention, thereby explicitly modulating domain variations while preserving anatomical structure.
• An Effect-Driven Consistency Calibration module is proposed to jointly leverage intervention effects and cross-branch prediction consistency to calibrate fusion weights, thereby preventing the weights from overrelying on domain-sensitive shortcuts.
• Extensive experiments on multi-center multi-modality MRI datasets demonstrate that IWCNet achieves consistent improvements in both accuracy and robustness under domain shift.

2 RelatedWork

2.1 Multi-modalityMRI Segmentation

Multi-modality MRI segmentation exploits complementary tissue contrasts across modalities to improve lesion delineation. Early approaches [Kamnitsas et al., 2017] stacked modalities as channels and used a single encoder, which can blur modality-specific cues. Later works adopted multibranch encoders [Havaei et al., 2016; Dolz et al., 2018] to preserve modality-specific representations and fuse them in a shared latent space. Multi-level fusion [Kamnitsas et al., 2017] further injects interactions across resolutions to leverage both boundary detail and high-level semantics. Recently, correlation-driven fusion paradigms [Wang et al., 2021; Zhang et al., 2022] have become predominant, using crossmodal similarity/affinity as a proxy for inter-modality agreement and converting it into channel-/spatial-wise reweighting or cross-attention for adaptive fusion, while remaining computationally tractable [Zhao et al., 2022]. Despite strong within-domain performance, these strategies implicitly anchor the correlation-to-weight mapping to training-domain co-occurrence statistics, making them sensitive to domaininduced appearance variations and leaving robust fusion under domain shifts in real-world multi-center settings insufficiently addressed [Yan et al., 2019; Guan and Liu, 2021].

2.2 Controlled Interventions inMedical Image Analysis

Controlled interventions have been explored in medical imaging to probe model behavior and reduce shortcut reliance by creating paired examples that preserve clinically meaningful structure while inducing controlled appearance changes. Black-box explainers often generate progressively edited images, typically with generative models, to identify decisioncritical evidence and audit predictions [Mertes et al., 2022; Singla et al., 2023; Moreira et al., 2025]. Diffusion-based representations further enable smooth manipulations in semantically structured latent spaces, improving the plausibility and continuity of intervention trajectories for prediction analysis [Atad et al., 2024]. Related ideas have also been applied to segmentation and adaptation, where disentangling causal content from domain-dependent factors improves robustness under domain shift [Lv et al., 2025]. Different from these works that focus on explanation or single-input robustness, our goal is to use intervention effects to calibrate modality contributions for stable cross-modality fusion.

2.3 Domain Generalization inMedical Image Segmentation

Robust medical segmentation under domain shift is often studied through domain generalization and style-aware representation learning [Zhou et al., 2021], with the goal of reducing reliance on domain-dependent appearance factors. One representative direction improves generalization by augmenting style or feature statistics, such as frequency-space mixing [Li et al., 2023] and generative feature-style augmentation with regularization for more stable training [Huang et al., 2025]. Related work also models the distribution of style statistics to support randomized sampling or mixing during training [Safdari et al., 2025]. Diffusion-based formulations have been explored to improve segmentation robustness under domain shifts [Yang et al., 2025]. While effective for single-stream robustness, these methods do not target fusionweight calibration in multi-modality segmentation, nor do they use perturbation effects to quantify modality contributions for stable evidence aggregation under domain shift.

3 Methodology

Our IWCNet calibrates fusion weights for multi-modality MRI lesion segmentation using prediction effects under domain-embedding interventions. It keeps content features fixed, intervenes on the domain embedding to form an original and an intervened branch, and uses finite-difference logit changes to derive effect-based signals for sample-wise weighting and prediction regularization, thereby mitigating the impact of domain shift on cross-modality fusion. In addition, IWCNet introduces Operator-level Domain Modulation (OLDM), which injects the domain embedding into content features through channel-wise modulation and a lowfrequency spatial field in a residual form, enabling finegrained appearance modulation. Furthermore, IWCNet introduces Effect-Driven Consistency Calibration (EDCC), which maps effect scores to sample-wise fusion weights via Monotonic Effect Gating and enforces cross-branch prediction consistency to stabilize calibrated weighting.

3.1 IWCNet Network Architecture

IWCNet constructs an original branch and an intervened branch by keeping content features fixed and applying OLDM with an original domain embedding and its intervened counterpart. It measures prediction effects as finite-difference changes in segmentation logits between the two branches to compute per-modality effect scores, and maps these scores to sample-wise fusion weights to stabilize cross-modality aggregation under domain shift.

For each sample, the input is a set of aligned modalities X = {X(m) }Mm=1 with a corresponding lesion mask Y , and the goal is to predict ˆ Y by aggregating complementary evidence across modalities while maintaining stable fusion behavior under domain shift.

Given each modality X(m) , a shared encoder E(·) produces a feature pyramid {F (m) l }Ll=1 , where l indexes resolution levels. IWCNet factorizes the representation into a content feature map and a compact domain embedding via Spatial and Domain Dual Projection (SADP). The content feature preserves spatial structure relevant to anatomy and lesions

while the domain embedding encodes global appearance statistics that characterize domain factors

Here AttnPool(·) aggregates global information into a compact token, and hc(·) and hd(·) are lightweight projection heads.

To generate the intervened embedding, IWCNet applies a controlled perturbation to the original domain embedding during training, while keeping C (m) l fixed. Specifically, for each modality m, we construct

where ∆(m) is a unit-norm perturbation direction in the embedding space (e.g., randomly sampled or derived from minibatch statistics), and κ controls the intervention strength.

On top of this factorization, OLDM turns the domain embedding into intervenable operators and injects them into C (m) l in a residual manner. In the original branch, OLDM uses E(m) o to obtain domain-conditioned features ˜ F (m) l . In the intervened branch, OLDM replaces E(m) o with the intervened embedding E(m) i while keeping C (m) l fixed, yielding intervened features ˜ F (m) l,i .

Based on the paired features from the two branches, IWCNet performs weight calibration via EDCC. Concretely, a shared prediction head H(·) produces per-modality logits for the original and intervened branches, denoted as {L(m) o }Mm=1 and {L(m) i }Mm=1 , respectively. EDCC then quantifies an intervention effect score L(m) r for each modality by measuring the finite-difference change between L(m) o and L(m) i , which reflects the sensitivity of modality-specific predictions to domain-embedding interventions.

These effect scores are subsequently converted into sample-wise fusion weights {w(m) }Mm=1 through a Monotonic Effect Gating, such that modalities with smaller intervention effects receive larger weights. Using the calibrated weights, IWCNet fuses modality-specific features to obtain the fused prediction ˆ Y for the original branch, and analogously forms the fused prediction for the intervened branch. Finally, EDCC enforces prediction consistency between the fused outputs of the original and intervened branches.

During training, we optimize IWCNet with a supervised segmentation loss on the fused prediction of the original branch, together with the EDCC consistency regularizer. The overall objective is

where Rcons is defined in Eq. 14.

3.2 Operator-level DomainModulation

OLDM instantiates a structure-preserving domain modulation mechanism that turns the domain embedding into intervenable operators acting on content features. Given the factorized representation, OLDM parameterizes domain factors as a composition of a channel-wise appearance operator and a smooth spatial bias field, and injects them into the content stream in a residual form. This operator-based reparameterization makes domain-dependent appearance variation explicit in feature space and enables controlled embedding interventions to induce appearance changes while keeping anatomy and lesion structure invariant.

OLDM operates on the content feature maps {C (m) l }Ll=1 and the domain embeddings {E(m) }Mm=1 produced by SADP. For each modality m and level l, OLDM maps E(m) to channel-wise affine modulation parameters

where gγ,l(·) and gβ,l(·) are lightweight MLP heads producing vectors in RCl with Cl channels.

In addition, OLDM predicts a low-frequency spatial field from E(m) to capture smooth domain-dependent bias patterns. Concretely, it outputs a coarse grid and upsamples it to the feature resolution

where gs,l(·) outputs a low-resolution field and Up(·) denotes bilinear upsampling to match the spatial size of C (m) l .

OLDM injects these signals into the content feature map in a residual manner to obtain the domain-conditioned feature

where ⊙ denotes channel-wise multiplication with broadcasting. This formulation separates channel-wise appearance modulation from spatially smooth bias modulation, enabling fine-grained appearance control without changing the spatial layout encoded in C (m) l .

OLDM is applied to both the original and intervened branches. For the intervened branch, OLDM replaces the original embedding E(m) o with the intervened embedding E(m) i while keeping C (m) l fixed, yielding intervened features ˜ F (m) l,i . This design allows controlled embedding interventions to generate appearance-varying yet structure-consistent feature pairs, which EDCC uses to obtain paired branch logits for effect estimation and weight calibration.

3.3 Effect-Driven Consistency Calibration

EDCC calibrates fusion weights by converting prediction effects under controlled domain-embedding interventions into sample-wise weighting signals. It forms paired predictions under an original and an intervened embedding while keeping content features fixed, measures per-modality effects via finite-difference logit changes, and maps these effects to fusion weights through Monotonic Effect Gating. In parallel, EDCC enforces cross-branch prediction consistency to stabilize calibrated weighting and suppress domain-dependent shortcuts under domain shift.

EDCC takes as input the domain-conditioned features from OLDM for both branches, namely { ˜ F (m) s,l } and { ˜ F (m) s,l,i }, where s indexes samples in a mini-batch, m indexes modalities, and l indexes resolution levels.

EDCC estimates a per-modality effect score for each sample via the finite-difference change of segmentation logits under embedding interventions. Using a shared lightweight prediction head H(·), EDCC obtains per-modality logits for the original and intervened branches

where L denotes the coarsest level used for effect estimation. The intervention effect is quantified as the mean absolute logit change

where Ω denotes spatial locations and K is the number of classes. Intuitively, L(m) r,s measures how sensitive modalityspecific predictions are to domain-embedding interventions.

EDCC maps {L(m) r,s }Mm=1 to sample-wise fusion weights via Monotonic Effect Gating. Specifically, a monotonic decreasing mapping ψ(·) converts effect scores into unnormalized evidence

where we instantiate ψ(·) as a temperature-controlled monotonic mapping

and τ controls the sharpness of effect-to-evidence conversion. The fusion weights are then obtained by normalization

This mapping preserves interpretability and stability by ensuring that modalities with smaller intervention effects receive larger weights through a strictly monotonic rule.

Using the calibrated weights, EDCC fuses the original features at each scale by weighted summation

and feeds {F fuse s,l }Ll=1 to a shared decoder D(·) to obtain the final probabilities ps = σ(Lo,s). In the same manner, EDCC fuses the intervened features { ˜ F (m) s,l,i } to obtain intervened probabilities ps,i = σ(Li,s).

Finally, EDCC enforces prediction consistency between the original and intervened branches to stabilize both weight calibration and final predictions under domain shift. We apply a symmetric Bernoulli KL consistency regularizer between the foreground probability maps ps = σ(Lo,s) and ps,i = σ(Li,s):

which suppresses domain-dependent shortcuts that vary under domain-embedding interventions and encourages pathology-consistent aggregation across modalities.

4 Experiment

4.1 Datasets and EvaluationMetrics

The proposed IWCNet is evaluated on three multi-center, multi-modality MRI datasets for liver, brain, and prostate lesion segmentation. The Liver MRI dataset contains 450 patients from three clinical centers (250/120/80), including 280 hemangioma and 170 hepatocellular carcinoma cases, with three contrast-free modalities (T1/T2/DWI). The Brain MRI dataset contains 551 glioma patients from three centers (230/176/145), with three contrast-free modalities (T1/T2/FLAIR). The Prostate MRI dataset contains 370 prostate cancer patients from three centers (180/115/75), with three contrast-free modalities (T2W/DWI/ADC). All modalities and annotations are spatially aligned and resampled to a unified resolution for multi-modality fusion.

To reflect a realistic deployment setting, we train and select models using only the largest center as the source domain (C1), and evaluate generalization on the remaining two centers (C2 and C3) as external test domains. We utilized the Dice Similarity Coefficient (Dice) and 95th percentile Hausdorff distance (HD95) [Huttenlocher et al., 2002] for evaluating segmentation performance.

4.2 Implementation

All experiments are implemented in PyTorch. Hyperparameters are tuned on the validation set to achieve optimal performance. We use the Adam optimizer [Zhang, 2018] with a batch size of 1, and the initial learning rate is set to 1×10-4 , selected from {10-3 , 10-4 , 10-5 } with an exponential decay factor of 0.95. Models are trained for 300 epochs. Training takes approximately 16 hours, and inference requires 0.38 seconds on average per test sample, where each sample consists of aligned multi-modality inputs of spatial size 256 × 256, using two NVIDIA RTX 3090 Ti GPUs.

4.3 Comparisons with the State-of-the-art

To comprehensively evaluate the effectiveness of IWCNet, we compare it with eight representative multi-modality MRI segmentation baselines, including nnU-Net, TransUNet, Swin-UNet, A2FSeg, DM-FF, M2FNet, UMF-SegNet, and LCMF-Net. These methods represent a broad spectrum of widely used fusion paradigms, ranging from early fusion and multi-stream multi-level fusion to more recent adaptive fusion and Transformer-based cross-modality interaction mechanisms. To ensure a rigorous and fair comparison, all methods are trained and evaluated under an identical single-center training protocol, preprocessing pipeline, and optimization schedule.

As reported in Tab.1, IWCNet consistently achieves the best external-center performance across all three datasets and both external centers, yielding superior Dice and lower HD95. On the liver dataset, IWCNet improves externalcenter Dice by 1.81-4.04 points over the second-best method while reducing HD95 by 1.29-2.83. On the brain dataset, it boosts external Dice by 2.09-2.71 and reduces HD95 by 1.11-1.32. On the prostate dataset, it delivers larger gains, improving external Dice by 4.43-5.01 and reducing HD95 by 1.51-1.63. These results indicate that IWCNet provides more accurate lesion delineation and better boundary quality under cross-center domain shift. Fig.3 further provides qualitative comparisons, where IWCNet produces cleaner lesion boundaries and fewer false positives in challenging cases.

To further assess robustness under domain shift, Tab.2 reports the leave-one-center-out (LOCO) evaluation on the liver dataset, where In-center Dice is measured on the seen training centers, Cross-center Dice on the held-out unseen center, and Drop denotes the Dice decrease. Notably, IWCNet yields the smallest performance drop (2.31), substantially outperforming the competing fusion baselines. This narrowed generalization gap confirms that our approach learns more transferable cross-modality fusion behavior, effectively mitigating the performance degradation caused by domain shift.

4.4 Ablation Study

Ablation Study of Proposed Components. We conduct progressive module-level ablations on three datasets to quantify how each design in IWCNet contributes to robust crosscenter fusion. Tab. 3 reports Dice/HD95 averaged over the two external centers. All variants share the same backbone and training settings. A0 is the concatenation baseline. A1 replaces concatenation with a standard attention fusion block. A2 activates INT, which constructs an intervened branch by applying embedding intervention while keeping content features fixed, enabling the model to observe prediction effects. A3 further adds the cross-branch consistency regularizer (Cons), enforcing agreement between fused predictions from the original and intervened branches. A4 enables OLDM together with EDCC to perform effect-driven weight calibration, while disabling Cons to decouple effect-based weighting from regularization. A5 is the full IWCNet.

Overall, A1 provides only marginal improvements over A0 across all datasets, indicating that correlation-to-weight mappings learned on the source center transfer poorly under domain shift. Introducing INT alone (A2) yields clear gains on both Dice and HD95, improving external Dice by 4.13/1.90/2.92 and reducing HD95 by 3.29/1.57/1.51, respectively, suggesting that embedding intervention exposes the model to plausible domain variations and mitigates domaindependent shortcuts. Adding Cons on top of INT (A3) further stabilizes predictions, leading to additional HD95 reductions and consistent Dice improvements, demonstrating the benefit of enforcing invariance between the original and intervened fused outputs. When OLDM and EDCC are activated (A4), performance improves substantially, especially on boundary quality, with HD95 dropping to 8.95/7.58/8.41 and Dice increasing to 76.74/75.49/60.81, confirming that effect-driven calibration yields more reliable sample-wise fusion under domain shift. Finally, the full model (A5) achieves the best overall performance on all datasets, indicating that effect-driven calibration and cross-branch consistency act complementarily to produce transferable cross-modality aggregation.

Sensitivity to the Intervention Strength κ. We investigate how the intervention strength affects robustness under domain shift. Specifically, we construct the intervened embedding by applying a controlled intervention to the original domain embedding with different magnitudes κ, and report cross-center performance on the liver dataset. As shown in Fig. 4(a), increasing κ from 0 to 0.05 consistently improves Cross-center Dice (from 75.17% to 79.36%) and reduces the cross-center drop, indicating that moderate embedding interventions expose the model to realistic center-induced appearance variations and yield more reliable effect signals. When κ becomes large, performance decreases, suggesting that overly strong interventions deviate from plausible domain variations and destabilize the intervention effect.

Sensitivity to the Effect-Gating Temperature τ. We further study the temperature τ in Monotonic Effect Gating within EDCC, which controls the sharpness of mapping effect scores to fusion weights (Eq. 11). As shown in Fig. 4(b), very small τ leads to overly peaky, near-hard gating, causing the fusion to be dominated by a single modality and making the weights sensitive to estimation noise. Consequently, cross-center performance degrades. In contrast, large τ yields overly smooth (nearly uniform) weights, which diminishes the benefit of effect-driven calibration and approaches unweighted fusion. The best trade-off is achieved at τ = 1, where IWCNet attains the highest Cross-center Dice (79.36%) with the lowest Drop, demonstrating that an appropriately calibrated gating sharpness is crucial for translating effect evidence into robust and stable sample-wise fusion.

5 Conclusion

In this paper, we propose IWCNet for multi-center multimodality MRI lesion segmentation. IWCNet models domain shift as a controllable embedding and leverages explicit interventions to mitigate its impact on fusion weight estimation, enabling reliable cross-modality aggregation under domain shift. We evaluate IWCNet on three multi-center multimodality datasets and compare it with eight state-of-the-art methods. The results consistently show that IWCNet achieves the best overall performance, demonstrating robust and accurate lesion segmentation under domain shift.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (62476001, 62406111), Anhui Province Excellent Scientific Research and Innovation Team (2024AH010004), the Anhui Province Clinical Medicine Research Transformation Project (202304295107020031), the Research Fund of Anhui Institute of Translational Medicine (2022zhyx-C72), and the Research Fund of Health Commission of Anhui Province (AHWJ2023A20115).

Contribution Statement

Ronghui Qi and Wenlong Song contributed equally to this work. Wanyin Shi and Chenchu Xu served as the corresponding authors.