---
title: "ReDi-FM: Frozen Foundation Model for Continual Test-Time Adaptation in Medical Image Segmentation"
authors:
  - "Jianhang Ji"
  - "Zhiming Cheng"
  - "Jianxiang Zhao"
  - "Tingyu Wang"
  - "Bingtao Ma"
  - "Yuhan Gao"
  - "Zuobin Ying"
  - "Shuai Wang"
doi: "10.24963/ijcai.2026/140"
url: "https://ijcai.org/proceedings/2026/140"
pdf: "https://ijcai.org/proceedings/2026/0140.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "1252-1260"
track: "Main Track"
keywords:
  - "Computer Vision: Transfer, low-shot, semi- and un- supervised learning"
  - "Computer Vision: Biomedical image analysis"
  - "Computer Vision: Segmentation, grouping and shape analysis"
---

# ReDi-FM: Frozen Foundation Model for Continual Test-Time Adaptation in Medical Image Segmentation

Jianhang Ji, Zhiming Cheng, Jianxiang Zhao, Tingyu Wang, Bingtao Ma, Yuhan Gao, Zuobin Ying, Shuai Wang

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 1252-1260. DOI: [10.24963/ijcai.2026/140](https://doi.org/10.24963/ijcai.2026/140)

## Abstract

Continual test-time adaptation (CTTA) adapts a pre-trained medical segmentation model online to an unlabeled target stream whose distribution changes over time. However, most existing CTTA methods rely on pseudo-labeling and self-supervised objectives, which inevitably yield noisy supervision under domain shifts. To mitigate this limitation, we introduce off-the-shelf Vision Foundation Models (VFMs) as external knowledge sources. Zero-shot VFMs are insufficient for medical segmentation because they lack medical semantics, yet they contain rich and heterogeneous generic knowledge. To exploit such external knowledge, we propose Reciprocal Distillation with a Frozen Foundation Model (ReDi-FM), a novel framework with two core components: using structure-aware prompts from the source model to guide the frozen VFM to generate target-adapted supervision, and distilling the resulting knowledge back into the adapting model. For robust distillation, we introduce two complementary objectives: uncertainty-driven hard distillation for precise guidance in ambiguous regions, and hard class-balanced soft distillation for richer supervision of under-represented and challenging structures. A consensus-aware gate further stabilizes adaptation when the two teachers disagree. Extensive experiments on multi-domain medical segmentation benchmarks demonstrate that ReDi-FM outperforms state-of-the-art CTTA methods. Code is available at https://github.com/M4cheal/ReDi-FM.