Research map / Label-efficient and robust learning / Domain adaptation and generalization

Perturbating, Tuning, and Collaborating: Harnessing Vision Foundation Models for Single Domain Generalization on Medical Imaging

AAAI 2025 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Foundation-model domain generalization · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CollaSU-SDG, a framework that combines a SAM-based vision foundation model with a specialized U-Net via frequency perturbation, a domain expansion adapter, hierarchical feature transfer, and dense prompts for single-source domain generalization in medical segmentation.

Problem

Single-source domain generalization (SDG) for medical image segmentation, where models trained on one source domain must generalize to unseen target domains, and where SAM-style foundation models need explicit prompts and adaptation to medical domains.

Approach

Jointly trains a specialized U-Net and a SAM-based universal model with model-aware adaptive frequency perturbation injected into both, adapters tuning the ViT for medical data, adaptive hierarchical transfer of universal features into the specialized decoder, and dynamic dense prompts from the specialized model replacing manual SAM prompts.

Key results

• On prostate segmentation, CollaSU-SDG reaches 81.29% average DSC vs 72.58% for the best listed baseline (Med-SAM-full), a gain of 8.39 points.
• On fundus segmentation, average DSC reaches 84.00% (ablation Table 4), a 7.31-point gain over the baseline of 67.53%.
• Collaborating-only variant reaches 82.78 fundus and 77.69 polyp, above LoRA, Adapter and Rein fine-tuning baselines (e.g., Rein 80.36 fundus).

Measured effects reported in the paper

Surprising findings

• Full-box SAM and MedSAM baselines score lower than the ERM-style U-Net baseline on some tasks, indicating that prompt-free vanilla foundation-model use is weak under domain shift.

Limitations

• Relies on explicit whole-image box prompts for SAM baselines, and the collaborative design depends on a specialized model providing dense prompts.
• Evaluated only on three 2D medical segmentation tasks with a single SAM-b backbone in the main results.

Experimental setup

Datasets: Prostate segmentation dataset (six domains) Fundus optic disc/cup dataset (five domains) Polyp segmentation dataset (four domains)

Benchmarks: Prostate segmentation Fundus optic disc/cup segmentation Polyp segmentation

Metrics: DSC ASD E-max S-alpha

Baselines: ERM MixStyle MAD DSU TriD CSDG MaxStyle SLAug SAM-full MedSAM-full EFDM SAN-SAW

Builds on: SAM MixStyle LoRA Rein MedSAM

Closest papers by meaning

• TinyMIG: Transferring Generalization from Vision Foundation Models to Single-Domain Medical Imaging (ICML 2025 · similarity 0.91)
• PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization (CVPR 2024 · similarity 0.88)
• Knowledge Transfer Scaling Laws for 3D Medical Imaging (NeurIPS 2026 · similarity 0.87)
• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.87)
• Domain Adaptive Diabetic Retinopathy Grading with Model Absence and Flowing Data (CVPR 2025 · similarity 0.86)
• MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.86)