Research map / Image generation / Fast diffusion sampling
MedSegDiff-V2: Diffusion-Based Medical Image Segmentation with Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MedSegDiff-V2, a transformer-based diffusion model for medical image segmentation that combines an Anchor Condition (with Uncertain Spatial Attention) and a Semantic Condition (SS-Former with a timestep-adaptive frequency-domain band-pass filter).
Problem
Naively combining transformers with diffusion-based (UNet) medical segmentation models performs poorly, due to feature incompatibility between the transformer condition features and the noisy-mask diffusion features and high variance of transformer outputs across diffusion steps.
Approach
Two conditioning paths on the diffusion backbone: an Anchor Condition that injects smoothed condition-model decoder features into diffusion encoder features via Uncertain Spatial Attention, and a Semantic Condition using SS-Former, a cross-attention chain in Fourier space whose filter is a timestep-conditioned neural band-pass filter (NBP-Filter).
Key results
- MedSegDiff-V2 reaches average Dice 0.901 on AMOS and 0.895 on BTCV, outperforming MedSegDiff (0.889 and 0.879).
- On REFUGE2-Disc, BraTS, TNMIX, and ISIC, MedSegDiff-V2 reports the best Dice (e.g., 96.7 on REFUGE2-Disc, 90.8 on BraTS).
- MedSegDiff-V2 uses 46M parameters and 983 Gflops versus 25M and 1770 Gflops for MedSegDiff, with 10 ensemble runs instead of 25.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace spatial attention with U-SA in Anchor Condition (ablation) | MedSegDiff-V2 · AMOS | average Dice: 83.5 -> 86.7 | Helped |
| replace spatial attention with U-SA in Anchor Condition (ablation) | MedSegDiff-V2 · BTCV | average Dice: 85.8 -> 86.6 | Helped |
| replace spatial attention with U-SA in Anchor Condition (ablation) | MedSegDiff-V2 · OpticCup | Dice: 85.2 -> 85.7 | Helped |
| replace spatial attention with U-SA in Anchor Condition (ablation) | MedSegDiff-V2 · BrainTumor | Dice: 88.7 -> 89.4 | Helped |
| replace spatial attention with U-SA in Anchor Condition (ablation) | MedSegDiff-V2 · ThyroidNodule | Dice: 84.6 -> 86.5 | Helped |
| add SS-Former without NBP-Filter to U-SA Anchor Condition (ablation) | MedSegDiff-V2 · AMOS | average Dice: 86.7 -> 87.8 | Helped |
| add NBP-Filter to SS-Former (ablation) | MedSegDiff-V2 · AMOS | average Dice: 87.8 -> 90.1 | Helped |
| add NBP-Filter to SS-Former (ablation) | MedSegDiff-V2 · BTCV | average Dice: 87.1 -> 89.5 | Helped |
| add NBP-Filter to SS-Former (ablation) | MedSegDiff-V2 · OpticCup | Dice: 86.5 -> 87.9 | Helped |
| add NBP-Filter to SS-Former (ablation) | MedSegDiff-V2 · BrainTumor | Dice: 89.8 -> 90.8 | Helped |
| add NBP-Filter to SS-Former (ablation) | MedSegDiff-V2 · ThyroidNodule | Dice: 86.8 -> 88.7 | Helped |
| use MedSegDiff-V2 instead of MedSegDiff | medical image segmentation · AMOS | average Dice: 0.889 -> 0.901 | Helped |
Surprising findings
- Naively plugging TransUNet into MedSegDiff (MedSegDiff+TransUNet) performs worse than the original UNet-based MedSegDiff on average.
- EnsDiff and SegDiff, straightforward diffusion segmentation models, underperform transformer baselines UNetr and Swin-UNetr.
Limitations
- Parameter count is higher (46M vs 25M) than the UNet-based MedSegDiff.
- Gains on several individual organ/task entries are mixed; some baselines (e.g., Swin-UNetr, nnUNet) outperform on specific classes.
- Evaluation limited to 2D resized 256x256 inputs and relies on STAPLE ensembling of samples.
Experimental setup
Datasets: AMOS2022 BTCV REFUGE2 BraTS-2021 ISIC 2018 TNMIX
Benchmarks: AMOS2022 BTCV REFUGE2 BraTS-2021 ISIC 2018 TNMIX
Metrics: Dice IoU HD95 Generalized Energy Distance confidence interval
Baselines: nnUNet TransUNet UNetr Swin-UNetr EnsDiff SegDiff MedSegDiff MedSegDiff+TransUNet ResUNet BEAL TransBTS SwinBTS
Builds on: MedSegDiff denoising diffusion probabilistic models TransUNet NeRF
Closest papers by meaning
- L-Diffusion: Laplace Diffusion for Efficient Pathology Image Segmentation (ICML 2025 · similarity 0.86)
- Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation (CVPR 2025 · similarity 0.86)
- DDViT: Double-Level Fusion Domain Adapter Vision Transformer (Student Abstract) (AAAI 2024 · similarity 0.86)
- MedFlowSeg: Flow Matching for Medical Image Segmentation with Frequency-Aware Attention (NeurIPS 2026 · similarity 0.85)
- EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models (ICLR 2024 · similarity 0.84)
- LangDAug: Langevin Data Augmentation for Multi-Source Domain Generalization in Medical Image Segmentation (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.