Research map / Detection and segmentation / Open-set and few-shot detection
S2S2: Semantic Stacking for Robust Semantic Segmentation in Medical Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S2S2, a domain-agnostic add-on training loss that pools feature maps from a stack of diffusion-generated images conditioned on segmentation masks, approximated by pairwise consistency between two samples, to improve in-domain and out-of-domain medical segmentation.
Problem
Medical image segmentation suffers from scarce, low-diversity training data, and domain-specific augmentations or architectures can hurt when domain knowledge is missing or misapplied, limiting robustness and generalization.
Approach
Borrow image stacking from denoising: estimate a denoised semantic feature map from a stack of synthetic images generated from the ground-truth mask; a Bayesian bound shows the deviation from this estimate is upper-bounded by pairwise distances, so only two images per iteration need to be compared via a cosine-based semantic consistency loss added to the segmentation loss.
Key results
- TransUNet on Synapse/ACDC improves from 82.86 to 85.80 mean Dice with S2S2 (+2.94), with Synapse 76.86 to 81.19.
- Out-of-domain on RGB polyp data, FCBFormer+S2S2 raises mean Dice from 88.81 to 90.79 (+1.98).
- Out-of-domain on 3D CT/MRI slices, SLAug+S2S2 reaches 86.11 mean vs 85.96 for SLAug (+0.15), with S2S2 not beating SLAug on CT-MRI and bSSFP-LGE settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add S2S2 to TransUNet | TransUNet · Synapse | Dice: 76.86 -> 81.19 | Helped |
| Add S2S2 to TransUNet | TransUNet · ACDC | Dice: 88.86 -> 90.40 | Helped |
| Add S2S2 to FCBFormer | FCBFormer · Kvasir-SEG and CVC-ClinicDB | Dice: 91.90 -> 93.20; 93.46 -> 94.88 | Helped |
| Add S2S2 to SLAug | SLAug · In-domain abdominal CT, abdominal MRI, and cardiac LGE | Dice: 82.66 -> 84.21; 90.60 -> 91.28; 87.35 -> 87.62 | Helped |
| Add S2S2 to SLAug | SLAug · In-domain cardiac bSSFP | Dice: 92.27 -> 92.16 | Hurt |
| Add S2S2 to SLAug | SLAug · Out-of-domain abdominal MRI-CT and cardiac LGE-bSSFP | Dice: 81.70 -> 83.15; 87.17 -> 87.49 | Helped |
| Add S2S2 to SLAug | SLAug · Out-of-domain abdominal CT-MRI and cardiac bSSFP-LGE | Dice: 88.55 -> 87.75; 86.42 -> 86.06 | Hurt |
| Add S2S2 to FCBFormer | FCBFormer · Out-of-domain Kvasir-CVC and CVC-Kvasir | Dice: 91.16 -> 92.85; 86.46 -> 88.72 | Helped |
| Add synthetic images without semantic consistency loss (ablation) | TransUNet · Synapse and ACDC | Dice: 76.86 -> 77.61; 88.86 -> 89.66 | Helped |
| Add encoder semantic consistency loss to synthetic images (ablation) | TransUNet · Synapse and ACDC | Dice: 77.61 -> 80.29; 89.66 -> 90.64 | Helped |
| Add decoder consistency loss to encoder consistency loss (ablation) | TransUNet · Synapse | Dice: 80.29 -> 81.19 | Helped |
| Add decoder consistency loss to encoder consistency loss (ablation) | TransUNet · ACDC | Dice: 90.64 -> 90.40 | Hurt |
Surprising findings
- Applying the semantic consistency loss with only photometric augmentation (Gaussian blur, color jitter) performs worse than the base method, suggesting the synthetic semantic stack matters.
- S2S2 does not outperform SLAug in the CT-MRI and bSSFP-LGE out-of-domain settings, which the authors attribute to SLAug's domain assumptions aligning with those shifts.
Limitations
- Depends on a fine-tuned generative model whose quality strongly affects outcomes and adds computational cost.
- Less suited to settings with abundant data such as natural image segmentation.
- Decoder-level loss can hurt out-of-domain performance and the gain over domain-specific SLAug is marginal or negative in some settings.
Experimental setup
Datasets: Synapse multi-organ CT ACDC Kvasir-SEG CVC-ClinicDB CHAOS T2-SPIR bSSFP and LGE cardiac MRI
Benchmarks: Synapse ACDC Kvasir-SEG CVC-ClinicDB CHAOS T2-SPIR bSSFP LGE
Metrics: Dice IoU precision recall Hausdorff distance
Baselines: TransUNet R50-AttnUNet ViT-CUP R50-ViT-CUP MSRF-Net PraNet SLAug FCBFormer CSDG RandConv
Builds on: SLAug FCBFormer TransUNet ControlNet Stable Diffusion
Closest papers by meaning
- A Semantic Knowledge Complementarity based Decoupling Framework for Semi-supervised Class-imbalanced Medical Image Segmentation (CVPR 2025 · similarity 0.87)
- ConDSeg: A General Medical Image Segmentation Framework via Contrast-Driven Feature Enhancement (AAAI 2025 · similarity 0.87)
- Versatile Medical Image Segmentation Learned from Multi-Source Datasets via Model Self-Disambiguation (CVPR 2024 · similarity 0.87)
- Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains (NeurIPS 2025 · similarity 0.87)
- A Unified Degradation-Robust Approach to SSL and UDA for 3D Medical Images (AAAI 2025 · similarity 0.87)
- DyCON: Dynamic Uncertainty-aware Consistency and Contrastive Learning for Semi-supervised Medical Image Segmentation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.