Research map / Image generation / Fast diffusion sampling
Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Siamese-Diffusion, which trains a ControlNet-based Stable Diffusion under mask-only and mask-image prior controls with a Noise Consistency Loss to make mask-only sampling produce higher-fidelity medical image-mask pairs for segmentation augmentation.
Problem
Medical image segmentation is limited by scarce annotated data, and diffusion models used to synthesize image-mask pairs also suffer from data scarcity, so mask-only generators yield low morphological fidelity while mask-image generators lose diversity.
Approach
A Siamese training scheme shares one diffusion model under mask-only (Mask-Diffusion) and mask-image (Image-Diffusion) conditions; a stop-gradient Noise Consistency Loss pulls Mask-Diffusion's predicted noise toward Image-Diffusion's, and only the mask-only branch is used at sampling, keeping diversity while improving fidelity.
Key results
- Siamese-Diffusion improves SANet mDice and mIoU on Polyps by 3.6% and 4.4% over the prior SOTA ArSDM (overall 83.0 mDice / 75.8 mIoU vs 81.5 / 74.1).
- On ISIC2018, UNet improves by 1.52% mDice and 1.64% mIoU with random-mask synthetic data.
- FID on polyp images drops to 62.706 from 98.085 for ArSDM and 70.630 for ControlNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| augment SANet training with Siamese-Diffusion polyp images | SANet · Polyps, overall | mDice: 79.4 -> 83.0 | Helped |
| augment UNet training with Siamese-Diffusion images made using transformed masks | UNet · ISIC2018 | mDice: 82.19 -> 83.71 | Helped |
| add Dense Hint Input (ablation) | Siamese-Diffusion · Polyps | FID: 70.630 -> 70.479 | Helped |
| add Online-Augmentation (ablation) | Siamese-Diffusion · Polyps | FID: 70.479 -> 65.927 | Helped |
| add Noise Consistency Loss (ablation) | Siamese-Diffusion · Polyps | SANet mDice: 80.5 -> 81.8 | Helped |
| set Noise Consistency Loss weight wc to 0.5 (ablation) | Siamese-Diffusion · Polyps | SANet mDice: 80.5 -> 82.0 | Helped |
| set Noise Consistency Loss weight wc to 1.0 (ablation) | Siamese-Diffusion · Polyps | SANet mDice: 80.5 -> 83.0 | Helped |
| set Noise Consistency Loss weight wc to 1.5 (ablation) | Siamese-Diffusion · Polyps | SANet mDice: 80.5 -> 82.4 | Helped |
| set Noise Consistency Loss weight wc to 2.0 (ablation) | Siamese-Diffusion · Polyps | SANet mDice: 80.5 -> 81.4 | Helped |
| use transformed masks rather than real masks for synthetic ISIC2018 training images (ablation) | UNet · ISIC2018 | mDice: 82.81 -> 83.71 | Helped |
| increase synthetic Polyps training data from threefold to fourfold (ablation) | SegFormer · Polyps | mDice | Hurt |
Surprising findings
- Mask-image ControlNet yields better image morphology but lowers segmentation performance for Polyp-PVT and CTNet, indicating diversity matters as much as fidelity.
- Segmentation models trained on synthetic images from random transformed masks outperform those trained on images from real masks on ISIC2018.
Limitations
- Performance gains decline when augmented data is increased to fourfold on Polyps, attributed to a long-tail problem.
- Image quality metrics such as FID do not reliably track segmentation gains.
Experimental setup
Datasets: Kvasir CVC-ClinicDB ISIC2016 ISIC2018 EndoScene ETIS ColonDB Stain (internal) Faeces (internal)
Benchmarks: EndoScene ClinicDB Kvasir ColonDB ETIS ISIC2016 ISIC2018 Stain Faeces
Metrics: FID KID CLIP-I LPIPS CMMD MOS mDice mIoU
Baselines: SinGAN-Seg T2I-Adapter ArSDM ControlNet Copy-Paste DFMGAN AnomalyDiffusion
Builds on: ControlNet Stable Diffusion DDIM ArSDM
Closest papers by meaning
- Factorized Diffusion Architectures for Unsupervised Image Generation and Segmentation (NeurIPS 2024 · similarity 0.86)
- MedSegDiff-V2: Diffusion-Based Medical Image Segmentation with Transformer (AAAI 2024 · similarity 0.86)
- Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustness (NeurIPS 2024 · similarity 0.85)
- DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction (NeurIPS 2024 · similarity 0.85)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.85)
- Simplified and Generalized Masked Diffusion for Discrete Data (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.