Research map / Image generation / Fast diffusion sampling
HSRDiff: A Hierarchical Self-Regulation Diffusion Model for Stochastic Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HSRDiff, a conditional diffusion model for stochastic semantic segmentation that jointly predicts noise and label maps with a learned self-regulating mixing parameter and hierarchical multi-scale condition features, evaluated on LIDC-IDRI, MS-Lesion, and Cityscapes.
Problem
Stochastic semantic segmentation in ambiguous, safety-critical settings needs multiple plausible segmentations, but existing diffusion methods rely on global conditioning and a single prediction mode, limiting diversity and fine-structure fidelity.
Approach
Two independent decoders predict noise and label map separately ("differentiation"), then a learned self-regulating parameter softmax-weights the two resulting posterior means at each denoising step ("unification"); multi-scale encoder features are fed to the decoders as hierarchical conditions rather than only a global condition.
Key results
- On LIDC-IDRI with 16 samples, HSRDiff reaches GED 0.181 and HM-IoU 0.697, versus 0.194 and 0.664 for CCDM.
- On Cityscapes stochastic setting (Experimental Group 1), HSRDiff achieves GED16 0.161 and HM-IoU16 0.643, versus 0.171 and 0.628 for CCDM.
- On Cityscapes classic segmentation at 512x1024 with 16 samples, HSRDiff reaches mIoU 0.779 versus 0.711 for CCDM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add global condition guidance to noise prediction (ablation) | HSRDiff ablation model · LIDC-IDRI | GED16: 0.425 -> 0.276 | Helped |
| Add hierarchical multi-scale condition guidance (ablation) | HSRDiff ablation model with global condition · LIDC-IDRI | GED16: 0.276 -> 0.219 | Helped |
| Add independent label-map prediction to noise prediction (ablation) | HSRDiff ablation model with hierarchical condition · LIDC-IDRI | GED16: 0.219 -> 0.230 | Hurt |
| Add multi-objective training for dual prediction (ablation) | HSRDiff ablation model with hierarchical condition · LIDC-IDRI | GED16: 0.230 -> 0.205 | Helped |
| Add self-regulating differentiation-to-unification prediction (ablation) | HSRDiff with hierarchical condition guidance · LIDC-IDRI | GED16: 0.205 -> 0.181 | Helped |
| Use the light model scale instead of HSRDiff | HSRDiff · MS-Lesion | GED16: 0.306 -> 0.329 | Hurt |
| Use the light model scale instead of HSRDiff | HSRDiff · MS-Lesion | MDM16: 0.833 -> 0.835 | Helped |
| Use 16 samples instead of one sample | HSRDiff · Cityscapes, deterministic, 512×1024 | mIoU: 0.761 -> 0.779 | Helped |
Surprising findings
- The lightweight HSRDiff-light (12.89M params) sometimes outperforms the full HSRDiff, e.g. on MS-Lesion GED and at lower resolutions on Cityscapes classic segmentation.
- The learned self-regulating parameter shows dataset-specific preferences: LIDC-IDRI favors noise prediction early, while Cityscapes relies almost entirely on label prediction.
Limitations
- Ablation and parameter trends are reported on a single dataset (LIDC-IDRI) in detail.
- Full HSRDiff has 96.14M parameters on LIDC-IDRI, larger than several baselines, though the light variant is smaller.
Experimental setup
Datasets: LIDC-IDRI MS-Lesion Cityscapes
Benchmarks: LIDC-IDRI MS-Lesion Cityscapes
Metrics: GED HM-IoU MDM mIoU
Baselines: Prob. U-Net HProb. U-Net PhiSeg MoSE AB CIMD CCDM DeepLabv3 DeepLabv3+ PSPNet HRNet UNet++
Builds on: DDPM Benny and Wolf dual-mode prediction Probabilistic U-Net CCDM
Closest papers by meaning
- Diffusion-Driven Two-Stage Active Learning for Low-Budget Semantic Segmentation (NeurIPS 2025 · similarity 0.87)
- Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation (NeurIPS 2024 · similarity 0.86)
- Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis (ICML 2024 · similarity 0.86)
- EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models (ICLR 2024 · similarity 0.85)
- On the Relaxation of Conditional Independence Assumption for Image Segmentation (NeurIPS 2026 · similarity 0.85)
- Flattening the Parent Bias: Hierarchical Semantic Segmentation in the Poincare Ball (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.