Research map / Image generation / Fast diffusion sampling
High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffDIS, a one-step latent diffusion segmentation model built on SD-Turbo/SD V2.1 U-Net that jointly predicts masks and edges for high-precision dichotomous image segmentation on DIS5K.
Problem
High-resolution, fine-grained dichotomous image segmentation requires both broad context and precise boundary detail, and multi-step diffusion models are slow and stochastic, conflicting with deterministic binary mask outputs.
Approach
Repurposes the pretrained SD U-Net (initialized from SD-Turbo) for single-step denoising in VAE latent space to predict masks and edges jointly, with batch-discriminative embeddings, mutual cross-domain attention (DBIA), and scale-wise conditional injection of the RGB latent.
Key results
- On DIS-VD, DiffDIS reaches F-max 0.908 and MAE 0.029, versus 0.904 and 0.034 for the next best (MVANet).
- On overall DIS-TE(1-4), DiffDIS reports F-max 0.911, weighted F 0.892, E-measure 0.950, MAE 0.028, outperforming BiRefNet and MVANet on most metrics.
- One-step SD-Turbo initialization gives 0.883 F-max at 0.33 s inference, versus 0.837 for SD V2.1 with 10-step denoising at 1.40 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add auxiliary edge prediction (ablation) | Single-step SD-Turbo segmentation model · DIS-VD | Fβmax: 0.882 -> 0.891 | Helped |
| Add DBIA (ablation) | Edge-assisted DiffDIS · DIS-VD | Fβmax: 0.891 -> 0.896 | Helped |
| Add SWCI (ablation) | Edge-assisted DiffDIS with DBIA · DIS-VD | Fβmax: 0.896 -> 0.897 | Helped |
| Add CutMix augmentation (ablation) | DiffDIS with edge, DBIA, and SWCI · DIS-VD | Fβmax: 0.897 -> 0.908 | Helped |
| Train from scratch rather than initialize with SD-Turbo (ablation) | Single-step DiffDIS · DIS-VD | Fβmax: 0.883 -> 0.682 | Hurt |
| Use SD V2.1 pretrained parameters instead of SD-Turbo (ablation) | Single-step DiffDIS · DIS-VD | Fβmax: 0.883 -> 0.853 | Hurt |
| Use fixed two-step denoising instead of fixed one-step denoising (ablation) | SD-Turbo DiffDIS · DIS-VD | Fβmax: 0.883 -> 0.879 | Hurt |
| Randomize the training timestep and use two-step inference (ablation) | SD-Turbo DiffDIS · DIS-VD | Fβmax: 0.883 -> 0.876 | Hurt |
| Use random-timestep training and ten-step inference (ablation) | SD V2.1 DiffDIS · DIS-VD | Fβmax: 0.853 -> 0.837 | Hurt |
| Replace targeted DBIA interaction with fusion-oriented interaction (ablation) | Edge-assisted DiffDIS with DBIA · DIS-VD | Fβmax: 0.896 -> 0.891 | Hurt |
| Use DiffDIS instead of MVANet | DIS segmentation models · DIS-TE(1-4) | Fβmax: 0.908 -> 0.911 | Helped |
Surprising findings
- Multi-step denoising (10 steps) performed markedly worse than one-step denoising, which the authors attribute to stochastic variation amplifying across steps.
- Training from scratch with one-step denoising performed far worse (F-max 0.682) than initializing from SD-Turbo (0.883).
Limitations
- Some overall DIS-TE metrics are not best (e.g., S-measure on DIS-TE4 and overall S-measure trail BiRefNet/MVANet).
- Evaluated only on DIS5K; generalization to other segmentation benchmarks is not shown.
- Ablations are run on DIS-VD only.
Experimental setup
Datasets: DIS5K
Benchmarks: DIS-VD DIS-TE1 DIS-TE2 DIS-TE3 DIS-TE4
Metrics: max F-measure weighted F-measure S-measure E-measure MAE inference time
Baselines: IS-Net FP-DIS UDUN InSPyReNet BiRefNet MVANet GenPercept BSANet ISDNet IFA PGNet
Builds on: Stable Diffusion SD-Turbo DDPM ControlNet GenPercept
Closest papers by meaning
- DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cut (NeurIPS 2024 · similarity 0.89)
- EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models (ICLR 2024 · similarity 0.89)
- $\mathbf{D^{3}S^{2}}$: Diffusion-Guided Dataset Distillation for Semantic Segmentation (NeurIPS 2026 · similarity 0.89)
- Seg4Diff: Unveiling Open-Vocabulary Semantic Segmentation in Text-to-Image Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models (ICLR 2025 · similarity 0.87)
- Diffusion Models Without Attention (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.