Research map / Image generation / Fast diffusion sampling
Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCDM, a conditional diffusion model for semantic image synthesis that diffuses semantic label maps with a masking-based discrete Label Diffusion (plus class-wise noise schedule) so generation is robust to noisy user label maps.
Problem
Semantic image synthesis degrades when test-time semantic label maps are noisy (jagged edges, incomplete or erroneous user marks), because models are trained only on clean labels.
Approach
Diffuse the semantic label map itself with an absorbing-state discrete diffusion (label masking) in both training and sampling, so the conditioning labels at intermediate timesteps of noisy and clean inputs converge to identical all-mask maps at t=T; add a class-wise masking schedule based on class area and frequency and an extrapolation step in x0 space.
Key results
- On noisy ADE20K labels (DS/Edge/Random), SCDM reaches FID 32.4/31.2/28.1 with mIoU 44.7/40.1/45.2, the best FID among diffusion and GAN baselines in all three setups.
- On clean ADE20K with 25 sampling steps, SCDM reaches FID 27.7, which beats the 1000-step baseline (FID 28.1) in the ablation.
- On clean CelebAMask-HQ, SCDM reports FID 17.4 vs 18.8 for SDM, the best among diffusion models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCDM instead of SDM | ADE20K DS noisy-label synthesis · ADE20K DS noisy labels | FID: 35.5 -> 32.4 | Helped |
| Use SCDM instead of OASIS | ADE20K Edge noisy-label synthesis · ADE20K Edge noisy labels | FID: 37.6 -> 31.2 | Helped |
| Use SCDM instead of LDM | ADE20K Random noisy-label synthesis · ADE20K Random noisy labels | FID: 36.3 -> 28.1 | Helped |
| Use SCDM instead of SDM | CelebAMask-HQ clean-label synthesis · CelebAMask-HQ | FID: 18.8 -> 17.4 | Helped |
| Use SCDM instead of SDM | ADE20K clean-label synthesis · ADE20K | FID: 27.5 -> 26.9 | Helped |
| Use SCDM instead of SDM | COCO-Stuff clean-label synthesis · COCO-Stuff | FID: 15.9 -> 15.3 | Helped |
| Add Label Diffusion to fixed-label conditioning (ablation) | ADE20K generation at 25 steps · ADE20K, 25 sampling steps | FID: 44.6 -> 39.5 | Helped |
| Add extrapolation to Label Diffusion (ablation) | ADE20K generation at 25 steps · ADE20K, 25 sampling steps | FID: 39.5 -> 27.7 | Helped |
| Use the class-wise noise schedule instead of linear and uniform scheduling (ablation) | SCDM · ADE20K | mIoU: 43.0 -> 49.4 | Helped |
| Add Label Diffusion instead of using the fixed-label baseline (ablation) | SCDM noisy-label generation · ADE20K DS noisy labels vs clean-label outputs | FID: 24.2 -> 19.0 | Helped |
| Add Label Diffusion instead of using the fixed-label baseline (ablation) | SCDM noisy-label generation · ADE20K Edge noisy labels vs clean-label outputs | FID: 32.7 -> 20.0 | Helped |
| Add Label Diffusion instead of using the fixed-label baseline (ablation) | SCDM noisy-label generation · ADE20K Random noisy labels vs clean-label outputs | FID: 145.6 -> 10.0 | Helped |
Surprising findings
- With only 25 sampling steps, the full SCDM outperforms the 1000-step baseline in FID on ADE20K (27.7 vs 28.1).
Limitations
- Only the authors' label perturbations (DS, Edge, Random) are tested; real human annotation noise is not measured.
- The class-wise schedule uses a single hyperparameter eta set to 1 without a broader search in the main experiments.
- The paper notes a potential reinforcement of dataset biases in generated samples (impact statement).
Experimental setup
Datasets: ADE20K CelebAMask-HQ COCO-Stuff
Benchmarks: ADE20K CelebAMask-HQ COCO-Stuff noisy SIS (DS, Edge, Random) on ADE20K
Metrics: FID mIoU LPIPS SSIM PSNR
Baselines: SPADE CC-FPSE DAGAN GroupDNet OASIS CLADE INADE SCGAN SAFM SDM LDM PITI
Builds on: DDPM D3PM classifier-free guidance SDM ADM
Closest papers by meaning
- Label-Noise Robust Diffusion Models (ICLR 2024 · similarity 0.88)
- Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis (CVPR 2025 · similarity 0.88)
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.88)
- Don't Drop Your Samples! Coherence-Aware Training Benefits Conditional Diffusion (CVPR 2024 · similarity 0.88)
- Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion (ICML 2024 · similarity 0.87)
- Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.