Research map / Image generation / Fast diffusion sampling
Rethinking Peculiar Images by Diffusion Models: Revealing Local Minima’s Role
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes adding naive and positive-negative momentum to diffusion sampling (DDIM) to escape local minima, reducing peculiar image generation (e.g., extra or missing limbs) at no extra compute cost.
Problem
Diffusion models frequently generate peculiar images with anatomical artifacts (extra fingers, wrong leg counts), and fast sampling with few steps worsens quality; existing fixes rely on heuristics, extra training, or post-processing.
Approach
Frames diffusion sampling and SGD as instances of generalized EM, hypothesizes peculiar images correspond to local minima, and applies momentum (including positive-negative momentum with negative beta) to the full DDIM update term to escape them without extra network evaluations.
Key results
- On CelebA with linear schedule, PLMS with positive-negative momentum at 50 NFE reaches FID 2.20 versus 4.52 without momentum, at the same compute.
- DDIM with momentum on CIFAR-10 (quadratic skip, 50 NFE) improves FID from 4.53 to 3.45 with beta=-0.3.
- Applied to Stable Diffusion with DDIM T=100, positive-negative momentum raises average OpenPose joint confidence for single persons from 0.4351 to 0.4383 and for groups from 0.1880 to 0.1991.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add momentum to PLMS sampling at T=50 | PLMS on CelebA · CelebA 64×64 | FID: 4.52 -> 2.20 | Helped |
| reduce sampling steps from T=250 to T=50 without momentum | PLMS on CelebA · CelebA 64×64 | FID: 2.45 -> 4.52 | Hurt |
| add momentum to PLMS sampling at T=50 | PLMS on CelebA · CelebA 64×64 | GFLOPS/img: 50×46.74 -> 50×46.74 | No clear effect |
| add positive-negative momentum (β=-0.8) to DDIM (ablation) | DDIM on CIFAR10 · CIFAR10, linear skip, 50 NFE | FID: 7.19 -> 13.24 | Hurt |
| add positive-negative momentum (β=-0.8) to DDIM (ablation) | DDIM on CIFAR10 · CIFAR10, quadratic skip, 50 NFE | FID: 4.53 -> 4.25 | Helped |
| add positive-negative momentum (β=-0.8) to DDIM (ablation) | DDIM on CelebA · CelebA, linear skip, 50 NFE | FID: 5.99 -> 13.75 | Hurt |
| add naive momentum (β=0.2) to DDIM (ablation) | DDIM on CelebA · CelebA, linear skip, 50 NFE | FID: 5.99 -> 3.91 | Helped |
| add naive momentum (β=0.2) to PLMS (ablation) | PLMS on CelebA · CelebA, linear skip, 50 NFE | FID: 4.52 -> 2.20 | Helped |
| add positive-negative momentum (β=-0.3) to DPM++ (ablation) | DPM++ on CIFAR10 · CIFAR10, quadratic skip, 50 NFE | FID: 3.99 -> 3.45 | Helped |
| add positive-negative momentum (β=-0.3) to DDIM sampling (ablation) | Stable Diffusion · single-person human images | average joint confidence: 0.4351 -> 0.4383 | Helped |
| add positive-negative momentum (β=-0.3) to DDIM sampling (ablation) | Stable Diffusion · group human images | average joint confidence: 0.1880 -> 0.1991 | Helped |
| add positive-negative momentum (β=-0.3) to DDIM sampling (ablation) | Stable Diffusion · single-person human images, 5–50 clusters | average RMSE to nearest cluster center: 0.2823 -> 0.2782; 0.2589 -> 0.2515; 0.2461 -> 0.2416; 0.2407 -> 0.2378; 0.2221 -> 0.2219 | Mixed |
Surprising findings
- Negative momentum (beta<0) worsens FID for small step counts (T<=25) but improves it for larger step counts, with the optimal beta roughly proportional to 1/T.
- Naive (positive) momentum sometimes degrades DDIM FID at low NFE (e.g., CelebA linear DDIM at 50 NFE rises from 5.99 to 3.91 with beta=0.2, but beta=-0.8 gives 13.75).
Limitations
- Peculiarity metrics only assess human images via OpenPose-based pose confidence and clustering distance.
- The local-minima hypothesis is supported by visual and qualitative analysis rather than a formal proof.
- Optimal beta depends on step count T and is tuned empirically.
Experimental setup
Datasets: CIFAR-10 CelebA MS COCO
Benchmarks: CIFAR-10 FID CelebA FID
Metrics: FID NFE average joint confidence (OpenPose) RMSE to nearest pose cluster center GFLOPS per image
Baselines: DDPM DDIM PLMS DPM-Solver++
Builds on: DDIM DDPM PLMS DPM-Solver++ SGD with momentum
Closest papers by meaning
- Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.89)
- Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.89)
- Diffusion Sampling with Momentum for Mitigating Divergence Artifacts (ICLR 2024 · similarity 0.89)
- CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation (NeurIPS 2025 · similarity 0.88)
- Accelerating Diffusion Sampling with Optimized Time Steps (CVPR 2024 · similarity 0.88)
- Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.