Research map / Image generation / Fast diffusion sampling
Beyond Scores: Proximal Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Proximal Diffusion Models (ProxDM), which replace score-based forward Euler-Maruyama sampling with backward-discretized (proximal) reverse-SDE updates, learning proximal operators of log-densities via proximal matching, with KL convergence guarantees and faster sampling on MNIST and CIFAR10.
Problem
Score-based diffusion models need many sampling steps for high quality and rely on forward discretization of the reverse SDE, which is sensitive to step size and to nonsmooth or irregular data distributions.
Approach
Discretize the reverse-time SDE backward (implicit Euler), so each update is a proximal map of the log-density rather than a score step; learn these proximal operators directly from data with a proximal-matching loss, parameterized as x minus sqrt(lambda) times a network, with fully backward and hybrid variants.
Key results
- On CIFAR10, ProxDM at 10 sampling steps matches the FID of the score ODE sampler at 20 steps and surpasses the score SDE sampler even at 100 steps.
- Theory: full backward ProxDM needs O~(d/sqrt(eps)) steps for eps-accurate KL, versus O~(d/eps) for hybrid and prior score-based vanilla rates.
- On MNIST and CelebA-HQ 256x256, ProxDM gives significant speedup over score SDE and ODE samplers at few steps; FID deteriorates at very large step counts (e.g., 1000 on CIFAR10).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ProxDM instead of score-based samplers | ProxDM · synthetic discrete 2D distribution | Wasserstein-2 distance | Helped |
| use hybrid ProxDM instead of score SDE or score ODE | ProxDM · MNIST | FID | Helped |
| use ProxDM instead of score-based samplers | ProxDM · CIFAR10 | FID | Helped |
| use ProxDM at 10 steps instead of score ODE at 20 steps | ProxDM · CIFAR10 | FID | No clear effect |
| use ProxDM at 10 steps instead of score SDE at 100 steps | ProxDM · CIFAR10 | FID | Helped |
| train ProxDM specifically for small step counts rather than across the full range (ablation) | ProxDM · CIFAR10 | FID | Helped |
| disregard some of the proposed heuristics when targeting very few steps (ablation) | ProxDM · CIFAR10 | FID | Helped |
| use hybrid ProxDM instead of score SDE | ProxDM · CelebA-HQ (256×256) | FID | Helped |
| use hybrid ProxDM instead of score ODE | ProxDM · CelebA-HQ (256×256) | FID | No clear effect |
| use residual parameterization for the proximal network instead of direct parameterization (ablation) | ProxDM · large sampling-step counts | empirical results | Helped |
| increase ProxDM sampling from 100 to 1000 steps (ablation) | ProxDM · CIFAR10 | FID | Hurt |
Surprising findings
- ProxDM's FID at 1000 steps is worse than at 100 steps on CIFAR10, attributed to inaccuracies in approximate implicit proximal updates accumulating.
- Standard Euler-Maruyama without a final denoising step fails with only 5 steps, while ProxDM works well without extra hyperparameters.
Limitations
- Theory assumes exact oracle proximal operators; learned approximations are not analyzed.
- Convergence results require additional regularity conditions (smoothness, Hessian Lipschitz-type bounds) not shown to be necessary.
- Quality degrades at large step counts due to accumulated proximal approximation errors.
Experimental setup
Datasets: MNIST CIFAR10 CelebA-HQ
Benchmarks: MNIST CIFAR10 CelebA-HQ
Metrics: FID Wasserstein-2 distance precision recall KL divergence (theory)
Baselines: Score SDE (Euler-Maruyama) Score ODE (probability flow) Standard Euler with denoising
Builds on: score-based SDE diffusion proximal matching (learned proximal networks) proximal Langevin algorithm Euler-Maruyama sampler probability flow ODE
Closest papers by meaning
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.93)
- Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.90)
- Dimension-free Score Matching and Time Bootstrapping for Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Accelerating Convergence of Score-Based Diffusion Models, Provably (ICML 2024 · similarity 0.90)
- Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.