Research map / Image generation / Autoregressive and latent generation
Constrained Synthesis with Projected Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Projected Diffusion Models (PDM) cast score-based diffusion sampling as constrained optimization, enforcing arbitrary (convex and non-convex) constraints by projecting each Langevin/gradient update onto the feasible set.
Problem
Diffusion models produce samples that often violate hard constraints or physical laws, and existing conditioning or post-hoc correction methods give no guarantee of feasibility or preserve sample quality.
Approach
Recasts the reverse score-based sampling as a constrained optimization over the chain and enforces feasibility by interleaving noisy gradient (SGLD-style) updates with Euclidean projections onto an arbitrary constraint set at every diffusion step, warm-starting the projections.
Key results
- On HumanML3D, PDM achieves zero penetration and zero float violations, versus 0.918-0.998 penetration and 2.601-3.173 float for the best PhysDiff settings, at FID 0.71 (PhysDiff best FID 0.43).
- On physics-informed motion, PDM gives exact (zero-tolerance) constraint satisfaction with FID comparable to the conditional model, while post-processing degrades image quality.
- On constrained trajectories with non-convex obstacles, Cond and Cond+ fail to find feasible paths in a single sample, whereas PDM reports feasible solutions with comparable path length.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply repeated projections during reverse diffusion instead of a single post-processing projection (ablation) | PDM on Constrained Trajectories · Constrained Trajectories | feasibility / success percentage | Helped |
| Apply a single post-processing projection instead of repeated projections during sampling (ablation) | PDM on Constrained Trajectories · Constrained Trajectories | feasibility | No clear effect |
| Add repeated projections during reverse diffusion | PDM on HumanML3D · HumanML3D | Penetrate: 0.998 -> 0.00 | Helped |
| Add repeated projections during reverse diffusion | PDM on HumanML3D · HumanML3D | Float: 2.601 -> 0.00 | Helped |
| Use PhysDiff (End 4, Space 1) instead of PDM | HumanML3D motion generation · HumanML3D | FID: 0.71 -> 0.43 | Helped |
| Use variational lower bound training objective instead of score-matching objective (ablation) | PDM · Physics-informed Motion | FID: 113.8 $\pm$ 4.9 | Hurt |
| Use variational lower bound training objective instead of score-matching objective (ablation) | PDM · Constrained Materials | FID: 388.2 $\pm$ 13.0 | Hurt |
| Use the SDE-based score model implementation (ablation) | PDM on Physics-informed Motion · Physics-informed Motion | Inception score: 24.2 | Helped |
| Apply post-processing projection after sampling instead of projecting throughout sampling (ablation) | Post+ on constrained material synthesis · Constrained Materials | image quality (FID) | Hurt |
Surprising findings
- Training with a variational lower-bound objective instead of score matching performed much worse for PDM (FID 113.8 on physics-informed motion and 388.2 on materials), contrary to the common DDPM practice.
- Projections in the forward process are unnecessary; the training data need not contain any feasible points.
Limitations
- Substantial computational overhead from projections, especially for non-convex constraints solved on CPU via interior point methods.
- Feasibility guarantees are proven only for convex constraint sets; non-convex cases are empirical.
- Constraints are not incorporated in the forward process, and projecting perturbed samples may diverge from the learned distribution (conjectured).
Experimental setup
Datasets: HumanML3D microstructure image patches (from authors of prior work) physics-informed projectile frames (synthetic) motion planning trajectory maps
Benchmarks: HumanML3D physics-informed motion (Earth and Moon gravity) constrained materials microstructures constrained trajectory maps Tp1 and Tp2
Metrics: FID constraint satisfaction rate at error tolerance penetration and float violations success percentage path length sampling run-time inception score
Baselines: Conditional diffusion model (Cond) Cond+ (conditional with post-processing projection) Post+ (unconditional score model with final projection) PhysDiff Motion Planning Diffusion
Builds on: score-based generative modeling stochastic gradient langevin dynamics denoising diffusion probabilistic models classifier-free guidance PhysDiff
Closest papers by meaning
- Constrained Generative Modeling with Manually Bridged Diffusion Models (AAAI 2025 · similarity 0.91)
- Training-Free Constrained Generation With Stable Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Constrained Diffusion with Trust Sampling (NeurIPS 2024 · similarity 0.89)
- Physics-Informed Diffusion Models (ICLR 2025 · similarity 0.87)
- Generative Modeling with Phase Stochastic Bridge (ICLR 2024 · similarity 0.87)
- On the Trajectory Regularity of ODE-based Diffusion Sampling (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.