Research map / Image generation / Fast diffusion sampling
Constrained Generative Modeling with Manually Bridged Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces manually bridged models (MBM), a diffusion framework that adds differentiable distance-function-based "manual bridge" terms to the score (with a conditioning path) to enforce multiple constraints while fitting data, evaluated on checkerboard, traffic scene generation, and watermarking tasks.
Problem
Diffusion models trained on constrained data (e.g., vehicles on roads without overlap) produce constraint violations at high rates, and existing constrained diffusion methods (diffusion bridges, reflected/mirror diffusion) are intractable or limited to simple convex constraints.
Approach
Define manual bridges as gradient terms -gamma(t) grad of an Omega-distance function, show that sums of such bridges give a bridge to the intersection of constraints, and use an architecture (MBM-arch) that both adds the bridge to the score and feeds the bridge as a conditioning input to stabilize training.
Key results
- Traffic scene generation: MBM-arch reduces infraction rate from 23.00% (standard diffusion) to 0.10% while keeping r-ELBO at -0.95 vs -0.94 for standard diffusion.
- Watermarking on AFHQv2: MBM-arch achieves 0.02% infraction rate with best FID of 2.65 versus 2.95 for EDM-finetuned.
- DB-arch alone reaches 0.25% infraction in traffic but degrades r-ELBO to -1.20; C-arch fails to remove infractions (24.52%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add bridge conditioning alongside the bridge score offset (MBM-arch instead of DB-arch) (ablation) | Manual bridge model · Traffic scene generation | r-ELBO: -1.20\pm 0.01 -> -0.95\pm 0.00 | Helped |
| Add bridge conditioning alongside the bridge score offset (MBM-arch instead of DB-arch) (ablation) | Manual bridge model · Traffic scene generation | Infraction (%): 0.25 \pm 0.03 -> 0.10\pm 0.00 | Helped |
| Condition on the bridge without adding it as a score offset (C-arch) (ablation) | Manual bridge model · Traffic scene generation | Infraction (%): 23.00\pm 0.14 -> 24.52\pm 0.23 | Hurt |
| Add the bridge as a score-function offset without conditioning the score network (DB-arch) (ablation) | Manual bridge model · Traffic scene generation | r-ELBO: -0.94\pm 0.01 -> -1.20\pm 0.01 | Hurt |
| Use guided diffusion with the manual bridge, without further training | Pre-trained diffusion model · Traffic scene generation | r-ELBO: -0.94\pm 0.01 -> -1.53\pm 0.11 | Hurt |
| Add the bridge as a score offset and condition the score network (MBM-arch instead of DB-arch) (ablation) | Manual bridge model · AFHQv2 watermarking | FID: 2.81 \pm 0.05 -> 2.65 \pm 0.07 | Helped |
| Add the bridge as a score offset and condition the score network (MBM-arch instead of DB-arch) (ablation) | Manual bridge model · AFHQv2 watermarking | r-ELBO (× 10²): -16.54 \pm 0.10 -> -15.83 \pm 0.03 | Helped |
| Condition on the bridge without adding it as a score offset (C-arch) (ablation) | Manual bridge model · AFHQv2 watermarking | Infraction (%): 91.44 \pm 0.25 -> 90.74 \pm 0.17 | No clear effect |
| Add the bridge as a score-function offset without conditioning the score network (DB-arch) (ablation) | Manual bridge model · AFHQv2 watermarking | r-ELBO (× 10²): -15.78 \pm 0.07 -> -16.54 \pm 0.10 | Hurt |
| Add a nonzero-indicator channel to the bridge conditioning signal (ablation) | Manual bridge models with C-arch or MBM-arch · AFHQv2 watermarking | Model performance | Helped |
Surprising findings
- Conditioning-only (C-arch) barely reduces or even increases infraction rate in traffic generation (24.52% vs 23.00% baseline), so the offset path is essential.
- Guided diffusion achieves zero infractions but with a much worse r-ELBO (-1.53), indicating strong distribution shift.
Limitations
- Requires differentiable distance functions, which may be hard to implement for some constraints.
- Requires substantial hyperparameter tuning (integration schedule, minimum noise, bridge scaling).
- Manual bridges are not guaranteed to be diffusion bridges, so SDE solutions are not formally guaranteed to satisfy constraints.
Experimental setup
Datasets: checkerboard 2D synthetic dataset traffic scene dataset (70 maps) AFHQv2 (watermarked)
Benchmarks: checkerboard constraint experiment traffic scene generation (collision/offroad) AFHQv2 watermarking
Metrics: collision rate offroad rate infraction rate r-ELBO infraction loss FID
Baselines: Standard diffusion Guided diffusion Diffusion bridge (DB-arch) Conditional diffusion (C-arch) EDM-pretrained EDM-finetuned MDM MDM-proj
Builds on: diffusion bridges EDM mirror diffusion models conditional diffusion
Closest papers by meaning
- Constrained Synthesis with Projected Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Constrained Diffusion with Trust Sampling (NeurIPS 2024 · similarity 0.89)
- Generative Modeling with Phase Stochastic Bridge (ICLR 2024 · similarity 0.88)
- Constrained Discrete Diffusion (NeurIPS 2025 · similarity 0.87)
- Model-based Diffusion for Trajectory Optimization (NeurIPS 2024 · similarity 0.87)
- Generative Trajectory Stitching through Diffusion Composition (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.