Research map / Image generation / Fast diffusion sampling
Don't be so Negative! Score-based Generative Modeling with Oracle-assisted Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Gen-neG, an oracle-guided score-based diffusion method that trains iterative Bayes-optimal binary classifiers on synthetic samples (with importance-sampled class priors) to suppress probability mass outside an oracle-defined valid region, optionally distilled into one model.
Problem
Score-based diffusion models assign probability to invalid samples (constraint violations such as collisions or ground penetration), and rejection sampling with an oracle is computationally costly for low-latency use.
Approach
Use a binary, diffusion-time-dependent classifier trained on synthetic samples from the baseline model labeled by a fixed oracle, with balanced sampling corrected by importance-sampling weights on the class prior, to guide sampling toward the oracle-approved region; stack iterations and optionally distill.
Key results
- On traffic scene generation, Gen-neG distillation of the iteration-2 model reduces infraction from 29.3% (baseline DM) to 5.6% with r-ELBO -27.0e-2 versus -27.5e-2 for baseline.
- On motion diffusion, Gen-neG lowers infraction from 27.66% to 24.25% and improves FID from 0.445 to 0.414 and r-ELBO from -1.06e-2 to -1.01e-2.
- On the checkerboard, Gen-neG reduces infraction rate far more than training the baseline on datasets up to 1000x larger, while keeping ELBO comparable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply oracle-assisted classifier guidance | Baseline DM · Motion diffusion (HumanML3D) | Infraction (%): 27.66 \pm 0.77 -> 24.25 \pm 0.35 | Helped |
| Apply oracle-assisted classifier guidance | Baseline DM · Motion diffusion (HumanML3D) | r-ELBO (× 10^-2): -1.06 \pm 0.02 -> -1.01 \pm 0.03 | Helped |
| Apply oracle-assisted classifier guidance | Baseline DM · Traffic scene generation | Infraction (%): 29.3\pm 0.64 -> 17.2\pm 0.44 | Helped |
| Add a second classifier-guidance iteration | Gen-neG iteration 1 · Traffic scene generation | Infraction (%): 17.2\pm 0.44 -> 12.2\pm 0.60 | Helped |
| Distill the iteration 2 guided model | Gen-neG iteration 2 · Traffic scene generation | Infraction (%): 12.2\pm 0.60 -> 5.6\pm 0.20 | Helped |
| Omit importance sampling in classifier training (ablation) | Gen-neG · Traffic scene generation | Infraction (%): 17.2\pm 0.44 -> 15.2\pm 0.50 | Helped |
| Omit importance sampling in classifier training (ablation) | Gen-neG · Traffic scene generation | r-ELBO (× 10^-2): -27.7\pm 0.01 -> -28.0\pm 0.01 | Hurt |
| Add importance sampling to classifier training (ablation) | Gen-neG without importance sampling · Motion diffusion (HumanML3D) | Infraction (%): 22.85 \pm 0.18 -> 24.25 \pm 0.35 | Hurt |
| Add importance sampling to classifier training (ablation) | Gen-neG without importance sampling · Motion diffusion (HumanML3D) | r-ELBO (× 10^-2): -1.13 \pm 0.06 -> -1.01 \pm 0.03 | Helped |
| Add importance sampling to classifier training (ablation) | Gen-neG without importance sampling · Motion diffusion (HumanML3D) | KID (× 10^-3): 8.40 \pm 1.83 -> 6.99 \pm 0.78 | Helped |
| Balance the synthetic classifier dataset and correct the class-prior shift with importance sampling (ablation) | Gen-neG · Traffic scene generation | Infraction (%): 18.6\pm 1.30 -> 17.2\pm 0.44 | Helped |
| Increase classifier dataset size from 8,000 to 800,000 samples (ablation) | Gen-neG · Traffic scene generation | Infraction (%): 24.5 \pm 0.5 -> 17.2\pm 0.4 | Helped |
Surprising findings
- Classifier guidance with a classifier trained on imbalanced synthetic data (balanced only by model infraction rate) degrades likelihood, showing class-prior correction is necessary.
- Distilled students sometimes outperform their teacher models on infraction and ELBO in the traffic experiment.
Limitations
- Training dataset only enters the baseline DM; later iterations use only synthetic data, so practical errors are not corrected by real data.
- Stacking classifiers increases computational cost linearly with the number of iterations.
- Near-zero infraction rates make balanced synthetic dataset generation slow.
Experimental setup
Datasets: 2D checkerboard (synthetic) private traffic scene dataset HumanML3D
Benchmarks: 2D checkerboard traffic scene generation (private dataset) HumanML3D motion generation
Metrics: infraction rate collision rate offroad rate reweighted ELBO MMD FID KID
Baselines: Baseline DM Normalizing flow traffic generator Time-independent classifier guidance Imbalanced classifier (ablation) Diffusion bridge GAN data redaction MDM
Builds on: classifier guidance EDM DDPM Motion Diffusion Model
Closest papers by meaning
- Evaluating the design space of diffusion-based generative models (NeurIPS 2024 · similarity 0.88)
- Zero-Shot Conditioning of Score-Based Diffusion Models by Neuro-Symbolic Constraints (AAAI 2025 · similarity 0.88)
- Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models (ICLR 2025 · similarity 0.87)
- Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving (CVPR 2024 · similarity 0.87)
- Dynamic Negative Guidance of Diffusion Models (ICLR 2025 · similarity 0.86)
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.