Research map / Image generation / Fast diffusion sampling
Scaling Inference Time Compute for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that searching over initial noises at inference (random search, zero-order search, search over paths) with various verifiers scales diffusion model sample quality beyond increasing denoising steps.
Problem
Diffusion model quality gains from more denoising steps plateau after a few dozen NFEs, so there is no clear way to further scale inference-time compute for diffusion models.
Approach
Treat the sampling noise as the search space: allocate extra function evaluations to searching for better initial noises (or noise trajectories) scored by verifiers, structured along verifier type and search algorithm axes.
Key results
- On ImageNet, random search with oracle verifiers substantially improves FID and IS as NFEs spent on search increase beyond denoising-step scaling.
- With FLUX.1-dev on T2I-CompBench, ImageReward verifier search improves Color from 0.7692 to 0.8303 and Spatial from 0.2429 to 0.3151 over no-search baseline.
- PixArt-Σ with search at about 2.59x FLUX compute reaches Aesthetic 6.20 and ImageReward 1.15, surpassing FLUX-1.dev without search (5.79, 0.97).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add random search using ImageReward as the verifier | FLUX.1-dev · T2I-CompBench validation | Color: 0.7692 -> 0.8303 | Helped |
| Add random search using ImageReward as the verifier | FLUX.1-dev · T2I-CompBench validation | Shape: 0.5187 -> 0.6274 | Helped |
| Add random search using ImageReward as the verifier | FLUX.1-dev · T2I-CompBench validation | Texture: 0.6287 -> 0.7364 | Helped |
| Add random search using ImageReward as the verifier | FLUX.1-dev · T2I-CompBench validation | Spatial: 0.2429 -> 0.3151 | Helped |
| Add random search using ImageReward as the verifier | FLUX.1-dev · T2I-CompBench validation | Numeracy: 0.6167 -> 0.6789 | Helped |
| Add random search using ImageReward as the verifier | FLUX.1-dev · T2I-CompBench validation | Complex: 0.3600 -> 0.3810 | Helped |
| Apply DPO finetuning and Verifier Ensemble search | SDXL · DrawBench | PickScore: 22.39 -> 23.54 | Helped |
| Add Verifier Ensemble search | PixArt-Σ · DrawBench | ImageReward: 0.70 -> 1.15 | Helped |
| Add Verifier Ensemble search | PixArt-Σ · DrawBench | LLM Grader: 84.67 -> 86.95 | Helped |
| Use Aesthetic Score as the search verifier | FLUX.1-dev · T2I-CompBench validation | Color: 0.7692 -> 0.7618 | Hurt |
| Use Aesthetic Score as the search verifier | FLUX.1-dev · T2I-CompBench validation | Spatial: 0.2429 -> 0.2593 | Helped |
| Use CLIP as the search verifier | FLUX.1-dev · T2I-CompBench validation | Shape: 0.5187 -> 0.5722 | Helped |
Surprising findings
- Searching against a single differentiable-free verifier (e.g., CLIP or Aesthetic) can overfit to its bias (verifier hacking), degrading other metrics.
- Small models with search (e.g., PixArt-Σ, SiT-L) can outperform larger models without search at equal inference compute.
Limitations
- Verifier hacking: per-sample verifiers bias search toward their own preferences, hurting other aspects.
- Oracle verifiers (FID/IS) are generally unavailable in practice, and best verifier depends on the task.
- First-order (gradient-based) search is too costly in time and memory for large models.
Experimental setup
Datasets: ImageNet DrawBench T2I-CompBench Pick-a-Pic
Benchmarks: ImageNet FID/IS DrawBench T2I-CompBench
Metrics: FID Inception Score CLIPScore Aesthetic Score ImageReward PickScore LLM Grader score T2I-CompBench attribute/spatial/numeracy/complex scores
Baselines: SiT-XL denoising-step scaling FLUX.1-dev without search Diffusion-DPO SDXL Random Search Zero-Order Search Search over Paths
Builds on: EDM Restart Sampling Diffusion-DPO
Closest papers by meaning
- Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing (NeurIPS 2025 · similarity 0.93)
- Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models (NeurIPS 2025 · similarity 0.92)
- Test-Time Scaling of Diffusion Models via Noise Trajectory Search (NeurIPS 2025 · similarity 0.91)
- Diffusion Tree Sampling: Scalable inference‑time alignment of diffusion models (NeurIPS 2025 · similarity 0.91)
- Denoising Time Matters: Diverse Generation in Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
- Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.