Research map / Image generation / Fast diffusion sampling
BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BayesDiff, which propagates pixel-wise uncertainty through the diffusion reverse process using last-layer Laplace approximation on the noise predictor, enabling sample-wise filtering of low-quality generations and resampling-based diversity/artifact correction.
Problem
Low-quality diffusion generations are hard to identify because standard metrics (FID, IS) are distributional rather than sample-wise; there is no principled per-sample, pixel-wise uncertainty for diffusion outputs.
Approach
Derives an uncertainty iteration principle that propagates mean and variance (plus pixel-wise covariance between state and predicted noise, estimated by Monte Carlo) through common samplers (DDPM, DDIM, DPM-Solver, Analytic-DPM), with the noise-prediction variance obtained from last-layer Laplace approximation; a skip-schedule variant (BayesDiff-Skip) reduces cost.
Key results
- On ImageNet 128 with ADM and DDIM, keeping the lower-uncertainty 50k of 60k generations gives FID 8.48 versus 8.68 for random selection.
- Images in the highest-uncertainty group show higher Recall and lower-uncertainty groups show higher Precision, reflecting a precision-recall trade-off on ImageNet and CelebA.
- BayesDiff-Skip achieves about 5x running-time reduction while keeping the most uncertain images identical to full BayesDiff in a consistency study.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Filter out images with highest uncertainty instead of selecting images randomly | ADM · ImageNet 128, DDIM | FID: $8.68 \pm 0.04$ -> 8.48 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | ADM · ImageNet 128, DDIM | Precision: 0.661 -> 0.665 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | ADM · ImageNet 128, DDIM | Recall: 0.655 -> 0.653 | Hurt |
| Filter out images with highest uncertainty instead of selecting images randomly | ADM · ImageNet 128, 2-order DPM-Solver | FID: $9.77 \pm 0.03$ -> 9.67 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | ADM · ImageNet 128, 2-order DPM-Solver | Precision: 0.657 -> 0.659 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | ADM · ImageNet 128, 2-order DPM-Solver | Recall: 0.649 -> 0.649 | No clear effect |
| Filter out images with highest uncertainty instead of selecting images randomly | U-ViT · ImageNet 256, 2-order DPM-Solver | FID: 7.24 -> 6.81 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | U-ViT · ImageNet 256, 2-order DPM-Solver | Precision: 0.698 -> 0.705 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | U-ViT · ImageNet 256, 2-order DPM-Solver | Recall: 0.658 -> 0.657 | Hurt |
| Filter out images with highest uncertainty instead of selecting images randomly | U-ViT · ImageNet 512, 2-order DPM-Solver | FID: 17.72 -> 16.87 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | U-ViT · ImageNet 512, 2-order DPM-Solver | Precision: 0.728 -> 0.732 | Helped |
| Filter out images with highest uncertainty instead of selecting images randomly | U-ViT · ImageNet 512, 2-order DPM-Solver | Recall: 0.602 -> 0.604 | Helped |
Surprising findings
- FID trends across uncertainty groups differ by dataset: on CelebA diversity dominates FID, while on ImageNet fidelity dominates.
Limitations
- Monte Carlo estimation of covariance requires S (about 10) extra noise-prediction evaluations per step, which is costly without the skip variant.
- The 2-order DPM-Solver requires an approximation via conditional independence rather than the direct derivation.
- Visual resampling success relies partly on human annotation of flawed samples.
Experimental setup
Datasets: ImageNet CelebA
Benchmarks: ImageNet CelebA
Metrics: FID Precision Recall
Baselines: random selection
Builds on: last-layer Laplace approximation DDPM DDIM DPM-Solver Analytic-DPM
Closest papers by meaning
- BUFF: Bayesian Uncertainty Guided Diffusion Probabilistic Model for Single Image Super-Resolution (AAAI 2025 · similarity 0.89)
- EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.89)
- Diffusion Models With Learned Adaptive Noise (NeurIPS 2024 · similarity 0.89)
- PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores (ICLR 2025 · similarity 0.88)
- Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution (NeurIPS 2025 · similarity 0.88)
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.