Research map / Image generation / Fast diffusion sampling
Stochastic Forward–Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows theoretically that learning a diffusion model from purely noisy samples converges very slowly, and proposes SFBD, which pretrains on a few clean images and then iteratively denoises the noisy set with the current denoiser and fine-tunes on the result.
Problem
Training diffusion models only on Gaussian-corrupted (noisy) samples, to avoid memorization and copyright exposure of the original data, is statistically very hard: the optimal MISE rate is only O((log n)^-2), so learning from noisy data alone is nearly infeasible in practice.
Approach
Pretrain the denoiser on a small clean set, then alternate backward SDE sampling of noisy samples back to clean estimates with denoiser fine-tuning on those estimates (a forward-backward deconvolution loop), which provably reduces KL between the learned and true data distributions and avoids the costly consistency loss.
Key results
- SFBD with 4% clean CIFAR-10 images reaches FID 6.31 (3.58 with 10% clean), while the consistency-based TweedieDiff reaches 167.23 without pretraining and 65.21 with pretraining at sigma 0.2.
- On CelebA at sigma 0.2 with 50 clean images, SFBD reaches FID 6.49, comparable to DDIM trained on clean data (6.53).
- Theory: optimal MISE rate for noisy-only density estimation is O((log n)^-2), so reducing MISE by a factor of four needs roughly n^2 extra samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SFBD instead of TweedieDiff with 50 clean pretraining images | Diffusion training on noisy CIFAR-10 · CIFAR-10, σζ=0.2 | FID: 65.21 -> 13.53 | Helped |
| Use SFBD instead of TweedieDiff with 50 clean pretraining images | Diffusion training on noisy CelebA · CelebA, σζ=0.2 | FID: 58.52 -> 6.49 | Helped |
| Add clean pretraining to TweedieDiff (ablation) | TweedieDiff on CIFAR-10 · CIFAR-10, σζ=0.2 | FID: 167.23 -> 65.21 | Helped |
| Add clean pretraining to TweedieDiff (ablation) | TweedieDiff on CelebA · CelebA, σζ=0.2 | FID: 246.95 -> 58.52 | Helped |
| Increase clean image ratio from 4% to 10% (ablation) | SFBD · CIFAR-10, σζ=0.59 | FID: 6.31 -> 3.58 | Helped |
| Increase clean image ratio from 10% to 20% (ablation) | SFBD · CIFAR-10, σζ=0.59 | FID: 3.58 -> 2.98 | Helped |
| Increase noise level from σζ=0.30 to σζ=0.59 (ablation) | SFBD with 4% clean images · CIFAR-10 | FID: 3.97 -> 6.31 | Hurt |
| Increase noise level from σζ=0.59 to σζ=1.09 (ablation) | SFBD with 4% clean images · CIFAR-10 | FID: 6.31 -> 9.43 | Hurt |
| Pretrain on truck images rather than no pretraining (ablation) | SFBD fine-tuned on CIFAR-10 classes · CIFAR-10 class fine-tuning | FID: 155.04 -> 10.39 | Helped |
| Increase clean CelebA pretraining images from 50 to 1,500 (ablation) | SFBD at σζ=1.38 · CelebA | FID: 23.63 -> 5.91 | Helped |
| Continue SFBD fine-tuning from iteration 1 to iteration 3 (ablation) | SFBD with 4% clean images and σ=0.30 · CIFAR-10 | FID: 6.16 -> 2.68 | Helped |
| Continue SFBD fine-tuning from iteration 3 to iteration 4 (ablation) | SFBD with 4% clean images and σ=0.59 · CIFAR-10 | FID: 6.31 -> 6.54 | Hurt |
Surprising findings
- Higher-quality results come from SFBD alone than from TweedieDiff+ only when more clean data is used; TweedieDiff+ outperforms SFBD with very few clean images (CIFAR-10 FID 8.05 vs 13.53 at sigma 0.2).
- Denoising noisy data with a classical denoiser (Restormer) yields much worse FID (99.99 at sigma 0.59) than SFBD's iterative procedure, which is not simply denoising.
Limitations
- Performance degrades substantially as the noise level sigma increases (e.g., FID 10.91 at sigma 1.92 on CIFAR-10), and the theory shows higher sigma needs far more samples.
- Theoretical guarantees assume infinitely many noisy samples and rely on mild smoothness assumptions; convergence of characteristic functions is argued heuristically at large frequencies.
- SFBD does not protect the clean pretraining data from leakage and relies on the assumption that such data is public and copyright-free.
Experimental setup
Datasets: CIFAR-10 CelebA
Benchmarks: CIFAR-10 CelebA
Metrics: FID MISE
Baselines: DDPM DDIM EDM SURE-Score EMDiff TweedieDiff TweedieDiff+ Restormer
Builds on: EDM Tweedie consistency loss (TweedieDiff) Ambient diffusion DDIM preprocessing
Closest papers by meaning
- Ambient Diffusion Omni: Training Good Models with Bad Data (NeurIPS 2025 · similarity 0.89)
- Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data (ICML 2024 · similarity 0.89)
- Slight Corruption in Pre-training Data Makes Better Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Fast constrained sampling in pre-trained diffusion models (NeurIPS 2025 · similarity 0.88)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.88)
- Sparsely Supervised Diffusion (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.