Research map / Image generation / Image restoration
From Posterior Sampling to Meaningful Diversity in Image Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues that posterior sampling in image restoration yields redundant, heavy-tailed outputs and proposes meaningful-diversity methods (farthest point sampling post-processing and a diversity-guidance step for diffusion restorers) validated with user studies.
Problem
Diverse image restoration via posterior sampling produces small sets dominated by near-identical, high-probability solutions, failing to convey semantically different plausible restorations within a small number of outputs.
Approach
Instead of sampling from the posterior, sub-sample a large pool of restorations with farthest point sampling in a semantic feature space (and push diffusion predictions apart with a nearest-neighbor dissimilarity guidance step) to obtain small sets covering distinct semantic modes.
Key results
- In user studies, the diversity of posterior sampling was preferred significantly less often than FPS, Uniformization and K-means subsets, with FPS judged most diverse.
- Diversity guidance raised LPIPS diversity for DPS on CelebAMask-HQ noisy 16x super-resolution from 0.29 to 0.34 while NIQE improved from 5.72 to 5.16.
- For inpainting on CelebAMask-HQ, RePaint LPIPS diversity rose from 0.08 to 0.09 with guidance, with NIQE essentially unchanged (5.07 to 5.05).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add diversity guidance to DPS | DPS · CelebAMask-HQ noisy 16× super-resolution (σ=0.05) | LPIPS Div.: 0.29 -> 0.34 | Helped |
| Add diversity guidance to DPS | DPS · CelebAMask-HQ noiseless 16× super-resolution (σ=0) | LPIPS Div.: 0.25 -> 0.28 | Helped |
| Add diversity guidance to DPS | DPS · PartImageNet noisy 4× super-resolution (σ=0.05) | LPIPS Div.: 0.31 -> 0.33 | Helped |
| Add diversity guidance to DPS | DPS · PartImageNet noiseless 4× super-resolution (σ=0) | LPIPS Div.: 0.33 -> 0.40 | Helped |
| Add diversity guidance to RePaint | RePaint · CelebAMask-HQ inpainting | LPIPS Div.: 0.08 -> 0.09 | Helped |
| Add diversity guidance to DPS | DPS · PartImageNet inpainting | LPIPS Div.: 0.10 -> 0.11 | Helped |
| Increase guidance step size η from 0 to 0.3 (ablation) | RePaint · CelebAMask-HQ inpainting | LPIPS Div.: 0.090 -> 0.113 | Helped |
| Increase guidance distance bound D from 0 to ∞ (ablation) | RePaint · CelebAMask-HQ inpainting | LPIPS Div.: 0.090 -> 0.141 | Helped |
| Add diversity guidance to DPS | DPS · PartImageNet noiseless 4× super-resolution (σ=0) | NIQE: 15.27 -> 20.48 | Hurt |
Surprising findings
- Roughly 12% of inspected masked face images had posterior kurtosis above 5, indicating heavy-tailed posteriors are common.
- Uniformization yielded low coverage of likely solutions and omitted the dominant mode in the toy and image experiments.
Limitations
- Guidance step size and distance bound must be tuned per method and domain.
- Some diffusion baselines (e.g., DPS) are inconsistent with inputs unless carefully tuned, and some configurations are missing from source code (N/A entries).
- Diversity is measured with LPIPS, a proxy that may not match semantic diversity; user studies use a limited set of 50 face images per task.
Experimental setup
Datasets: CelebAMask-HQ PartImageNet
Benchmarks: CelebAMask-HQ PartImageNet
Metrics: LPIPS diversity NIQE LR-PSNR user preference rate for diversity user preference rate for coverage
Baselines: RePaint DDRM DDNM DPS MAT K-means Uniformization
Builds on: RePaint DDRM DDNM DPS farthest point sampling
Closest papers by meaning
- Consistency Posterior Sampling for Diverse Image Synthesis (CVPR 2025 · similarity 0.87)
- Blind Image Restoration via Fast Diffusion Inversion (NeurIPS 2024 · similarity 0.86)
- DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restoration (NeurIPS 2025 · similarity 0.86)
- GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration (CVPR 2025 · similarity 0.85)
- Restoration by Generation with Constrained Priors (CVPR 2024 · similarity 0.85)
- Improving Diffusion Posterior Samplers with Lagged Temporal Corrections for Image Restoration (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.