Research map / Image generation / Fast diffusion sampling
Kernel Density Steering: Inference-Time Scaling via Mode Seeking for Image Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Kernel Density Steering (KDS), a training-free inference-time method that steers an N-particle diffusion ensemble toward patch-wise KDE modes for better image super-resolution and inpainting.
Problem
Diffusion-based image restoration suffers from inconsistent fidelity and artifacts, and the perception-distortion trade-off is hard to balance without retraining or external verifiers or known degradation models.
Approach
Compute patch-wise mean-shift (Gaussian KDE gradient) vectors across an ensemble of predicted clean latents at each sampling step and blend the resulting consensus score into the sampler's update, so particles move toward shared high-density regions; the final output is the particle closest to the ensemble mean.
Key results
- On RealSR with DiffBIR, KDS (N=10) raises PSNR from 23.21 to 24.39 and lowers LPIPS from 0.370 to 0.339 with DDIM (50 steps).
- On ImageNet center-box inpainting with LDM-inpainting, DDIM + KDS (N=10) improves PSNR from 21.03 to 21.35 and LPIPS from 0.140 to 0.131.
- Versus Best-of-N selection at N=10 on RealSR with DiffBIR, KDS gives PSNR 24.39 vs 23.77 for BoN and matched compute (~28s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add KDS to LDM-SR | LDM-SR with DDIM · DIV2K | PSNR: 22.05 -> 22.37 | Helped |
| Add KDS to DiffBIR | DiffBIR with DDIM · RealSR | LPIPS: 0.370 -> 0.339 | Helped |
| Add KDS to SeeSR | SeeSR with DDIM · DrealSR | PSNR: 26.97 -> 27.42 | Helped |
| Add KDS | LDM-inpainting with DPM-Solver · ImageNet inpainting | PSNR: 19.94 -> 21.29 | Helped |
| Use KDS instead of BoN selection | DiffBIR · RealSR | PSNR: 23.77 -> 24.39 | Helped |
| Use HDS instead of KDS | DiffBIR · RealSR | LPIPS: 0.339 -> 0.386 | Hurt |
| Increase patch size from 1 to 4 at N=10 (ablation) | KDS on LDM-SR · DIV2K | PSNR: 22.52 -> 22.35 | Hurt |
| Increase patch size from 1 to 16 at N=10 (ablation) | KDS on LDM-SR · DIV2K | PSNR: 22.52 -> 22.17 | Hurt |
| Replace Gaussian kernel with Laplacian kernel (ablation) | KDS on DiffBIR · RealSR | PSNR: 24.39 -> 24.30 | No clear effect |
| Replace kernel mode-seeking with L2 guidance (ablation) | KDS on DiffBIR · RealSR | LPIPS: 0.339 -> 0.349 | Hurt |
| Use automatic median-heuristic bandwidth instead of bw=0.3 (ablation) | KDS on DiffBIR · RealSR | PSNR: 24.39 -> 24.24 | No clear effect |
| Increase particle count to 40 and patch size to 4 (ablation) | KDS on LDM-SR · DIV2K | PSNR: 22.52 -> 22.70 | Helped |
Surprising findings
- Best-of-N selection with a non-reference metric (LIQE or CLIPIQA) can select artifact-laden particles and underperforms KDS on distortion metrics.
- Automatic median-heuristic bandwidth selection achieves nearly the same performance as manually tuned bandwidth.
Limitations
- Computational overhead scales linearly with the number of particles N.
- Patch-wise mechanism with small patches is needed because KDE is unreliable in high dimensions with few particles.
- Evaluated only on super-resolution and inpainting; other inverse problems and video left to future work.
Experimental setup
Datasets: DIV2K RealSR DRealSR ImageNet DF2K
Benchmarks: DIV2K RealSR DRealSR ImageNet
Metrics: PSNR SSIM LPIPS DISTS FID NIMA MANIQA CLIPIQA
Baselines: LDM-SR DiffBIR SeeSR Best-of-N (LIQE) Best-of-N (CLIPIQA) High Density Sampling (HDS) Density-Guided Sampling (DGS) InDI
Builds on: DDIM DPM-Solver++ Mean Shift Latent Diffusion Models
Closest papers by meaning
- Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Data (ICLR 2025 · similarity 0.87)
- FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.87)
- Scale Where It Matters: Training-Free Localized Scaling for Diffusion Models (NeurIPS 2026 · similarity 0.87)
- A General Framework for Inference-time Scaling and Steering of Diffusion Models (ICML 2025 · similarity 0.87)
- Diffusion Priors for Variational Likelihood Estimation and Image Denoising (NeurIPS 2024 · similarity 0.86)
- Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.