Research map / Image generation / Fast diffusion sampling
Repulsive Latent Score Distillation for Solving Inverse Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Repulsive Latent Score Distillation (RLSD), a particle-based variational sampler with kernel repulsion and an augmented pixel/latent variational distribution for solving linear and nonlinear inverse problems with Stable Diffusion at 512x512.
Problem
Score distillation based posterior sampling for inverse problems with latent diffusion models suffers from mode collapse (unimodal Gaussian variational approximation) and blurry/inverted results from latent-space optimization due to autoencoder compression, at high resolution.
Approach
Use an interacting particle ensemble with a kernel-based repulsive force (RBF on DINO features) inside the Wasserstein gradient flow of score distillation to promote diversity, plus an augmented variational variable in pixel space coupled to the latent via a half-quadratic splitting-style two-step optimization to avoid latent inversion artifacts.
Key results
- On half-face inpainting, RLSD (gamma=50) reaches 24.69 dB PSNR, 31.41 FID and diversity 0.015 versus PSLD's 22.72 dB, 57.7 FID and diversity 0.03.
- On phase retrieval, NonAug-RLSD (gamma=30) achieves 24.21 dB PSNR and 130.09 FID versus Latent-DPS 14.98 dB and 291.68 FID.
- On HDR, NonRepuls-RLSD (no repulsion) reaches 27.10 dB PSNR and 38.57 FID, outperforming Latent-RED-diff at 25.68 dB and 93.71 FID.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add repulsion (γ = 50) (ablation) | RLSD with augmentation · Inpainting half face | Diversity: 0.004 -> 0.015 | Helped |
| remove repulsion (γ = 0) (ablation) | RLSD with augmentation · Inpainting half face | PSNR [dB]: 24.69 -> 24.98 | Helped |
| use hybrid interaction: three particles interact and one propagates independently (ablation) | RLSD · Inpainting half face | PSNR [dB]: 24.69 -> 24.72 | Mixed |
| remove distribution augmentation while retaining repulsion (ablation) | RLSD · Phase retrieval | PSNR [dB]: 20.43 -> 24.21 | Helped |
| add repulsion to the augmented method (ablation) | RLSD · Phase retrieval | PSNR [dB]: 18.33 -> 20.43 | Helped |
| add repulsion (γ = 30) (ablation) | RLSD with augmentation · HDR | PSNR [dB]: 27.10 -> 25.53 | Hurt |
| use augmented formulation without repulsion (ablation) | RLSD · Super resolution ×8, FFHQ 512 | PSNR [dB] | Helped |
| use augmented formulation without repulsion (ablation) | RLSD · Motion deblurring, FFHQ 512 | PSNR [dB] | Helped |
| use augmented formulation without repulsion (ablation) | RLSD · Random inpainting 80%, FFHQ 512 | PSNR [dB] | Helped |
| use augmented formulation without repulsion (ablation) | RLSD · Free-mask inpainting, ImageNet 512 | PSNR [dB] | Helped |
Surprising findings
- The augmented formulation is unstable on nonlinear phase retrieval, where the non-augmented variant performs better.
- Without repulsion, the augmented variant (NonRepuls-RLSD) has very low diversity (0.004) on inpainting, so quality and diversity trade off sharply.
Limitations
- Repulsion term increases computational cost, complicating real-time use.
- The RBF repulsion kernel may be suboptimal at high noise levels and adds hyperparameters.
- Augmented formulation is unstable for nonlinear problems such as phase retrieval.
Experimental setup
Datasets: FFHQ ImageNet
Benchmarks: FFHQ 512 inpainting (half face) phase retrieval HDR super-resolution x8 motion deblurring random inpainting 80% ImageNet free-mask inpainting
Metrics: PSNR LPIPS FID diversity (1 - cosine similarity of DINO features)
Baselines: PSLD Latent RED-Diff RED-Diff (pixel) DPS (pixel) Latent DPS ProlificDreamer
Builds on: Score Distillation Sampling Stein variational gradient descent RED-Diff PSLD DPS
Closest papers by meaning
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.90)
- Noise Conditional Variational Score Distillation (ICML 2025 · similarity 0.89)
- Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction (NeurIPS 2024 · similarity 0.89)
- SITCOM: Step-wise Triple-Consistent Diffusion Sampling For Inverse Problems (ICML 2025 · similarity 0.88)
- Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency (ICLR 2024 · similarity 0.88)
- Solving Inverse Problems with FLAIR (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.