Research map / Image generation / Editing and style transfer
PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Direct Inversion, a 3-line, optimization-free inversion correction that disentangles source and target diffusion branches for diffusion-based image editing, and introduces the PIE-Bench editing benchmark.
Problem
Diffusion-based image editing requires accurate inversion of a real source image into diffusion latents; existing optimization-based inversion methods (e.g., Null-Text Inversion) are slow and push learned embeddings off the model's input distribution, hurting both content preservation and edit fidel
Approach
Disentangle source and target branches: add the source-branch inversion residual (z*_t - z''_t) back to the source latent only, leaving the target branch unaltered, so no optimization is needed and the model's input distribution is preserved.
Key results
- Direct Inversion with Prompt-to-Prompt reduces structure distance from 13.44 (Null-Text Inversion) to 11.65 x10^-3 while improving PSNR from 27.03 to 27.22 and edited-region CLIP similarity from 21.86 to 22.10.
- Direct Inversion takes 28.17 s per image versus 148.48 s for Null-Text Inversion and 382.98 s for StyleDiffusion.
- Across eight editing methods, Direct Inversion improves background/structure preservation (up to 20.2%) and edit fidelity (up to 2.5%) over their DDIM-inversion baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| disentangle source and target branches by leaving the target branch unaltered (ablation) | Null-Text Inversion · PIE-Bench, P2P | Edited-region CLIP similarity: 21.86 -> 22.11 | Helped |
| add source-branch inversion distance at scale 1 (ablation) | Direct Inversion · PIE-Bench, P2P | Structure Distance ×10³: 13.44 -> 11.65 | Helped |
| increase the added source-distance scale from 0.4 to 0.8 (ablation) | Direct Inversion · PIE-Bench, P2P | Structure Distance ×10³: 13.55 -> 11.90 | Helped |
| add source distance to the target latent (ablation) | Direct Inversion · PIE-Bench, target-latent ablation | Edited-region CLIP similarity: 22.10 -> 21.88 | Hurt |
| add target distance to the target latent (ablation) | Direct Inversion · PIE-Bench, target-latent ablation | Edited-region CLIP similarity: 22.10 -> 20.27 | Hurt |
| use inverse guidance scale 2.5 and forward guidance scale 7.5 (ablation) | Direct Inversion · PIE-Bench, guidance-scale ablation | Edited-region CLIP similarity: 22.10 -> 22.15 | Helped |
| replace DDIM inversion with Direct Inversion | P2P · PIE-Bench, P2P | Background LPIPS ×10³: 208.80 -> 54.55 | Helped |
| replace DDIM inversion with Direct Inversion | MasaCtrl · PIE-Bench, MasaCtrl | Structure Distance ×10³: 28.38 -> 24.70 | Helped |
| replace DDIM inversion with Direct Inversion | P2P-Zero · PIE-Bench, P2P-Zero | Edited-region CLIP similarity: 20.54 -> 21.05 | Helped |
| replace DDIM inversion with Direct Inversion | PnP · PIE-Bench, PnP | Edited-region CLIP similarity: 22.55 -> 22.62 | Helped |
| use Direct Inversion instead of Null-Text Inversion | P2P · PIE-Bench, P2P | Inference time (s): 148.48 -> 28.17 | Helped |
Surprising findings
- Adding the target-prompt inversion residual to the target latent sharply lowers edit CLIP similarity (22.10 to 20.27), showing the target branch should stay unaltered.
- Optimization-based Null-Text Inversion's average residual gap is only partially removed when optimization is truncated, and adding a scaled distance reveals a trade-off between preservation and edit fidelity.
Limitations
- Relies on the chosen editing method's forward process; gains depend on the base editing method and hyperparameters such as guidance scales.
- Benchmark masks and annotations are manually produced by a small team, and metrics depend on those annotations.
Experimental setup
Datasets: PIE-Bench
Benchmarks: PIE-Bench
Metrics: structure distance PSNR LPIPS MSE SSIM CLIP similarity (whole image) CLIP similarity (edited region) inference time
Baselines: DDIM inversion Null-Text Inversion Negative-Prompt Inversion StyleDiffusion Edit Friendly DDPM EDICT Proximal Guidance InstructPix2Pix InstructDiffusion Blended Latent Diffusion
Builds on: DDIM inversion Null-Text Inversion Prompt-to-Prompt Negative-Prompt Inversion StyleDiffusion
Closest papers by meaning
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.92)
- EditInfinity: Image Editing with Binary-Quantized Generative Models (NeurIPS 2025 · similarity 0.91)
- Object-Aware Inversion and Reassembly for Image Editing (ICLR 2024 · similarity 0.91)
- Lightning-Fast Image Inversion and Editing for Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.91)
- h-Edit: Effective and Flexible Diffusion-Based Editing via Doob's h-Transform (CVPR 2025 · similarity 0.90)
- DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.