Research map / Image generation / Editing and style transfer
Noise Map Guidance: Inversion with Spatial Context for Real Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Noise Map Guidance (NMG), an optimization-free DDIM-inversion-based real-image editing method that conditions the reverse diffusion process on noise maps (via energy guidance) plus text, preserving spatial context.
Problem
Editing real images with text-guided diffusion models: classifier-free guidance pushes the reconstruction path off the DDIM inversion trajectory, degrading reconstruction and edit fidelity. Existing fixes (Null-text Inversion) need costly per-timestep optimization, and optimization-free ones (NPI, P
Approach
Treat DDIM-inversion latents (noise maps) as an energy-based condition: an L1 energy between the one-step denoised latent and the noise map guides the reverse process, with a separate guidance scale for noise map and text, yielding spatially faithful reconstruction without optimization.
Key results
- On Prompt-to-Prompt global editing, NMG reaches TIFA 0.8991 vs 0.8302 for NTI and 0.8463 for ProxNPI, with CLIP 0.3221.
- In a 50-participant user study, NMG was selected in 72.5% of comparisons versus 10.0% for NTI and 12.5% for ProxNPI.
- Reconstruction takes 5.97 s for NMG versus 104.85 s for NTI on MS-COCO, with comparable MSE (0.0124 vs 0.0127).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NMG instead of NTI | Prompt-to-Prompt local editing · Prompt-to-Prompt local | CLIP: 0.2983 -> 0.3007 | Helped |
| Use NMG instead of NTI | Prompt-to-Prompt local editing · Prompt-to-Prompt local | TIFA: 0.9125 -> 0.8955 | Hurt |
| Use NMG instead of NTI | Prompt-to-Prompt global editing · Prompt-to-Prompt global | TIFA: 0.8302 -> 0.8991 | Helped |
| Use NMG instead of ProxNPI | MasaCtrl editing · MasaCtrl | TIFA: 0.8188 -> 0.8548 | Helped |
| Use NMG instead of ProxNPI | image editing · User study | selection ratio: 12.5% -> 72.5% | Helped |
| Add NMG to NTI (ablation) | reconstruction · MS-COCO validation set | SSIM: 0.7292 -> 0.7302 | Helped |
| Add NMG to NTI (ablation) | reconstruction · MS-COCO validation set | Time: 104.85 -> 107.72 | Hurt |
| Reverse the conditioning order, using text before noise map (ablation) | NMG · local editing | editing quality | No clear effect |
| Replace the L1 energy distance with L2 (ablation) | NMG · image editing | background quality | Hurt |
| Increase text guidance relative to noise map guidance (ablation) | NMG · image editing | preservation of input image nuance | Hurt |
| Increase noise map guidance excessively (ablation) | NMG · image editing | editing success | Hurt |
| Use balanced noise map and text guidance scales (ablation) | NMG · image editing | editing quality | Helped |
Surprising findings
- NMG reconstruction is comparable in MSE to NTI but has a slightly worse LPIPS (0.1673 vs 0.1588), and the authors note high reconstruction quality does not guarantee better editing.
Limitations
- Inherits limitations of inversion-based editing, e.g. Prompt-to-Prompt cannot make structural changes beyond the original object's structure.
- Text dependence makes precise spatial edits (removing a specific individual, placing objects at a given location) difficult.
- Not compatible with non-inversion-based editing methods such as SGC-Net for relationship changes.
Experimental setup
Datasets: MS-COCO validation set
Benchmarks: MS-COCO val (reconstruction, 100 images)
Metrics: CLIPScore TIFA MSE SSIM LPIPS runtime user study selection rate
Baselines: DDIM inversion Null-text Inversion Negative-prompt Inversion ProxNPI ProxMasaCtrl
Builds on: Null-text Inversion Prompt-to-Prompt EGSDE energy guidance classifier-free guidance DDIM inversion
Closest papers by meaning
- Editable Noise Map Inversion: Encoding Target-image into Noise For High-Fidelity Image Manipulation (ICML 2025 · similarity 0.93)
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.91)
- Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editing (NeurIPS 2024 · similarity 0.91)
- An Edit Friendly DDPM Noise Space: Inversion and Manipulations (CVPR 2024 · similarity 0.90)
- DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing (NeurIPS 2025 · similarity 0.89)
- PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.