Research map / Image generation / Editing and style transfer
Exploring Low-Dimensional Subspace in Diffusion Models for Controllable Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that the posterior mean predictor of diffusion models is locally linear with a low-rank Jacobian at mid noise levels, and uses its singular vectors for unsupervised, single-step, training-free localized image editing (LOCO Edit) extended to text-to-image models (T-LOCO Edit).
Problem
Diffusion models lack a clear understanding of their semantic latent spaces, so precise, localized, disentangled editing usually needs extra training, text supervision, or heuristics.
Approach
Within a range of noise levels the posterior mean predictor is locally linear and its Jacobian is low-rank, so the top singular vectors of the (masked, nullspace-projected) Jacobian give single-step editing directions that are linear, homogeneous, transferable and composable; a mixture-of-low-rank-Gaussians analysis supports this.
Key results
- LOCO Edit achieves a Local Edit Success Rate of 0.80 versus 0.55 for BlendedDiffusion and 0.47 for Asyrp in the comparison table.
- LOCO Edit reaches a Transfer Success Rate of 0.91, the highest among compared methods (NoiseCLR 0.66, Asyrp 0.58).
- LOCO Edit learning takes 79s on a single image and single timestep, versus 120s for BlendedDiffusion and 475s for Asyrp.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LOCO Edit instead of BlendedDiffusion | localized image editing · Local image editing | Local Edit Success Rate: 0.55 -> 0.80 | Helped |
| use LOCO Edit instead of BlendedDiffusion | localized image editing · Local image editing | LPIPS: 0.03 -> 0.08 | Hurt |
| use LOCO Edit instead of BlendedDiffusion | localized image editing · Local image editing | SSIM: 0.94 -> 0.71 | Hurt |
| use LOCO Edit instead of NoiseCLR | image editing transfer · Image editing transfer | Transfer Success Rate: 0.66 -> 0.91 | Helped |
| remove nullspace projection (ablation) | LOCO Edit · Localized image editing | Local edit ability | Hurt |
| use nullspace projection with r′ = 5 (ablation) | LOCO Edit · Localized image editing | Local edit ability | Helped |
| increase editing timestep (ablation) | LOCO Edit · Image editing at different noise levels | Edit coarseness | Helped |
| decrease editing timestep (ablation) | LOCO Edit · Image editing at different noise levels | Edit coarseness | Hurt |
| vary editing strength within [-15, 15] (ablation) | LOCO Edit · Localized image editing | Edit strength linearity: [-15, 15] | Helped |
Surprising findings
- Transferring an editing direction to an image lacking the corresponding semantics (e.g., eyes) produces almost no edit, so transfer works only when the semantic exists in the target.
Limitations
- Theory relies on a mixture-of-low-rank-Gaussians assumption and does not fully cover text-supervised editing.
- Jacobian is only low-rank and linear within a range of noise levels (roughly t in [0.2, 0.7]).
- Transfer only works when the target image contains the corresponding semantics.
Experimental setup
Datasets: CIFAR-10 CelebA ImageNet LAION-5B LSUN-Church Flower AFHQ CelebA-HQ FFHQ MetFaces
Benchmarks: LSUN-Church Flower AFHQ CelebA-HQ FFHQ
Metrics: Local Edit Success Rate LPIPS SSIM Transfer Success Rate Learning Time Transfer Edit Time
Baselines: Asyrp Pullback NoiseCLR BlendedDiffusion dε/dx Jacobian baseline
Builds on: DDIM Asyrp Pullback classifier-free guidance
Closest papers by meaning
- Unsupervised Region-Based Image Editing of Denoising Diffusion Models (AAAI 2025 · similarity 0.93)
- NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models (CVPR 2024 · similarity 0.91)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.90)
- EditInfinity: Image Editing with Binary-Quantized Generative Models (NeurIPS 2025 · similarity 0.89)
- Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation (CVPR 2024 · similarity 0.89)
- Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.