Research map / Image generation / Editing and style transfer
Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes zero-shot inversion and editing for rectified flow models (e.g., Flux) via a controlled ODE derived from a linear quadratic regulator, with an equivalent rectified SDE, requiring no training, latent optimization, or attention processors.
Problem
Inverting real or corrupted images with rectified flow models such as Flux to achieve faithful reconstruction and editability, which existing diffusion inversion methods handle only with costly optimization or extra training.
Approach
Derives an optimal controller via LQR that steers the reverse rectified flow toward a reference image, interpolating with the unconditional flow by a controller guidance parameter; proves the resulting controlled ODE is equivalent to a rectified SDE, giving a stochastic sampler for Flux.
Key results
- On LSUN Bedroom stroke-to-image, the method is 4.7% more faithful than SDEdit-SD1.5 and 13.79% more realistic (KID 0.025 vs 0.029 for SDEdit-SD1.5 and 0.095 for NTI).
- Large-scale MTurk user study (9,000 comparisons, 126 participants) shows users prefer the method over all baselines by at least 59.67% in overall satisfaction.
- On SFHQ face editing with 'wearing glasses', the method matches NTI+P2P in face recognition (0.442 vs 0.443) and improves CLIP-I to 0.900 vs 0.845, with 54.11% runtime gain over NTI+P2P as claimed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use controlled rectified-flow inversion instead of SDEdit-SD1.5 | Stroke2Image generation · LSUN Bedroom | KID: 0.029 -> 0.025; 13.79% more realistic | Helped |
| Use controlled rectified-flow inversion instead of SDEdit-SD1.5 | Stroke2Image generation · LSUN Bedroom | L2: 86.72 -> 82.65; 4.7% more faithful | Helped |
| Use controlled rectified-flow inversion instead of NTI | Stroke2Image generation · LSUN Bedroom | KID: 0.095 -> 0.025; 73% more realistic | Helped |
| Add P2P to NTI | Stroke2Image generation · LSUN Bedroom | KID: 0.095 -> 0.234 | Hurt |
| Use controlled rectified-flow inversion instead of SDEdit-Flux | Stroke2Image generation · LSUN Church | KID: 0.081 -> 0.059 | Helped |
| Use controlled rectified-flow inversion instead of SDEdit-Flux | Stroke2Image generation · LSUN Church | L2: 92.47 -> 80.36 | Helped |
| Use controlled rectified-flow inversion instead of SDEdit-Flux | Face editing · SFHQ, wearing glasses | Face Rec.: 0.632 -> 0.442 | Helped |
| Use controlled rectified-flow inversion instead of SDEdit-Flux | Face editing · SFHQ, wearing glasses | DINO: 0.892 -> 0.951 | Helped |
| Use controlled rectified-flow inversion instead of SDEdit-Flux | Face editing · SFHQ, wearing glasses | CLIP-I: 0.710 -> 0.900 | Helped |
| Use controlled rectified-flow inversion instead of SDEdit-Flux | Face editing · SFHQ, wearing glasses | CLIP-T | No clear effect |
| Use controlled rectified-flow inversion instead of NTI+P2P | Face editing · SFHQ, wearing glasses | Face Rec.: 0.443 -> 0.442 | No clear effect |
| Use controlled rectified-flow inversion instead of NTI+P2P | Face editing · SFHQ, wearing glasses | DINO: 0.953 -> 0.951 | No clear effect |
Surprising findings
- NTI+P2P achieves the lowest L2 on stroke-to-image but produces unrealistic images, showing faithfulness to a corrupted input can hurt realism.
Limitations
- Lacks comparison with expensive diffusion-based editing methods, which are unavailable or not applicable to Flux's MM-DiT architecture.
- Evaluation on stroke-to-image and face editing covers limited datasets and tasks; stylization results are mainly qualitative.
- Uses a larger (~12B) backbone, so runtime comparisons depend on model size.
Experimental setup
Datasets: LSUN-Bedroom LSUN-Church SFHQ
Benchmarks: LSUN-Bedroom LSUN-Church SFHQ
Metrics: L2 KID user preference (%) face recognition L2 DINO CLIP-T CLIP-I runtime (s)
Baselines: SDEdit-SD1.5 SDEdit-Flux DDIM inversion NTI NTI+P2P
Builds on: rectified flow SDEdit DDIM inversion null-text inversion prompt-to-prompt
Closest papers by meaning
- Taming Rectified Flow for Inversion and Editing (ICML 2025 · similarity 0.92)
- Text-to-Image Rectified Flow as Plug-and-Play Priors (ICLR 2025 · similarity 0.90)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.90)
- FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing (ICML 2025 · similarity 0.90)
- DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing (NeurIPS 2025 · similarity 0.90)
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.