Research map / Image generation / Editing and style transfer
The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a unified probabilistic view of diffusion-based image editing, proves KL between SDE marginals contracts while ODE marginals stay invariant, and builds SDE counterparts of ODE editing baselines plus SDE-Drag and the DragBench benchmark.
Problem
Diffusion-based image editing modifies a latent variable so it deviates from the marginal distribution of the sampling SDE/ODE, and prior work lacks a principled understanding of how the SDE vs. ODE choice affects editing quality.
Approach
Unify editing methods as edit-then-resample pipelines and prove that, under distribution mismatch at time t0, the KL divergence between SDE marginals contracts over time while ODE marginals do not; use this to design SDE counterparts of ODE editing baselines and a copy-and-paste SDE-Drag method.
Key results
- Inpaint-SDE at 25 steps achieves FID 5.18 on small masks versus 5.84 for Inpaint-ODE at 25 steps, and matches or beats ODE at 100 steps.
- In a user study on DragBench with 6 participants, SDE-Drag is preferred over ODE-Drag, DragDiffusion, and DragGAN with significant margins.
- Cycle-SDE with double precision reconstructs images at PSNR 24.28 under CFG 7.5, versus 16.97 for DDIM inversion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace DDIM with DDPM sampling for inpainting | Inpaint-SDE versus Inpaint-ODE · Places, small and large masks; 25, 50, and 100 steps | FID: 5.84 -> 5.18; 5.86 -> 4.97; 5.86 -> 4.91; 17.73 -> 16.43; 17.57 -> 15.48; 17.50 -> 15.27 | Helped |
| Replace DDIM with DDPM sampling for inpainting | Inpaint-SDE versus Inpaint-ODE · Places, small and large masks; 25, 50, and 100 steps | LPIPS: 0.227 -> 0.218; 0.227 -> 0.215; 0.227 -> 0.215; 0.363 -> 0.351; 0.361 -> 0.345; 0.361 -> 0.343 | Helped |
| Use Cycle-SDE instead of the editing ODE formulation | DiffEdit-SDE versus DiffEdit-ODE · COCO image-to-image translation | FID | Helped |
| Use Cycle-SDE instead of the editing ODE formulation | DiffEdit-SDE versus DiffEdit-ODE · COCO image-to-image translation | LPIPS | Helped |
| Use Cycle-SDE instead of the editing ODE formulation | DiffEdit-SDE versus DiffEdit-ODE · COCO image-to-image translation | CLIP-Score | Helped |
| Use SDE-Drag instead of ODE-Drag | Diffusion-based point dragging · DragBench | user preference | Helped |
| Use SDE-Drag instead of DragDiffusion | Diffusion-based point dragging · DragBench | user preference | Helped |
| Use SDE-Drag instead of DragGAN | Point dragging · DragBench eligible domains | user preference | Helped |
| Use double precision instead of Float32 (ablation) | Cycle-SDE image reconstruction · COCO validation, CFG scale 7.5 | PSNR: 11.83 -> 24.28 | Helped |
| Use double precision instead of Float32 (ablation) | DDIM inversion image reconstruction · COCO validation, CFG scale 7.5 | PSNR: 11.86 -> 16.97 | Helped |
| Increase t0 from 0.4T toward 0.8T (ablation) | SDE-Drag · Dragging image fidelity and faithfulness | fidelity | Helped |
| Increase t0 from 0.4T toward 0.8T (ablation) | SDE-Drag · Dragging image fidelity and faithfulness | faithfulness | Hurt |
Surprising findings
- Cycle-SDE reconstruction fails under CFG in single precision (PSNR 11.83), due to numerical instability rather than discretization error.
- SDE at 25 steps surpasses ODE at 100 steps in inpainting, reversing the usual advantage of ODE solvers at few steps.
Limitations
- Theory ignores model approximation and discretization errors.
- No convergence rate is obtained for Cycle-SDE.
- Open-set dragging remains far from solved, with failure cases noted.
Experimental setup
Datasets: Places COCO COCO-BISON LAION-5B
Benchmarks: Places COCO DragBench
Metrics: FID LPIPS CLIP-Score PSNR user preference rate
Baselines: ODE-Drag DragDiffusion DragGAN DiffEdit-ODE Inpaint-ODE
Builds on: DDPM DDIM DiffEdit DragDiffusion DragGAN SDEdit CycleDiffusion
Closest papers by meaning
- Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation (CVPR 2024 · similarity 0.89)
- DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing (CVPR 2024 · similarity 0.88)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.87)
- DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models (ICLR 2024 · similarity 0.87)
- Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimators (ICLR 2024 · similarity 0.87)
- Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.