Research map / Video / Video segmentation and tracking
FreeInv: Free Lunch for Improving DDIM Inversion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreeInv, a training-free DDIM inversion enhancement that applies the same random transformation (e.g., rotation) to the latent at each time-step during inversion and reconstruction, approximating a multi-trajectory ensemble that reduces trajectory mismatch at negligible cost.
Problem
DDIM inversion suffers from trajectory deviation: the reconstruction latent trajectory drifts from the inversion trajectory, because the local linear assumption on the noise predictor does not hold, degrading reconstruction and editing fidelity.
Approach
Motivated by a statistical argument that averaging noise predictions over an ensemble of trajectories reduces expected mismatch error, FreeInv samples one random transformation per time-step (shared between the matching inversion and reconstruction steps), so that each step implicitly uses one branch of a multi-branch ensemble at no extra cost.
Key results
- On PIE-benchmark with SD1.5 U-Net, FreeInv reaches PSNR 27.69 vs 25.04 for DDIM baseline at about the same time (4 s) and memory (3031 MB) as DDIM.
- With FLUX, FreeInv raises PSNR from 14.92 (FLUX baseline) to 29.24, exceeding RF-Solver (26.38) at 7 s vs 15 s.
- In video reconstruction with TokenFlow, FreeInv adds about 2 MB GPU memory and gives the highest PSNR among DDIM, STEM-Inv and FreeInv.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FreeInv to DDIM inversion | Stable Diffusion 1.5 U-Net · PIE benchmark, reconstruction | PSNR: 25.04 -> 27.69 | Helped |
| Add FreeInv to FLUX | FLUX DiT · PIE benchmark, reconstruction | PSNR: 26.38 -> 29.24 | Helped |
| Add FreeInv to DDIM inversion | Stable Diffusion 1.5 with P2P · PIE benchmark, P2P editing | CLIP Similarity, Whole: 25.18 -> 25.30 | No clear effect |
| Use one MC sample per time-step instead of four-branch MBDI (ablation) | FreeInv multi-branch sampling · PIE benchmark, reconstruction | PSNR: 28.14 -> 28.13 | No clear effect |
| Increase MC sampling from one time to two or four times (ablation) | FreeInv sampling · PIE benchmark, reconstruction | PSNR: 28.13 -> 28.14 | No clear effect |
| Replace distinct-image branches with random rotation at each time-step (ablation) | Multi-branch DDIM inversion · PIE benchmark, reconstruction | PSNR: 28.14 -> 27.64 | Hurt |
| Replace rotated-image branches with random rotation at each time-step (ablation) | Multi-branch DDIM inversion · PIE benchmark, reconstruction | PSNR: 27.73 -> 27.64 | Hurt |
| Use random flipping, patch shuffle, or color jitter instead of rotation (ablation) | FreeInv · PIE benchmark, reconstruction | PSNR: 27.47; 27.61; 27.53; rotation 27.64 | Mixed |
| Combine transformation types instead of using rotation alone (ablation) | FreeInv · PIE benchmark, reconstruction | PSNR: 27.64 -> 27.69 | No clear effect |
| Apply inverse transformation to predicted noise (ablation) | FreeInv · PIE benchmark, reconstruction | PSNR: 27.64 -> 27.64 | No clear effect |
| Add FreeInv to NTI (ablation) | NTI · PIE benchmark, reconstruction | PSNR: 26.74 -> 28.15 | Helped |
| Add FreeInv to DDIM inversion (ablation) | SDXL · PIE benchmark, reconstruction | PSNR: 24.78 -> 26.68 | Helped |
Surprising findings
- One-time Monte Carlo sampling per time-step performs on par with multi-time sampling and full multi-branch DDIM inversion (PSNR 28.13 vs 28.14), suggesting a single random branch per step suffices.
- Rotating the latent for the noise prediction without applying the inverse transformation to the noise yields the same reconstruction quality, so the inverse step is only needed for editing structure.
Limitations
- Can overemphasize source preservation, limiting editability.
- Editing quality depends on the integrated editing framework rather than on FreeInv itself.
- Gains in editing are modest compared with reconstruction; some baselines (e.g., NTI, PI) achieve comparable editing alignment.
Experimental setup
Datasets: PIE-benchmark DAVIS
Benchmarks: PIE-benchmark DAVIS
Metrics: PSNR LPIPS MSE SSIM time memory structure distance background preservation CLIP similarity user preference
Baselines: DDIM inversion Null-text Inversion EDICT DDPM Inversion Virtual Inversion PnP Inversion ReNoise BELM STEM Inversion RF-Solver
Builds on: DDIM Null-text Inversion PnP Inversion TokenFlow Prompt-to-Prompt
Closest papers by meaning
- EasyInv: Toward Fast and Better DDIM Inversion (ICML 2025 · similarity 0.91)
- Exploring Fixed Point in Image Editing: Theoretical Support and Convergence Optimization (NeurIPS 2024 · similarity 0.87)
- A Video is Worth 256 Bases: Spatial-Temporal Expectation-Maximization Inversion for Zero-Shot Video Editing (CVPR 2024 · similarity 0.86)
- Tuning-Free Inversion-Enhanced Control for Consistent Image Editing (AAAI 2024 · similarity 0.86)
- DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing (NeurIPS 2025 · similarity 0.85)
- Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applications (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.