Research map / Image generation / Editing and style transfer
Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes invertible Consistency Distillation (iCD), a multi-boundary forward/reverse consistency distillation framework with dynamic CFG that gives few-step (3-4) image generation and accurate inversion, enabling fast zero-shot text-guided editing of SD1.5 and SDXL.
Problem
Distilled few-step text-to-image diffusion models lack real image inversion, which is needed for inversion-based precise image editing; classifier-free guidance further worsens inversion.
Approach
Train a forward consistency model (fCD) that maps trajectory points to latent noise alongside a reverse CD model, both split into multiple boundary segments for deterministic multistep inversion, add forward/reverse preservation losses, and apply step-function dynamic CFG (guidance off at high noise) to improve inversion without extra compute.
Key results
- iCD-SD1.5 with dynamic CFG (K config) reaches DinoV2 0.749 and PSNR 19.66 in guided decoding, close to the unguided teacher inversion (0.834 / 22.98).
- On COCO editing, iCD-XL with 8 steps gets human preference of 91% vs ReNoise SDXL at 150 steps and 95% vs ReNoise LCM at 35 steps.
- iCD-SD1.5 inverts an image in 0.959 s versus 116.4 s for NTI.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase the number of forward and reverse steps from 2 to 4 (ablation) | iCD-SD1.5 · MS-COCO inversion, unguided decoding | LPIPS: 0.332 -> 0.276 | Helped |
| add the forward preservation loss (ablation) | iCD-SD1.5 with 4-step fCD/CD · MS-COCO inversion, unguided decoding | LPIPS: 0.276 -> 0.248 | Helped |
| add the reverse preservation loss alongside the forward preservation loss (ablation) | iCD-SD1.5 with 4-step fCD/CD · MS-COCO inversion, unguided decoding | LPIPS: 0.248 -> 0.198 | Helped |
| remove the forward consistency loss and train the encoder with forward preservation loss only (ablation) | iCD-SD1.5 with 4-step fCD/CD · MS-COCO inversion, unguided decoding | LPIPS: 0.198 -> 0.484 | Hurt |
| add dynamic CFG to guided decoding (ablation) | iCD-SD1.5, CFG scale 8 · MS-COCO inversion, guided decoding | LPIPS: 0.476 -> 0.370 | Helped |
| add forward and reverse preservation losses to dynamic-CFG decoding (ablation) | iCD-SD1.5, CFG scale 8 · MS-COCO inversion, guided decoding | LPIPS: 0.370 -> 0.273 | Helped |
| use 3 rather than 4 forward and reverse steps (ablation) | iCD-SD1.5 with both preservation losses · MS-COCO inversion, unguided decoding | LPIPS: 0.198 -> 0.192 | No clear effect |
| use 3 rather than 4 forward and reverse steps (ablation) | iCD-SD1.5 with both preservation losses and dynamic CFG · MS-COCO inversion, guided decoding | LPIPS: 0.273 -> 0.263 | No clear effect |
| add dynamic CFG with τ=0.8 (ablation) | CD⁴-SD1.5 · MS-COCO text-to-image generation | FID-5K: 18.45 -> 16.25 | Helped |
| add dynamic CFG with τ=0.8 (ablation) | CD⁴-SD1.5 · MS-COCO text-to-image generation | ImageReward: 0.044 -> -0.193 | Hurt |
| use dynamic CFG rather than constant CFG during decoding (ablation) | SD1.5 DDIM inversion, CFG scale 8 · MS-COCO inversion | LPIPS: 0.479 -> 0.279 | Helped |
| encode with dynamic CFG rather than constant CFG (ablation) | SD1.5 DDIM inversion · COCO latent quality | FID-5k: 100.5 -> 36.6 | Helped |
Surprising findings
- Encoding with dynamic CFG yields more plausible latents than constant CFG, but unguided encoding (w=1) still gives the highest likelihood and best reconstruction, so the encoder is trained unguided.
- Dynamic CFG improves FID but degrades ImageReward in generation, showing a trade-off with perceived quality.
Limitations
- Boundary time steps must be fixed before distillation, limiting flexibility.
- Preservation losses increase training compute.
- Editing performance is sensitive to hyperparameters and inconsistent across prompts; struggles with faces and hands and can oversaturate at high guidance.
Experimental setup
Datasets: LAION 2B subset MS-COCO 2014
Benchmarks: MS-COCO 2014 PieBench
Metrics: FID CLIP score ImageReward LPIPS DinoV2 similarity PSNR MSE human preference
Baselines: Null-text Inversion Negative-prompt Inversion InfEdit Edit-friendly DDPM ReNoise MasaCTRL SDXL-Turbo SDEdit InstructPix2Pix
Builds on: consistency distillation consistency models multistep sampling DDIM classifier-free guidance CADS dynamic guidance Prompt-to-Prompt
Closest papers by meaning
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.92)
- PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code (ICLR 2024 · similarity 0.90)
- EditInfinity: Image Editing with Binary-Quantized Generative Models (NeurIPS 2025 · similarity 0.89)
- Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis (NeurIPS 2024 · similarity 0.89)
- Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generation (ICLR 2025 · similarity 0.89)
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.