Research map / Image generation / Editing and style transfer
EditInfinity: Image Editing with Binary-Quantized Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts the binary-quantized autoregressive text-to-image model Infinity-2B to text-driven image editing via inversion with exact token supervision (learnable prompt plus LoRA) and token-level blending with a piecewise linear smoothing kernel.
Problem
Diffusion-based image editing via inversion-then-edit suffers from approximation errors during inversion because intermediate generative states lack exact supervision, limiting background preservation and text alignment.
Approach
Exploit the exact multi-scale quantized tokens of a binary-quantized autoregressive model (Infinity) as supervision for inversion: learn a textual embedding (learnable prompt tokens) and LoRA to reconstruct the source tokens, then edit by blending source and target tokens at each scale with a Manhattan-distance piecewise linear mask-smoothing kernel.
Key results
- On PIE-Bench (700 cases), EditInfinity reaches PSNR 27.95, LPIPS 33.08e-3, SSIM 92.12 and IR 5.88, outperforming the best diffusion baselines in background preservation and text alignment.
- Achieves 3.64 s per edit after inversion versus 54.07 s for RF-Edit, though inversion takes 107.06 s.
- In a user study with 60 volunteers, 43.2% preferred EditInfinity, the highest among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adapt Infinity for text-driven image editing | EditInfinity · PIE-Bench | IR$_{10}$: 5.30 -> 5.88 | Helped |
| add a piecewise linear smoothing kernel instead of no smoothing (ablation) | EditInfinity · PIE-Bench random class | IR$_{10}$: 2.85 -> 5.39 | Helped |
| use a linear rather than Gaussian smoothing kernel (ablation) | EditInfinity · PIE-Bench random class | IR$_{10}$: 4.61 -> 5.39 | Helped |
| use a user-provided mask rather than a cross-attention mask (ablation) | EditInfinity · PIE-Bench random class | LPIPS$_{10^3}$: 44.97 -> 31.58 | Helped |
| use a user-provided mask rather than a cross-attention mask (ablation) | EditInfinity · PIE-Bench random class | IR$_{10}$: 5.40 -> 5.39 | No clear effect |
| blend source tokens autoregressively at every scale rather than only after generation (ablation) | EditInfinity · PIE-Bench random class | IR$_{10}$: 3.54 -> 5.39 | Helped |
| apply LoRA to FFN layers rather than attention layers only (ablation) | EditInfinity inversion · PIE-Bench random class | IR$_{10}$: 3.93 -> 5.39 | Helped |
| apply LoRA to FFN layers rather than both FFN and attention layers (ablation) | EditInfinity inversion · PIE-Bench random class | IR$_{10}$: 4.23 -> 5.39 | Helped |
| add a learnable prompt for textual rectification (ablation) | EditInfinity inversion · PIE-Bench image editing | target-prompt alignment | Helped |
| add LoRA for style preservation (ablation) | EditInfinity inversion · PIE-Bench image editing | source style consistency | Helped |
| continue LoRA training beyond 20 iterations (ablation) | EditInfinity inversion · PIE-Bench image editing | editing intent adherence | Hurt |
| remove the learnable prompt and LoRA (ablation) | EditInfinity inversion · PIE-Bench image editing | structural consistency | Hurt |
Surprising findings
- Blending source tokens non-autoregressively (only at scale end) instead of per scale drops PSNR from 28.50 to 25.50, showing per-scale guidance matters.
- Adding LoRA to attention layers in addition to FFN reduces text alignment, so FFN-only LoRA was chosen.
Limitations
- Inversion is slow (about 107 s), and the method struggles with extreme style changes where little background is preserved, causing slight structural degradation.
- Requires user-provided masks by default; automatic cross-attention masks are a secondary option.
Experimental setup
Datasets: PIE-Bench FFHQ MagicBrush
Benchmarks: PIE-Bench GenEval FFHQ MagicBrush
Metrics: PSNR LPIPS MSE SSIM CLIP score (whole) CLIP score (edited) Image Reward ArcFace CLIP-I
Baselines: P2P MasaCtrl P2P-Zero NTI PnP-Inv NP StableFlow RF-Edit Gemini 2.0
Builds on: BSQ LoRA PIE-Bench
Closest papers by meaning
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.91)
- PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code (ICLR 2024 · similarity 0.91)
- Object-Aware Inversion and Reassembly for Image Editing (ICLR 2024 · similarity 0.91)
- Textualize Visual Prompt for Image Editing via Diffusion Bridge (AAAI 2025 · similarity 0.90)
- SplitFlow: Flow Decomposition for Inversion-Free Text-to-Image Editing (NeurIPS 2025 · similarity 0.89)
- Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.