Research map / Image generation / Editing and style transfer
Visual Instruction Inversion: Image Editing via Image Prompting
Thao Nguyen, Yuheng Li, Utkarsh Ojha, Yong Jae Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CLIP loss to MSE loss (caption init, random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Visual CLIP similarity: 0.254 -> 0.299 | Improved |
| add CLIP loss to MSE loss (caption init, random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Directional CLIP similarity: 0.115 -> 0.126 | Improved |
| add CLIP loss to MSE loss (caption init, random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Image CLIP similarity: 0.845 -> 0.823 | Hurt |
| add CLIP loss (ground-truth init, random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Visual CLIP similarity: 0.247 -> 0.231 | Hurt |
| initialize instruction from caption of after image instead of ground-truth instruction (MSE+CLIP, random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Visual CLIP similarity: 0.231 -> 0.299 | Improved |
| initialize from caption instead of ground-truth instruction (MSE+CLIP, random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Directional CLIP similarity: 0.131 -> 0.126 | Mixed |
| use caption directly as InstructPix2Pix instruction without training (ablation) | InstructPix2Pix · Clean-InstructPix2Pix | Directional CLIP similarity: 0.126 -> 0.090 | Hurt |
| reuse training noise at test time (full method) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Image CLIP similarity: 0.823 -> 0.847 | Improved |
| reuse training noise at test time (full method) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Visual CLIP similarity: 0.299 -> 0.251 | Hurt |
| reuse training noise at test time (full method) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Directional CLIP similarity: 0.126 -> 0.113 | Hurt |
| increase training pairs from 1 to 4 (random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Directional CLIP similarity: 0.126 -> 0.148 | Improved |
| increase training pairs from 1 to 4 (random noise) (ablation) | Visual Instruction Inversion · Clean-InstructPix2Pix | Image CLIP similarity: 0.823 -> 0.798 | Hurt |
Closest papers by meaning
- Textualize Visual Prompt for Image Editing via Diffusion Bridge (AAAI 2025 · similarity 0.94)
- Text-Vision Co-Instructed Image Editing (NeurIPS 2026 · similarity 0.91)
- EditInfinity: Image Editing with Binary-Quantized Generative Models (NeurIPS 2025 · similarity 0.90)
- An Item Is Worth a Prompt: Versatile Image Editing with Disentangled Control (AAAI 2025 · similarity 0.90)
- RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers (NeurIPS 2025 · similarity 0.90)
- Prompt Augmentation for Self-supervised Text-guided Image Manipulation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
