Research map / Vision-language and multimodal / Multimodal LLMs
ADIEE: Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluation
Sherry X. Chen, Yi Wei, Luowei Zhou, Suren Kumar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add score token and MLP decoder (ablation) | ADIEE scorer · ImagenHub | Spearman correlation with human: 0.2931 -> 0.3450 | Improved |
| add score token and MLP decoder (ablation) | ADIEE scorer · AURORA-Bench (point-wise) | Spearman correlation with human: 0.3558 -> 0.4744 | Improved |
| add score token and MLP decoder (ablation) | ADIEE scorer · GenAI-Bench | pairwise accuracy: 58.32 -> 59.41 | Improved |
| add score token and MLP decoder (ablation) | ADIEE scorer · AURORA-Bench (pair-wise) | pairwise accuracy: 20.50 -> 52.38 | Improved |
| add reward conditioning prompts when fine-tuning MagicBrush on scored data (ablation) | MagicBrush · ImagenHub (MagicBrush editing) | avg score by ADIEE scorer: 4.27 -> 6.67 | Improved |
| train reward-conditioned model on all samples instead of only high-score samples (ablation) | MagicBrush · ImagenHub (MagicBrush editing) | avg score by ADIEE scorer: 5.09 -> 6.67 | Improved |
| fine-tune MagicBrush with ADIEE reward conditioning | MagicBrush · ImagenHub | avg score by ADIEE scorer: 5.90 -> 6.43 | Improved |
| fine-tune MagicBrush with reward feedback learning | MagicBrush · ImagenHub | avg score by ADIEE scorer: 5.90 -> 6.30 | Improved |
| reward conditioning on MagicBrush, CLIP-I | MagicBrush · MagicBrush editing | CLIP-I: 95.231 -> 95.215 | No clear effect |
| reward conditioning on MagicBrush, CLIP-D | MagicBrush · MagicBrush editing | CLIP-D: 0.13604 -> 0.13279 | Hurt |
| reward conditioning on MagicBrush, DINO-I | MagicBrush · MagicBrush editing | DINO-I: 90.643 -> 89.935 | Hurt |
| reward conditioning on MagicBrush, GPT-4o score | MagicBrush · MagicBrush editing | GPT-4o score: 4.02 -> 4.36 | Improved |
Closest papers by meaning
- EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editing (ICLR 2026 · similarity 0.88)
- I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing (NeurIPS 2024 · similarity 0.88)
- ImgEdit: A Unified Image Editing Dataset and Benchmark (NeurIPS 2025 · similarity 0.88)
- VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark (NeurIPS 2024 · similarity 0.88)
- HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing (ICLR 2025 · similarity 0.87)
- InsightEdit: Towards Better Instruction Following for Image Editing (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
