Research map / Vision-language and multimodal / Multimodal LLMs
KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces KRIS-Bench, a knowledge-grounded benchmark of 1,267 image-editing instances across 22 tasks and 7 reasoning dimensions (factual, conceptual, procedural), scoring 10 editing models with VLM judges plus a new Knowledge Plausibility metric.
Problem
Instruction-based image editing models produce visually plausible outputs, but their knowledge-based reasoning ability is under-evaluated by existing benchmarks, which are coarse or task-based.
Approach
Organizes editing tasks by knowledge type drawn from the revised Bloom taxonomy (factual, conceptual, procedural), and adds a Knowledge Plausibility metric scored by a VLM using manually written knowledge hints, calibrated against human ratings.
Key results
- GPT-4o achieves the highest overall average score of 80.09 among evaluated models (GPT-Image-1 / GPT-4o) under GPT-4o judging.
- BAGEL-Think is the best open-source model with overall average 53.36, versus 47.76 for BAGEL without reasoning.
- Knowledge Plausibility scores are consistently lower than Instruction Following scores across models; for Logical Reasoning, GPT-4o reaches 51.00 KP versus 59.25 IF.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a reasoning process to BAGEL (ablation) | BAGEL · KRIS-Bench | overall average: 47.76 -> 53.36 | Helped |
| include knowledge hints in evaluation prompts | VLM evaluation protocol · User study on three closed-source models | Pearson correlation and MAE | Helped |
| use BAGEL-Think instead of Gemini 2.0 or Doubao | image editing models · KRIS-Bench Biology and Chemistry tasks | Knowledge Plausibility | Helped |
Surprising findings
- Models do not consistently score worse on conceptual than factual knowledge; several models (GPT-4o, BAGEL, Step1X-Edit, AnyEdit) score slightly lower on factual tasks.
- Open-source BAGEL-Think outperforms Gemini 2.0 and Doubao on Biology and Chemistry Knowledge Plausibility.
Limitations
- Relatively small benchmark scale
- Potential over-representation of certain knowledge types
- Culturally specific assumptions in task design
Experimental setup
Datasets: Amazon-Berkeley Objects (ABO) video object segmentation datasets virtual try-on datasets
Benchmarks: KRIS-Bench
Metrics: Visual Consistency Visual Quality Instruction Following Knowledge Plausibility
Baselines: GPT-4o Gemini 2.0 Flash Doubao OmniGen Emu2 BAGEL Step1X-Edit AnyEdit MagicBrush InstructPix2Pix
Builds on: RISEBench revised Bloom taxonomy GPT-4o judge
Closest papers by meaning
- Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing (NeurIPS 2025 · similarity 0.91)
- ReasonEdit: Towards Reasoning-Enhanced Image Editing Models (CVPR 2026 · similarity 0.91)
- UnicEdit-10M: A Dataset and Benchmark Breaking the Scale-Quality Barrier via Unified Verification for Reasoning-Enriched Edits (CVPR 2026 · similarity 0.91)
- Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning (ICML 2026 · similarity 0.90)
- GIR-Bench: Versatile Benchmark for Generating Images with Reasoning (ICLR 2026 · similarity 0.90)
- Learning Action and Reasoning-Centric Image Editing from Videos and Simulation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.