Research map / Image generation / Editing and style transfer
ImgEdit: A Unified Image Editing Dataset and Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ImgEdit, a 1.2M-pair image-editing dataset (incl. 110k multi-turn), the ImgEdit-E1 editing model (Qwen2.5-VL-7B + SigLIP + FLUX DiT), the ImgEdit-Bench benchmark, and ImgEdit-Judge.
Problem
Open-source image-editing models lag proprietary ones due to limited high-quality editing data and benchmarks that lack difficulty stratification, multi-turn support, and reliable metrics.
Approach
A multi-stage automated pipeline (VLM captioning, open-vocabulary detection, SAM2 segmentation, FLUX/SDXL inpainting workflows, GPT-4o filtering) builds high-resolution single- and multi-turn edit pairs, paired with a VLM-conditioned FLUX editor and a GPT-4o-based tiered benchmark with a fine-tuned judge.
Key results
- ImgEdit dataset averages 4.71 GPT score vs 4.55 for HQ-Edit (best prior) and fake score 0.050, with Tiny-ROE ratio 0.8%.
- GPT-4o-Image scores 4.70 UGE score vs ImgEdit-E1 3.20 and Step1X-Edit 3.11 among the evaluated models; ImgEdit-E1 is best among open-source on attribute alteration (4.04) and style transfer (4.38).
- ImgEdit-Judge agrees with human judgments in almost 70% of cases on a 60-image study (within one point).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train ImgEdit-E1 on ImgEdit | ImgEdit-E1 · ImgEdit-Bench, across single-turn tasks | editing score | Helped |
| Use ImgEdit-E1 instead of Step1X-Edit | open-source editing models · ImgEdit-Bench, Addition, Replacement, Motion Change, Style Transfer, Object Extraction, Hybrid Edit, and UGE | score: 3.82 vs 3.90; 2.80 vs 3.45; 3.21 vs 3.43; 4.38 vs 4.44; 2.55 vs 1.87; 2.87 vs 2.52; 3.20 vs 3.11 | Mixed |
| Use ImgEdit-E1 instead of open-source baselines | open-source editing models · ImgEdit-Bench, Object Extraction | score: 2.55 vs 2.02 | Helped |
| Use ImgEdit-E1 instead of open-source baselines | open-source editing models · ImgEdit-Bench, Hybrid Edit | score: 2.87 vs 2.52 | Helped |
| Use GPT-4o-Image instead of ImgEdit-E1 | image editing models · ImgEdit-Bench, Addition, Removement, Replacement, Attribute Alter, Motion Change, Style Transfer, Background Change, Object Extraction, Hybrid Edit, and UGE | score: 4.65 vs 3.82; 3.81 vs 2.40; 4.49 vs 2.80; 4.26 vs 4.04; 4.76 vs 3.21; 4.75 vs 4.38; 4.62 vs 3.38; 2.96 vs 2.55; 4.54 vs | Helped |
| Add ImgEdit-Judge fine-tuning on 200k rating records | Qwen2.5-VL-7B evaluator · human judgment agreement | agreement: almost $70\%$ | Helped |
Surprising findings
- GPT-4o-Image scores only 2.96 on object extraction, and the paper attributes this to scarce high-quality extraction data, including for GPT-4o.
- All evaluated models receive fake scores near 1.0, so forensic detectors easily identify their edits.
Limitations
- ImgEdit-E1 was not subjected to detailed ablations of architecture, data, or training.
- Multi-turn benchmark is small (10 images per task, 3 turns) and multi-turn scores are human yes/no judgments.
- ImgEdit-E1 is weak at text generation in images, a limitation shared by open-source models.
Experimental setup
Datasets: LAION-Aesthetics Open-Sora Plan in-house videos ImgEdit
Benchmarks: ImgEdit-Bench
Metrics: instruction adherence score editing quality score detail preservation score UGE score fake score (FakeShield recall) human agreement rate
Baselines: GPT-4o-Image Step1X-Edit UltraEdit AnySD MagicBrush InstructPix2Pix
Builds on: FLUX Qwen2.5-VL SigLIP Step1X-Edit ControlNet
Closest papers by meaning
- UltraEdit: Instruction-based Fine-Grained Image Editing at Scale (NeurIPS 2024 · similarity 0.91)
- AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea (CVPR 2025 · similarity 0.90)
- I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing (NeurIPS 2024 · similarity 0.90)
- InsightEdit: Towards Better Instruction Following for Image Editing (CVPR 2025 · similarity 0.90)
- EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editing (ICLR 2026 · similarity 0.90)
- HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.