Research map / Image generation / Editing and style transfer

OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision

ICLR 2025 · Image generation › Editing and style transfer · approach: Instruction-based diffusion editing · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds OmniEdit, an instruction-based image editor on SD3 Medium with a new EditNet control-branch architecture, trained on 1.2M specialist-generated, GPT-4o-filtered (via distilled InternVL2) multi-aspect-ratio edit pairs across seven editing tasks.

Problem

Existing end-to-end instruction-based image editing models are trained on biased synthetic data with skewed editing skills, noisy CLIP-filtered pairs, and only square low-resolution images, so they generalize poorly to real-world edits.

Approach

Use seven task-specific specialist models to generate balanced editing pairs, filter them with importance weights from an LMM scorer (GPT-4o distilled into InternVL2-8B) instead of CLIP-score, and train a generalist editor via EditNet, a control branch that replicates each DiT layer and interacts bidirectionally with the main branch and text tokens.

Key results

• On Omni-Edit-Bench, OmniEdit reaches VIEScore (GPT4o) overall 6.98 vs 6.00 for the best baseline CosXL-Edit.
• Human evaluation accuracy of 0.55 vs 0.35 for CosXL-Edit (best baseline).
• Removing importance sampling drops GPT4o overall score from 6.98 to 3.30, and ControlNet-style variants score 4.75-4.89.

Measured effects reported in the paper

Surprising findings

• Channel-wise concatenation fine-tuning of SD3 degraded its original generation ability (e.g., altered Batman representation), motivating the adapter-style design.
• Specialists outperform OmniEdit on their own tasks in some metrics (e.g., Obj-Remove specialist PQ 9.10 vs 8.45 under GPT4o).

Limitations

• Quality is bounded by the SD3 base model, which the authors say limits performance.
• Evaluation uses a small benchmark of 62 images and 434 edits and relies on GPT-4o/Gemini judges plus four human raters.
• Training data is entirely synthetic, generated by specialists that have their own biases.

Experimental setup

Datasets: LAION-5B OpenImages V6 Omni-Edit training set (synthetic) GPT-4o-scored samples

Benchmarks: Omni-Edit-Bench

Metrics: VIEScore PQ VIEScore SC VIEScore overall human PQ human SC human overall human accuracy

Baselines: InstructPix2Pix MagicBrush UltraEdit HQ-Edit CosXL-Edit HIVE DiffEdit SDEdit

Builds on: Stable Diffusion 3 ControlNet InstructPix2Pix VIEScore Prompt-to-Prompt

Closest papers by meaning

• ImgEdit: A Unified Image Editing Dataset and Benchmark (NeurIPS 2025 · similarity 0.88)
• AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea (CVPR 2025 · similarity 0.88)
• Instruct-CLIP: Improving Instruction-Guided Image Editing with Automated Data Refinement Using Contrastive Learning (CVPR 2025 · similarity 0.88)
• UnicEdit-10M: A Dataset and Benchmark Breaking the Scale-Quality Barrier via Unified Verification for Reasoning-Enriched Edits (CVPR 2026 · similarity 0.87)
• UltraEdit: Instruction-based Fine-Grained Image Editing at Scale (NeurIPS 2024 · similarity 0.87)
• HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing (ICLR 2025 · similarity 0.87)