Research map / Image generation / Editing and style transfer
AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds AnyEdit, a 2.5M-pair instruction image-editing dataset across 25 editing types with an adaptive pipeline and filtering, plus AnySD (SD1.5 with task-aware MoE routing and learnable task embeddings) and the AnyEdit-Test benchmark.
Problem
Instruction-based image editing models struggle with complex instructions because existing training datasets are low quality and cover few editing types.
Approach
Categorize editing into 25 types across five groups, build a large dataset with type-specific adaptive editing pipelines, counterfactual synthetic scenes to balance concepts, and multi-stage quality filtering; then train AnySD with visual prompt projector, task-aware MoE routing and learnable task embeddings.
Key results
- On EMU-Edit test, AnySD trained on AnyEdit reaches CLIPim 0.872 and CLIPout 0.285, versus 0.859 and 0.231 for EMU-Edit.
- On MagicBrush test, AnySD reaches DINO 0.881 versus 0.879 for EMU-Edit, setting new records on both benchmarks.
- AnyEdit-Test shows UltraEdit drops 19.2% in DINO on common tasks relative to standard benchmarks, and removing task-aware routing raises L1 from 0.070 to 0.154 on EMU-Edit test.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Stable Diffusion 1.5 with AnyEdit instead of MagicBrush data | Stable Diffusion 1.5 · EMU-Edit Test | CLIPim: 0.838 -> 0.866 | Helped |
| Add task-aware routing (ablation) | AnySD · EMU-Edit Test | CLIPim: 0.838 -> 0.872 | Helped |
| Add task embeddings (ablation) | AnySD · EMU-Edit Test | L1: 0.107 -> 0.070 | Helped |
| Add AnyEdit-Composition data (ablation) | AnySD · EMU-Edit Test | CLIPout: 0.271 -> 0.285 | Helped |
Surprising findings
- Existing SOTA models show notable consistency drops on AnyEdit-Test even for common editing tasks, suggesting standard benchmarks are limited.
- Removing task embeddings barely affects semantic alignment but substantially hurts visual consistency.
Limitations
- Benchmark AnyEdit-Test is small, with 50 curated pairs per type.
- Relies on LLM and T2I generators whose biases may propagate into the data.
- Data quality filtering depends on automatic metrics such as CLIP and DINO.
Experimental setup
Datasets: AnyEdit MSCOCO LLaVA-CC3M-Pretrain MVImgNet
Benchmarks: EMU-Edit test MagicBrush test AnyEdit-Test
Metrics: CLIPim CLIPout L1 DINO SSIM CLIPin
Baselines: PnP Null-Text InstructPix2Pix MagicBrush HIVE EMU-Edit UltraEdit Uni-controlnet
Builds on: InstructPix2Pix Stable Diffusion 1.5 MagicBrush UltraEdit
Closest papers by meaning
- InsightEdit: Towards Better Instruction Following for Image Editing (CVPR 2025 · similarity 0.93)
- UltraEdit: Instruction-based Fine-Grained Image Editing at Scale (NeurIPS 2024 · similarity 0.92)
- SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models (CVPR 2024 · similarity 0.91)
- Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer (NeurIPS 2025 · similarity 0.90)
- ImgEdit: A Unified Image Editing Dataset and Benchmark (NeurIPS 2025 · similarity 0.90)
- ReasonEdit: Towards Reasoning-Enhanced Image Editing Models (CVPR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.