Research map / Image generation / Editing and style transfer
The Promise of RL for Autoregressive Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compares SFT, RL (GRPO with a Qwen2.5-VL-72B verifier), and chain-of-thought supervision for text-guided image editing on the autoregressive Emu3-8B, and releases EARL, which outperforms prior AR editors and nearly matches Omnigen.
Problem
Text-guided image editing, especially complex edits (counting, spatial, action), often fails; existing editors rely on diffusion models and SFT, and autoregressive editors are underexplored.
Approach
Use a fully autoregressive unified multimodal model (Emu3-8B) and apply online GRPO after SFT, with a large MLLM (Qwen2.5-VL-72B) as a VIEScore-style reward verifier; also systematically compare SFT data mixes and chain-of-thought supervision in one framework.
Key results
- EARL (SFT S then RL S+C, scaled) achieves average VIEScore 4.80 across six benchmarks vs 4.70 for Omnigen and 3.88 for SFT (S).
- RL on SFT (S) improves average from 3.88 to 4.57 (RL S+C at 1.6K sample pool), while adding complex data during SFT lowers average to 3.32.
- Chain-of-thought SFT (think S) reduces average to 3.50 from 3.88, and RL on it reaches only 3.68.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply RL (S+C) after SFT (S) | Emu3 image editor · AVG | VIEScore: 3.88 -> 4.57 | Helped |
| Scale RL (S+C) training to 2000 steps using a 32K-sample training run | SFT (S) → RL (S+C) · AVG | VIEScore: 4.57 -> 4.80 | Helped |
| Include complex data in SFT alongside simple data (ablation) | Emu3 image editor · AVG | VIEScore: 3.88 -> 3.32 | Hurt |
| Train SFT in two stages: simple data, then complex data (ablation) | Emu3 image editor · AVG | VIEScore: 3.88 -> 3.69 | Hurt |
| Use RL (S) instead of RL (S+C) (ablation) | SFT (S) checkpoint · AVG | VIEScore: 4.57 -> 4.06 | Hurt |
| Use RL (C) instead of RL (S+C) (ablation) | SFT (S) checkpoint · AVG | VIEScore: 4.57 -> 4.30 | Hurt |
| Use RL (C) rather than RL (S) after SFT (S) (ablation) | Emu3 image editor · Complex edit benchmarks | VIEScore | Helped |
| Use SFT (S+C) rather than SFT (S) before RL (S+C) (ablation) | Emu3 image editor · AVG | VIEScore: 4.57 -> 4.49 | Hurt |
| Replace Qwen2.5-VL-72B verifier with Qwen-7B (ablation) | SFT (S) → RL (S+C) · AVG | VIEScore: 3.88 -> 2.81 | Hurt |
| Add chain-of-thought supervision to SFT on simple data (ablation) | Emu3 image editor · AVG | VIEScore: 3.88 -> 3.50 | Hurt |
| Add complex reasoning data in two-stage CoT SFT (ablation) | SFT think (S) · AVG | VIEScore: 3.50 -> 1.52 | Hurt |
| Apply RL (S) after CoT-supervised SFT (ablation) | SFT think (S) · AVG | VIEScore: 3.50 -> 3.68 | Helped |
Surprising findings
- Adding complex edit data during SFT hurts performance on both simple and complex benchmarks, while adding it during RL helps.
- Chain-of-thought reasoning does not improve editing and sometimes hurts, with the model producing plausible plans but poor final edits.
Limitations
- Verifier (Qwen2.5-VL-72B) is unreliable on large count changes and fine-grained spatial edits, and its correlation with humans is moderate (about 0.40).
- Base Emu3 is weaker than diffusion baselines (GenEval 0.64 vs 0.70 for Omnigen) and was not pretrained on editing or interleaved image-text-image data.
- Smaller verifiers (Qwen-7B) cause RL to degrade performance.
Experimental setup
Datasets: OmniEdit HumanEdit MagicBrush VisMin Aurora-Kubric Aurora-ActionGenome SomethingSomething-v2
Benchmarks: OmniEdit EmuEdit AURORA-Bench MagicBrush VisMin I2EBench
Metrics: VIEScore
Baselines: MagicBrush InstructPix2Pix Aurora EditAR Omnigen
Builds on: Emu3 GRPO VIEScore
Closest papers by meaning
- Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing (NeurIPS 2026 · similarity 0.90)
- EditAR: Unified Conditional Generation with Autoregressive Models (CVPR 2025 · similarity 0.89)
- NEP: Autoregressive Image Editing via Next Editing Token Prediction (NeurIPS 2025 · similarity 0.89)
- RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward (CVPR 2026 · similarity 0.89)
- Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO (NeurIPS 2025 · similarity 0.88)
- GoT-R1: Unleashing Reasoning Capability of Autoregressive Visual Generation with Reinforcement Learning (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.