Research map / Image generation / Editing and style transfer
Beyond Simple Edits: X-Planner for Complex Instruction-Based Image Editing
Chun-Hsiao Yeh, Yilin Wang, Nanxuan Zhao, Richard Zhang, Yuheng Li, Yi Ma, Krishna Kumar Singh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add X-Planner decomposition, masks and boxes | UltraEdit · COMPIE-Eval | MLLMti: 0.6652 -> 0.7061 | Improved |
| add X-Planner decomposition, masks and boxes | InstructPix2Pix* · COMPIE-Eval | MLLMti: 0.6727 -> 0.7408 | Improved |
| add masks and boxes on top of decomposed instructions (ablation) | X-Planner + UltraEdit · COMPIE-Eval | L1: 0.1253 -> 0.1188 | Improved |
| add X-Planner on top of UltraEdit | UltraEdit · COMPIE-Eval | CLIPout: 0.2698 -> 0.2569 | Hurt |
| replace GPT-4o training data with open-source Pixtral-Large data (ablation) | X-Planner · COMPIE-Eval | MLLMti: 0.7061 -> 0.7102 | No clear effect |
| use X-Planner predicted masks and boxes instead of human masks | UltraEdit · MagicBrush single-turn | L1: 0.0575 -> 0.0513 | Improved |
| add bounding boxes to X-Planner masks (ablation) | X-Planner + UltraEdit · MagicBrush single-turn | L1: 0.0528 -> 0.0513 | Improved |
| fine-tune X-Planner as a decomposing planner versus GLaMM+Llama3 anchoring (ablation) | mask generation · PIE benchmark | IoU: 0.44 -> 0.67 | Improved |
| add pseudo-labeling for insertion box prediction (ablation) | X-Planner bbox · MULAN (K=1) | IoU: 0.37 -> 0.63 | Improved |
| add box enlargement (ablation) | X-Planner bbox · MULAN (K=1) | IoU: 0.63 -> 0.75 | Improved |
| add combined mask and box input (ablation) | X-Planner bbox · MULAN (K=1) | IoU: 0.75 -> 0.73 | No clear effect |
| add closed-loop MLLM verification with GPT-4o, max=1 retry (ablation) | X-Planner + UltraEdit · COMPIE-Eval | MLLMti: 0.7061 -> 0.7113 | Improved |
Closest papers by meaning
- Guiding Instruction-based Image Editing via Multimodal Large Language Models (ICLR 2024 · similarity 0.90)
- SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models (CVPR 2024 · similarity 0.89)
- ReasonEdit: Towards Reasoning-Enhanced Image Editing Models (CVPR 2026 · similarity 0.89)
- AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea (CVPR 2025 · similarity 0.88)
- Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning (ICML 2026 · similarity 0.88)
- InsightEdit: Towards Better Instruction Following for Image Editing (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
