Research map / Image generation / Text-to-image diffusion
DreamOmni2: Multimodal Instruction-based Generation and Editing
Bin Xia, bohao peng, Yuechen Zhang, Junjia Huang, Jiyang Liu, Jingyao Li, Haoru Tan, Sitong Wu, Chengyao Wang, Yitong Wang, Bei Yu, Jiaya Jia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train generation/editing LoRA on the synthetic multimodal data (no VLM) (ablation) | Flux Kontext · Editing, concrete objects | success rate: 0.1220 -> 0.3659 | Improved |
| train generation/editing LoRA on the synthetic multimodal data (no VLM) (ablation) | Flux Kontext · Editing, abstract attributes | success rate: 0.0122 -> 0.3171 | Improved |
| add VLM training only (no editing model training) (ablation) | Flux Kontext · Generation, concrete objects | success rate: 0.3750 -> 0.5417 | Improved |
| add joint training of VLM and editing model over training the editing model alone (ablation) | DreamOmni2 editing · Editing, concrete objects | success rate: 0.3659 -> 0.6585 | Improved |
| add joint training of VLM and editing model over training the editing model alone (ablation) | DreamOmni2 editing · Editing, abstract attributes | success rate: 0.3171 -> 0.6280 | Improved |
| add joint training of VLM and generation model over training the generation model alone (ablation) | DreamOmni2 generation · Generation, abstract attributes | success rate: 0.3444 -> 0.6333 | Improved |
| add joint training over VLM-only training (ablation) | DreamOmni2 editing · Editing, concrete objects | success rate: 0.2439 -> 0.6585 | Improved |
| add index encoding (with position shift) (ablation) | DreamOmni2 · Editing, concrete objects | success rate: 0.4634 -> 0.6585 | Improved |
| add index encoding (without position shift) (ablation) | DreamOmni2 · Editing, concrete objects | success rate: 0.2439 -> 0.3415 | Improved |
| add position encoding shift (with index encoding) (ablation) | DreamOmni2 · Editing, concrete objects | success rate: 0.3415 -> 0.6585 | Improved |
| add position encoding shift (with index encoding) (ablation) | DreamOmni2 · Generation, abstract attributes | success rate: 0.4556 -> 0.6333 | Improved |
| use DreamOmni2 vs Nano Banana for abstract attribute editing | DreamOmni2 · Editing, abstract attributes (human) | success rate: 0.3293 -> 0.6829 | Improved |
Later papers that compared with this method
4 later accepted papers report DreamOmni2 as a baseline in their results tables; 4 of them report a better number on most of the results they share with it.
- Re-Align: Structured Reasoning-guided Alignment for In-Context Image Generation and Editing (CVPR 2026 · better on 7 of 7 shared results)
- Scaling Multi-Reference Image Generation with Dynamic Reward Optimization (ECCV 2026 · better on 2 of 2 shared results)
- OrionEdit: Bridging Reference and Source Images for Generalized Cross-Image Editing (CVPR 2026 · better on 1 of 1 shared result)
- RCEdit-500K: Reference Completion for Image-Conditioned Image Editing (ECCV 2026 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- DreamOmni: Unified Image Generation and Editing (CVPR 2025 · similarity 0.92)
- InsightEdit: Towards Better Instruction Following for Image Editing (CVPR 2025 · similarity 0.90)
- SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models (CVPR 2024 · similarity 0.90)
- Guiding Instruction-based Image Editing via Multimodal Large Language Models (ICLR 2024 · similarity 0.90)
- Text-Vision Co-Instructed Image Editing (NeurIPS 2026 · similarity 0.89)
- OmniGen2: Towards Instruction-Aligned Multimodal Generation (CVPR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
