Research map / Vision-language and multimodal / Multimodal LLMs

Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a self-adaptive interleaved reasoner that trains a unified model (Emu3.5) to switch among direct generation, self-reflection, and multi-step planning for X2I tasks, using a 50k hierarchical dataset, SFT, and GRPO with step-wise reasoning and intra-group complexity rewards.

Problem

Unified multimodal models understand user intent but fail to translate it into precise pixel-level generation; complex prompts cause blind planning (attention entanglement) and unstructured feedback fails to refine images (visual refinement bottleneck) in anything-to-image tasks.

Approach

Adaptive switching among three execution modes (direct, reflect, multi-step plan) learned from a hierarchical data pipeline that escalates instructions through Analyzer/Generator loops, with selective loss masking in SFT and GRPO rewards that include step-wise reasoning validity and an intra-group penalty favoring fewer generated images.

Key results

• Achieves 80.18 overall on KRIS-Bench, above Emu3.5 (73.75) and the closed-source GPT4o (80.09 close to parity, best among listed open models).
• Reaches 9.35 average on OmniContext versus 8.82 for Emu3.5 and 8.80 for GPT-4o.
• GenEval overall 0.89 versus 0.86 for Emu3.5; RL cuts average generated images from 2.45 (SFT only) to 1.56.

Surprising findings

• Removing the intra-group complexity penalty slightly raises KRIS-Bench and OmniContext scores (80.25, 9.38) but increases average image count to 2.73, showing a fidelity-efficiency trade-off.

Limitations

• Training and inference rely on an internal proprietary distributed infrastructure, and compute scale is not reported.
• Data pipeline depends on proprietary models (Gemini-3-Pro-Image) and a 235B judge, limiting reproducibility.
• Reported judge-based metrics use an LMM evaluator that may introduce bias.

Experimental setup

Datasets: UnicEdit-10M X2Edit AnyEdit Pick-a-Pic UltraEdit

Benchmarks: GenEval KRIS-Bench OmniContext

Metrics: GenEval overall score KRIS-Bench factual/conceptual/procedural/overall OmniContext average score average generated images

Baselines: Emu3.5 Plan-then-Generate Generate-then-Reflect VACoT Uni-CoT BAGEL BAGEL-thinking Qwen-Image-Edit-2509 ReasonEdit-Q

Builds on: GRPO Emu3.5 chain-of-thought

Closest papers by meaning

• Latent Implicit Visual Reasoning (CVPR 2026 · similarity 0.90)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.90)
• Generative Universal Verifier as Multimodal Meta-Reasoner (ICLR 2026 · similarity 0.90)
• DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models (ICML 2026 · similarity 0.90)
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.89)
• Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models (CVPR 2026 · similarity 0.89)