Research map / Vision-language and multimodal / Multimodal LLMs

Understanding vs. Generation: Navigating Optimization Dilemma in Multimodal Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes R3 (Reason-Reflect-Refine), which turns text-to-image generation into a generate-understand-regenerate loop trained with tree-structured GRPO/FlowGRPO on BAGEL, reporting gains in both generation and self-evaluated understanding.

Problem

Unified multimodal models trade off generation and understanding: optimizing one tends to degrade the other, and naive co-training yields only negligible gains.

Approach

Recasts single-shot generation as an iterative chain (reason -> generate draft -> reflect on alignment with the prompt -> refine via editing instruction) so the model's own understanding is used inside generation, with an outcome-based reward and a tree-based RL rollout that supervises intermediate stages.

Key results

• On GenEval++ (GPT-4.1 judge), BAGEL + R3 reaches 0.689 overall vs. 0.371 for BAGEL and 0.679 for Echo-4o.
• On the proposed VQA benchmark, BAGEL + R3 improves overall accuracy from 86.48 to 89.63, with Count rising from 79.30 to 84.63.
• On the proposed ITA benchmark, overall accuracy rises from 60.60 to 73.37; on TIIF testmini, overall score is 82.02 vs. 70.97 for BAGEL.

Surprising findings

• Reasoning-only inference with the trained model gains little on understanding (ITA +1.16, VQA +0.24), while the reflect-refine stages drive most of the understanding gain.
• Cross-topic training shows understanding improvements are localized to the trained attribute category rather than generalizing broadly.

Limitations

• Understanding gains appear domain-specific: training on one attribute category does not improve other categories.
• Self-termination can accept images with residual errors (e.g., spelling mistakes in text rendering).
• Relies on large external VLM reward (Qwen-2.5-VL-72B) and GPT-4.1 evaluation, and understanding benchmarks use Gemini-2.5-Flash ground truth, so evaluation depends on proprietary models.

Experimental setup

Datasets: GenEval GenEval++ TIIF

Benchmarks: GenEval GenEval++ TIIF ITA (proposed) VQA (proposed)

Metrics: accuracy overall score

Baselines: BAGEL Echo-4o FLUX.1-dev FLUX.1-Kontext Janus-Pro T2I-R1 GPT-4o Qwen-Image-RL FlowGRPO BAGEL w/ self-CoT

Builds on: BAGEL GRPO FlowGRPO MixGRPO GoT-R1 T2I-R1

Closest papers by meaning

• WeGen: A Unified Model for Interactive Multimodal Generation as We Chat (CVPR 2025 · similarity 0.88)
• DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.87)
• Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation (ICML 2026 · similarity 0.87)
• Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models (CVPR 2026 · similarity 0.87)
• Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners (ICML 2026 · similarity 0.86)
• Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation (CVPR 2026 · similarity 0.86)