Research map / Vision-language and multimodal / Multimodal LLMs

Turning Internal Gap into Self-Improvement: Promoting the Generation-Understanding Unification in MLLMs

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows unified MLLMs have a generation-understanding gap mostly due to weak generation, and proposes label-free internal-gap self-improvement (SFT/DPO scored by the model's own understanding branch) plus curriculum learning, with a learning-dynamics (eNTK) explanation of co-improvement.

Problem

Unified MLLMs are supposed to unify generation and understanding but show an internal gap where understanding outperforms generation, so generated images are often rejected by the model's own understanding branch (non-unification).

Approach

Use the model's own understanding branch to score its generations and build chosen/rejected image data for SFT/DPO on the generation side, with no external reward models; extend learning-dynamics theory with a shared empirical NTK to explain why understanding also improves (co-improvement), and add curriculum learning to reuse previously discarded prompts.

Key results

• Self-improved Janus-Pro-7B with SFT improves generation by up to ~20% on T2I-CompBench++ subtasks and reduces non-unification score by up to ~16%.
• Non-unification scores reach up to ~58% for VILA-U on the hard/medium subtasks, with weak generation accounting for over 50% to 100% of misalignments.
• About 80% of understanding gains come from false positive correction; curriculum SFT (C-SFT) further improves Janus-Pro and Show-o and surpasses T2I-R1 and HermesFlow on several subtasks.

Surprising findings

• Understanding gains from self-improvement are mostly (about 80%) false positive corrections, i.e., generation training improves the model's judging of misaligned images.
• Non-unification score does not reflect absolute performance: models with larger internal gaps (e.g., BAGEL, BLIP3-o) can be stronger overall.

Limitations

• Evaluated only on a limited set of unified MLLMs (mainly Janus-Pro and Show-o); validation on models like BAGEL is left to future work.
• The shared-eNTK explanation is supported by similarity proxies rather than direct eNTK computation, and the reason NTK sharing arises is not explained.
• Weak generation relies on Qwen2.5-VL-72B judgments; human-Qwen discrepancy grows on hard tasks (up to ~20%).

Experimental setup

Datasets: T2I-CompBench++ GenEval Science-T2I

Benchmarks: T2I-CompBench++ GenEval Science-T2I

Metrics: generation score understanding win rate non-unification score weak generation rate

Baselines: T2I-R1 HermesFlow SDv1.5 Janus-Pro-7B Show-o

Builds on: DPO learning dynamics theory Janus-Pro Show-o T2I-CompBench++

Closest papers by meaning

• Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models (ICLR 2025 · similarity 0.88)
• Progress or Regress? Self-Improvement Reversal in Post-training (ICLR 2025 · similarity 0.87)
• B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners (ICLR 2025 · similarity 0.86)
• The Future of Facts: Tracing the Factual Generation-Verification Gap (NeurIPS 2026 · similarity 0.86)
• Learning to Self-Verify Makes Language Models Better Reasoners (ICML 2026 · similarity 0.85)
• A Task-centric Theory for Iterative Self-Improvement with Easy-to-Hard Curricula (ICML 2026 · similarity 0.85)