Research map / Vision-language and multimodal / Multimodal LLMs

MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MindOmni, a unified VLM-plus-diffusion model trained in three stages (connector pretraining, CoT SFT, and RGPO RL with format and CLIP consistency rewards plus dual KL regularizers) to enable reasoning-driven image generation.

Problem

Text-to-image systems lack multimodal input handling and explicit reasoning for knowledge-, logic-, or spatiotemporal-dependent generation, and unified MLLMs rarely exploit their LLM reasoning for generation.

Approach

Builds a unified model from Qwen2.5-VL and OmniGen with a connector to a decoder-only diffusion module, then applies GRPO-style RL (RGPO) with multimodal rewards (format plus CLIP-based consistency of generated image vs. ground-truth prompt) and two separate KL regularizers for text and visual rollouts.

Key results

• MindOmni reaches 0.71 overall on WISE in thinking mode, versus 0.70 for BAGEL and 0.55 for MetaQuery-XL, per the WISE table.
• MindOmni achieves 89.7 overall on GenEval (0.83 for the higher-quality-data variant), with 83.0 DPG-Bench overall.
• Ablation: skipping Stage 2 CoT SFT and running RGPO alone yields WISE 0.49 versus 0.60 with all stages.

Measured effects reported in the paper

Surprising findings

• Merely increasing the length of output thinking content does not improve end results, according to the paper.
• RGPO alone without CoT SFT lowers GenEval to 0.72 from 0.81 with the full pipeline.

Limitations

• Chain-of-thought content is plain text only; multimodal CoT is left for future work.
• Reported GenEval/WISE numbers are inconsistent across tables and text in the draft (e.g., WISE reported as 71 and 0.71 and as 59.8 in ablations).

Experimental setup

Datasets: Laion COCO JourneyDB X2I dataset GenEval WISE

Benchmarks: GenEval DPG-Bench WISE MMMU MMBench RealworldQA

Metrics: overall score accuracy

Baselines: OmniGen Janus-Pro MetaQuery-XL BAGEL GPT-4o Gemini-2.5 FLUX SDXL

Builds on: GRPO DeepSeek-R1 OmniGen Qwen2.5-VL

Closest papers by meaning

• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.92)
• DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models (ICML 2026 · similarity 0.91)
• Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.90)
• Toward Cognitive Supersensing in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.90)
• Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models (NeurIPS 2025 · similarity 0.90)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.90)