Research map / Vision-language and multimodal / Multimodal LLMs

RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces RealUnify, a 1,000-instance human-annotated benchmark with direct and stepwise protocols that tests whether unified understanding-generation models synergize, finding current unified models fall short of end-to-end synergy.

Problem

Existing benchmarks evaluate understanding and generation in isolation and do not test whether unified multimodal models gain from bidirectional synergy between the two capabilities.

Approach

A benchmark with two axes (understanding enhances generation and generation enhances understanding) across 10 categories and 32 subtasks, plus a dual protocol: direct end-to-end evaluation and stepwise decomposition into understanding and generation phases, which diagnoses whether failures come from weak core abilities or poor integration.

Key results

• Under direct evaluation the best open-source unified model reaches 37.5% on UEG tasks versus 63.0% for Nano Banana.
• BAGEL improves from 32.7% to 47.7% on UEG under stepwise (understanding-then-generation) evaluation.
• Combining Gemini 2.5 Pro with GPT-Image-1 stepwise reaches 72.7% on UEG tasks, an oracle upper bound.

Surprising findings

• Stepwise decomposition degrades GEU performance for all models, suggesting models rely on understanding shortcuts rather than generation.
• Open-source unified models outperform proprietary models on GEU understanding tasks.

Limitations

• Judge reliability depends on Gemini 2.5 Pro polling, with human agreement checked on a small expert panel.
• Benchmark size is 1,000 instances, with 100 per subtask.

Experimental setup

Datasets: BLINK HR-Bench

Benchmarks: RealUnify

Metrics: accuracy

Baselines: Nano Banana GPT-Image-1 Qwen-Image FLUX.1 Kontext Gemini 2.5 Pro GPT-4.1 Qwen2.5-VL-7B

Builds on: BAGEL thinking with images

Closest papers by meaning

• Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation (ICML 2026 · similarity 0.90)
• Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation (ICML 2026 · similarity 0.89)
• GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models (CVPR 2026 · similarity 0.88)
• Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation (CVPR 2025 · similarity 0.88)
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging (ICML 2025 · similarity 0.88)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.87)