Research map / Vision-language and multimodal / Multimodal LLMs

Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Unison, a 2,169-sample benchmark evaluating unified multimodal models on four synergy dimensions (internal consistency, understanding-guided generation, generation-guided understanding, mutual enhancement), plus a fine-tuned judge Unison-Judge.

Problem

Existing evaluations assess understanding and generation of unified multimodal models in isolation, missing the synergy between comprehension and generation.

Approach

A benchmark decomposing unified capability into four synergy dimensions with both unified and decoupled scoring tracks, plus an MLLM judge (Unison-Judge, fine-tuned from Qwen3-VL-8B) aligned with human judgments.

Key results

• BAGEL achieves the highest open-source overall score of 53.2, versus 71.3 for GPT-5.2 and 69.8 for Gemini 3 Pro among closed-source models.
• Unison-Judge reaches average human alignment 0.887 (Spearman), above Gemini 3 Pro (0.884) and GPT-5.2 (0.869).
• OmniGen2 scores 51.3 overall and 47.7 unified in mutual enhancement, the best open-source result on that dimension.

Surprising findings

• Understanding scores typically exceed generation scores, and high comprehension or generation ability does not guarantee good internal consistency (e.g., D-DiT beats TokenFlow on unified consistency despite lower understanding).
• Imperfectly generated images can sometimes improve understanding by providing contrastive boundaries.

Limitations

• Iterative mutual-enhancement evaluation is computationally expensive and hard to scale to high-resolution or video.
• Scores may be sensitive to prompt engineering and generative stochasticity.
• Judge alignment validated on only 231 samples.

Experimental setup

Datasets: RefCOCOg LAION-Aesthetics GenEval-style captions DSG-generated VQA

Benchmarks: Unison

Metrics: accuracy IoU normalized editing rating human alignment (Spearman rho)

Baselines: Show-o Show-o2 Janus-Pro D-DiT ILLUME+ OmniGen2 TokenFlow BAGEL SEED-X UniWorld Gemini 3 Pro GPT-5.2

Builds on: GenEval ImgEdit Davidsonian Scene Graph

Closest papers by meaning

• MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models (ICLR 2026 · similarity 0.90)
• RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark (CVPR 2026 · similarity 0.89)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.88)
• HEMM: Holistic Evaluation of Multimodal Foundation Models (NeurIPS 2024 · similarity 0.87)
• OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.87)
• UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark (CVPR 2026 · similarity 0.87)