Research map / Vision-language and multimodal / Multimodal LLMs

UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces UniM, a 31K-instance benchmark for any-to-any interleaved multimodal understanding and generation across 7 modalities and 30 domains, with a three-dimensional evaluation suite and an agentic baseline UniMA.

Problem

Existing interleaved multimodal benchmarks cover only text and image and use single-capability instances, so they do not evaluate any-to-any interleaved understanding and generation across diverse modalities and domains.

Approach

A benchmark of 31,026 QA instances with interleaved text, image, audio, video, document, code and 3D inputs/outputs, scored by a new suite (SQCS, StS/LeS, ICS, supporting rate), plus UniMA, an agentic baseline with a Traceable Evidence Reasoning module that generates, checks and backtracks over a structured report before generating interleaved outputs.

Key results

• UniMA reaches SQCS around 57-73% across fields and StS/LeS 2-6x higher than AnyGPT, versus baselines mostly below 20% SQCS.
• Pearson correlation with human evaluation is r=0.974 for SQCS and r=0.960 for ICS.
• Ablation: removing TER drops StS from 52.7 to 16.4 and LeS from 82.6 to 21.8.

Surprising findings

• Baselines show uniformly low scores across difficulty levels, so they do not distinguish task complexity.
• Social science gives the best SQCS while natural science, despite being expected to be structured, is weakest on semantic alignment.

Limitations

• UniMA invokes up to six external tools, adding system complexity and computational overhead.
• UniMA's performance depends on the capabilities of the external tools it calls.
• The benchmark covers only seven modalities and a restricted set of modality combinations.

Experimental setup

Datasets: Video-MME HowTo100M MMCode ScreenSpot-Pro CodeEditorBench FoodieQA MedTrinity-25M ESC DeepFashion

Benchmarks: UniM

Metrics: SC GQ SQCS StS LeS HC SH ICS supporting rate

Baselines: AnyGPT NExT-GPT MIO

Builds on: Chain-of-Thought

Closest papers by meaning

• MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models (ICLR 2026 · similarity 0.90)
• UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation (NeurIPS 2025 · similarity 0.89)
• MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models (ICLR 2025 · similarity 0.89)
• OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.89)
• WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation (CVPR 2026 · similarity 0.89)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.89)