Research map / Vision-language and multimodal / Multimodal LLMs

ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ROVER, a human-annotated benchmark testing reciprocal cross-modal reasoning in unified multimodal models across verbally-augmented visual generation and visually-augmented verbal generation, evaluating 17 models with a GPT-4.1 judge.

Problem

Existing evaluations score unified multimodal models' text and image abilities in isolation, missing whether one modality can guide, verify, or refine outputs in the other.

Approach

A benchmark with two settings, verbal reasoning guiding image generation (ROVER-TG) and interleaved visual intermediates aiding verbal answers (ROVER-IR), scored by a rubric-based GPT-4.1 judge on reasoning process, alignment, visual quality, and answer accuracy.

Key results

• Closed-source Nano Banana reaches overall RV 73.2 on verbally-augmented visual generation, versus 52.7 for the best open-source BAGEL-Think.
• On visually-augmented reasoning, the best overall IR is only 38.8 (Nano Banana), and open-source models lag substantially.
• Visual reasoning improves Qwen2.5-VL-7B by +3.5% on world modeling and +3.8% on visual perception but degrades logic tasks by -1.4%.

Surprising findings

• Flawed visual reasoning performs worse than no visual reasoning; models show minimal or negative accuracy gains on logic and math tasks.
• Combining strong unimodal models (e.g., GPT-4o-refined prompts with FLUX) fails to match the cross-modal reasoning of unified models.

Limitations

• Judging relies on GPT-4.1 VLM-as-judge, with larger discrepancies from human experts on reasoning metrics.
• Human agreement study covers only 4 experts and 200 instances per model across three models.

Experimental setup

Datasets: web image datasets robotics datasets physical simulation videos logic puzzles perception tasks

Benchmarks: ROVER-TG (verbally-augmented visual generation) ROVER-IR (visually-augmented verbal generation) EditWorld

Metrics: Reasoning Process Reasoning Visual Reasoning Alignment Visual Consistency Image Quality Interleaved Reasoning quality Final Answer Accuracy Pearson correlation MAE CLIP-T

Baselines: Nano Banana Gemini 2.0 Flash GPT-5 BAGEL-Think BAGEL UniCoT Step1X-Edit BLIP3o-NEXT Ovis-U1 UniPic2-Metaquery-9B ILLUME+ Emu2-Gen

Builds on: Bagel MetaQuery Zebra-CoT RISEBench

Closest papers by meaning

• RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation (AAAI 2026 · similarity 0.92)
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.92)
• GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models (CVPR 2026 · similarity 0.92)
• DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models (ICML 2026 · similarity 0.92)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.92)
• RBench-V: A Primary Assessment for Visual Reasoning Models with Multimodal Outputs (NeurIPS 2025 · similarity 0.91)