Research map / Vision-language and multimodal / Multimodal LLMs

VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VisuLogic, a 1,000-question vision-centric visual reasoning benchmark across six categories, shows most MLLMs score near chance, and shows RLOO reinforcement learning on a 4,296-item supplementary set improves Qwen2.5-VL-7B and InternVL2.5-38B.

Problem

Existing multimodal reasoning benchmarks can be solved through text descriptions of images and mostly do not measure vision-centric reasoning, so MLLMs' genuine visual reasoning is hard to assess.

Approach

A benchmark of 1,000 single-choice visual logic problems designed so that textual descriptions of the image lose critical cues, paired with an auxiliary training set and rule-based RLOO fine-tuning to show RL gains on visual reasoning.

Key results

• Most evaluated MLLMs score below 30% on VisuLogic versus 51.4% for human participants and 25% random baseline; best closed-source MLLM Doubao-1.5-Vision-Pro-32k reaches 28.1%.
• InternVL2.5-38B-RL reaches 31.1% overall, up from 25.5% for the base model, and surpasses its SFT control (27.9%).
• Hint prompts raise Claude-3.7-Sonnet, Gemini-2.0-Pro, and Doubao-1.5-Vision-Pro-32k by over 8 points, reaching above 35%, while CoT prompting changes most models by less than 1 point.

Surprising findings

• Chain-of-thought prompting yields minimal gains on visual reasoning, with most models changing by under 1 point.
• Text-only LLMs given GPT-4o-generated image descriptions score near chance (e.g., Qwen2.5-72B-Instruct at 28.0%).

Limitations

• Human baseline uses 100 graduate students each solving 10 questions, a limited sample.
• RL experiments are small-scale (80 to 100 steps) and only two models are trained.
• Benchmark is single-choice with four options, and most models score close to the 25% chance level.

Experimental setup

Datasets: VisuLogic supplementary training set

Benchmarks: VisuLogic

Metrics: accuracy

Baselines: Human Random Deepseek-R1 Qwen2.5-72B-Instruct QwQ-32B GPT-4o Doubao-1.5-Vision-Pro-32k Gemini-2.0-Pro Claude-3.7-Sonnet InternVL3-78B Qwen2.5-VL-7B-Instruct-SFT InternVL2.5-38B-SFT

Builds on: RLOO DeepSeek-R1 VLM-R1

Closest papers by meaning

• VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs (CVPR 2026 · similarity 0.93)
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.93)
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.93)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.92)
• MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods (NeurIPS 2026 · similarity 0.91)
• VGR: Visual Grounded Reasoning (ICLR 2026 · similarity 0.91)